EP0315673A1

EP0315673A1 - Process and device for temporal interpolation of images

Info

Publication number: EP0315673A1
Application number: EP88905206A
Authority: EP
Inventors: Philippe Robert
Original assignee: Thomson Grand Public
Current assignee: Thomson Grand Public
Priority date: 1987-06-04
Filing date: 1988-05-31
Publication date: 1989-05-17
Also published as: FR2616248B1; JP3161467B2; JPH01503830A; FR2616248A1; WO1988010046A1; EP0294282B1; ATE91580T1; DE3882285T2; ES2041816T3; DE3882285D1; EP0294282A1

Abstract

The invention concerns a process for reconstituting the luminance values of a field (Tj) from the luminance values of the pixels of two known fields (Ta, Tb). It consists essentially in estimating a velocity vector (V) for each pixel (6) of a single field (Ti) corresponding to an instant Ti between Ta and Tb, or being Ta or Tb; determining successively for each pixel (G) of this image (Ti), called the pixel father, the co-ordinate of the centre (Fj) of a pixel to be interpolated (H), called the pixel son, corresponding to the centre (Cj) of the pixel father by a translation of which the vector is a function of the velocity vector (V) of the pixel father multiplied by the time Ti-Tj; associating successively with each pixel to be interpolated (H) a velocity vector constituted by the velocity vector (V) of its pixel father (G); determining in the two known fields (Ta, Tb) the co-ordinates and then the luminance of two points (A, B) corresponding to the centre (Fj) of the pixel to be interpolated (H) by a translation, in the plane of one of the known images (Ta, Tb), of which translation the vector is a function of the velocity vector associated with the pixel to be interpolated multiplied respectively by the durations Tj-Ta and Tb-Tj; determining a luminance valve for the pixel to be interpolated (H) by calculating a linear combination of the luminance values of these two points (A, B). Application to changes in the standard of television images and to the reconstitution of missing images in the low-output transmission of images with the elimination of certain images.

Description

Method and device for temporal interpolation of images.

The invention relates to a method of temporal interpolation of images making it possible to reconstruct the luminance values of the pixels of a missing image in a series of images representing the same object. Such a method can be used, for example, for transmission of video images at very low bit rate, consisting in transmitting only certain frames, with coding reducing the information bit rate, and in restoring the frames not transmitted, by interpolation. from the transmitted and decoded frames. It can also be used for converting a series of video images from a standard having a frequency of 50 Hz to a standard having a frequency of 60 Hz, or vice versa. For such a conversion, most of the frames must be interpolated from the frames available at the frequency of 50 Hz, since the instants of shooting at the frequency of 60 Hz do not coincide, with the instants of shooting at the frequency of 50 Hz.

The purpose of the interpolation is therefore to determine a luminance value L (IX, IY, T _j ) for each pixel of a frame to be interpolated corresponding to an instant T _j , from the luminance values L (IX, IY, T _a ) and L (IX, IY, T _b ) of the pixels of two known frames corresponding to instants T _a and T _b such that T _j . is between T _a and T _b . The pixels of the frames are identified by the coordinates (IX, IY) of their center in an ortho-normalized reference which is common for all the frames.

A first class of temporal interpolation methods is based on a simulated linear interpolation. pie consisting in calculating, for each pixel to be interpolated, a linear combination of two luminance values of the pixels having homologous coordinates (IX, IY) in the two known frames corresponding to the instants T _a and T _b , weighted by the durations T _b -T _j and T _j -T _a , according to the formula:

L (IX, IY, T _j ). L (IX, IY, T _a ) +. L (IX, IY, T.)

The homologous coordinates are exactly identical if the two frames T _a and T _b have the same parity, and are identical to the nearest half-line if they have different parities.

This class of methods allows good restitution of the fixed zones in a series of frames, because then the luminance values used for the interpolation effectively correspond to the same point of the object represented by the series of frames. On the other hand, the moving areas are poorly reproduced and are all the more blurred as the speed of the movement is higher. A second class of processes takes into account the movement of the object represented by the two known frames. These methods consist in supposing that each point of a moving object represented by the sequence of frames moves from the first known frame to the second known frame by an elementary translation at constant speed. Known methods make it possible to determine a speed vector Y = (VX, VY) for each pixel of the frame to be interpolated. This vector translates an elementary translation of the pixel, which does not necessarily correspond to the speed of the object represented but corresponds to the variations in luminance of the pixels representing this object. These second class methods then consist in calculating an interpolated luminance value for each pixel, taking into account two displacements of opposite directions respectively in the two known frames. These two displacements are in the direction of the speed vector associated with the pixel to be interpolated, and have respectively a modulus proportional to the time interval separating the frame to be interpolated and the known frame considered. Each interpolated luminance value is therefore calculated according to the formula:

L (IX, IY, T _j ) =

L (IX-VX. (T _j -T _a ), IY-VY. (T _j -T _a ), T _a )

L (IX + VX. (T _b -T _j ), IY + VY. (T _b -T _j ), T _b )

This class of processes makes it possible to reproduce more precisely the moving areas, as well as the fixed areas of the frames to be interpolated. However, the implementation of this class of methods involves an important difficulty which is the determination of a speed vector to be associated with each pixel to be interpolated. A first known process, described by H.C. Bergman in: Motion Adaptive Frame Interpolation 1984 International

Zurich Seminar on Digital Communications, consists in associating with each pixel to interpolate the speed vector estimated for the pixel having the same coordinates in the first known frame. This association is always more or less inaccurate since in moving areas these two pixels do not represent the same point on the moving object. This association is only exact insofar as neighboring points of the moving object have the same velocity vector, which is not necessarily the case, especially if the object shown is rotated. The fidelity of image restitution therefore varies, in moving areas, depending on the type of movement of these areas. A second known method, described by M. Bierling and R. Thoma in: Motion compensating field interpolation using a hierachically structured displacement estimator, consists in estimating a speed vector for each pixel of each frame to be interpolated, independently for each frame to be interpolated. This estimate is made from the same pair of known images, for all the frames to be interpolated corresponding to instants between T _a and T _b , but it takes account of the duration of the time intervals separating the frame to be interpolated and the known images, therefore the speed vector associated with each pixel to be interpolated has a much more precise value than that associated by the method described above. The fidelity of restitution of the frames in the moving areas is therefore made independent of the type of movement. On the other hand, this second method has the drawback of multiplying the computation time necessary for the estimation of the speed vectors, since it is proportional to the number of frames to be interpolated comprised between the two known frames.

The object of the invention is to propose a temporal interpolation method providing good fidelity of restitution of the frames in the moving areas, whatever the type of movement, without requiring a significant computation time for the estimation of the vectors. pixel speeds of the frames to be interpolated.

The object of the invention is a method belonging to the second class and which essentially consists in estimating a speed vector for each pixel of a single frame which is called motion-carrying frame, which can be one of the two known frames, and then to determine, in each of the frames to be interpolated, the pixels known as child pixels, which have a speed vector equal to the speed vector of a pixel called parent pixel corresponding to the same point of the object represented by the sequence of frames, this point being assumed to move with the same speed vector during the time interval from T _a to T _b . Naturally, the method according to the invention is also applicable to images which are not interlaced frames.

According to the invention, a method of temporal interpolation of images, for determining a value of interpolated luminance for each pixel of an image called image to interpolate, from the luminance values of the pixels of a first and of d 'a second image called known images, taken at times T _a and T _b respectively, T _j being between T _a and T _b ; consisting in: associating the values of the components of a speed vector with each pixel of the image to be interpolated, said pixel to be interpolated;

- determine in the two known images the coordinates of two points called base points of the temporal interpolation, each of these two points corresponding to the center of the pixel to be interpolated, by a translation in the plane of one of the known images, translation whose vector is a function of the speed vector associated with the pixel to be interpolated, multiplied respectively by the durations T _j -T _a and T _b -T _j ;

- determining the luminance value of each of the two base points by spatial interpolation between pixels neighboring each of these base points, respectively in the two known images; - determining a luminance value for the pixel to be interpolated, by calculating a linear combination of the two luminance values of the base points, 'weighted by the durations T _j -T _a and T _b -T _j respectively; is characterized in that, to associate the components of a speed vector with each pixel to be interpolated, it consists in:

- Estimate by a conventional process the components of a speed vector for each pixel of a single image, called the carrier of the movement, corresponding to an instant T _i . between T _a and T _b , or being T _a or

T _b ;

- successively determining for each pixel of the carrier image, known as the father pixel, the coordinates of the center of a pixel to be interpolated, known as the child pixel, corresponding to the center of the father pixel by a translation whose vector is a function of the speed vector of the parent pixel, multiplied by the duration T _i -T _j ; - successively associating each pixel to be interpolated with a speed vector consisting of the speed vector of its parent pixel or one of its parent pixels, if there is at least one parent pixel for this pixel to be interpolated. The invention will be better understood and other details will appear with the aid of the description below and the accompanying figures:

- Figure 1 illustrates the principle of the second class of known methods; - Figure 2 illustrates the method according to the invention;

- Figure 3 shows a flowchart illustrating the method according to the invention;

- Figures 4 and 7 show two organi grams illustrating a first variant implementation of the method according to the invention;

- Figure 5 shows a part of this flowchart; - Figure 6 illustrates the calculation performed in accordance with the flow diagram of Figure 5;

- Figures 8 and 9 show a flow diagram of a second variant of the implementation of the method according to the invention; - Figures 10 and 11 show a flow diagram of a third variant of the implementation of the method according to the invention;

- Figures 12 and 15 show the block diagram of an exemplary embodiment of a device for implementing the method according to the invention;

- Figures 13, 14, 16 show a more detailed block diagram of three parts of this embodiment.

In the example shown in FIG. 1, a frame to be interpolated, called frame T _j because it corresponds to an instant T _j , is to be interpolated from a first and a second known frame called respectively frame T _a and frame T _b because they correspond to an instant Ta which precedes T _j and to an instant T _b which follows T _j , All these frames are identified in the same ortho-normed reference frame PXY having the dimensions of a pixel as units. Points having identical (X, Y) coordinates are differentiated by the instants corresponding to the frames where these points are located. In what follows, we will distinguish points and pixels, each grid point has infinitely small dimensions while each pixel has finite dimensions determined by the characteristics of the sampling. The known frames are supplied by a source of video signals such as than a conventional television camera followed by a conventional analog-to-digital converter. Each pixel is identified by the coordinates of its center and is represented by a dotted square. Each has a luminance value equal to the average of the luminances of all the points contained in this pixel. By assumption, each pixel to be interpolated, centered on a point M _j in the frame to be interpolated T _j , has an associated speed vector There then exist in the known frames T _a and T _b two points M _a and M _b having coordinates which are homologous to those of M _j and which are each the center of a pixel represented by a dotted square.

The first class of known methods consists in calculating the luminance value of the pixel to be interpolated, by linearly combining the luminance values of the pixels centered on M _b and on M _a . There is an error due to the fact that in the frame T _b the object represented is displaced, with respect to the frame T _j , in the direction, and in the direction of the vector while. in the frame T _a on the contrary, the object represented is moved, in the direction and in the direction opposite to the vector The second class of known methods consists in calculating an interpolated luminance value from the luminance values of a point B of the frame T _b and a point A of the frame T _a , determined by taking into account a translation of the object in the frames T _a and T _b , with respect to the frame T _j . The point A is deduced from M _a per a translation - (T _j -T _a ) and point B is deduced from

M _b by a translation: (T _b -T _j ), because any point of an object represented by the sequence of frames is supposed to keep a constant speed between the instants T _a and T _b .

Points A and B are called, in what follows, basic points of the temporal interpolation. The method according to the invention is based on the same assumption. FIG. 2 illustrates the step of the method according to the invention consisting in associating a speed vector to a pixel to be interpolated, centered on a point F _j of the frame to be interpolated T _j . In this example, there is only one frame T _j to be interpolated, but in other cases there may be several frames to be interpolated, corresponding to times between T _a and T _b . For all the frames to be interpolated between T _a and T _b , the associated speed vectors are deduced from the speed vectors of the pixels of a single frame T _i , called the frame carrying the motion, corresponding to an instant between T _a and T _b or can be T _a or T _b . The calculations necessary to determine the speed vectors of the pixels of the carrier frame are carried out only once regardless of the number of frames to be interpolated between the instants T _a and

T _b .

Each pixel of the frame T _i carrying motion is considered to be the father of a single pixel, called the child pixel, in the frame to be interpolated T _j , these two pixels being supposed to represent the same point of the moving object. This point of the moving object has a speed vector which is known in the frame T _i . This point is supposed to keep this same speed vector at all times between T _a and T _b while it is represented by a series of pixels having different coordinates and which are all the children of the same pixel father of the frame T _i . The method according to the invention consists in determining in each frame to interpolate T _j , the center of a child pixel corresponding to a father pixel given in the frame T _i carrying the movement, then in assigning to this child pixel an equal speed vector at the velocity vector of its parent pixel, or of one of its parent pixels. For example, in FIG. 2, a pixel G centered on the point C _i of the frame T _i has for vector speed a vector known. It has a child pixel H in the frame T _j , which corresponds approximately to it by a vector translation (T _i -T _j ) from a point C _j having, in the frame T _j , coordinates homologous to those of the point C _i in the frame T _i . This translation makes a point E _j correspond exactly to point C _j . In general, the point E _j does not coincide with the center of a pixel. The method then consists in taking as pixel son the pixel of the frame T _i having a center F _j closest to the point E _j , by taking the nearest integer value for each of the coordinates. The speed vector of the father pixel G is then associated with this child pixel H.

It should be noted that certain pixels of the frame T _i carrying the movement do not have child pixels, for example the pixels located at the edge of the carrier frame do not have child pixels when the speed vector is directed towards the inside of the carrier frame. It should also be noted that a pixel of the frame to be interpolated T _j can be the child pixel of several parent pixels simultaneously. For example, if two parent pixels have respectively two non-collinear velocity vectors, it may happen that the two translation vectors corresponding to these velocity vectors converge in the same child pixel. These particular cases are treated differently according to three alternative embodiments of the method according to the invention, which will be described later.

In a particular case, the carrier frame T _i considered is the frame T _a or the frame T _b . If, for example, the carrier frame considered is the frame T _b , the estimation of the speed vectors of its pixels can consist of calculating, by a known process, the elementary translation undergone for each point of the object between the screen T _a and the frame T _b , deducing therefrom an elementary translation vector (DX, DY) for each pixel of the frame T _b , then calculate a velocity vector for each of these pixels, according to the formula: - (3) FIG. 3 represents a flow diagram of the essential steps of the method according to the invention, whatever the variant of implementation which is considered. This flowchart represents the steps of the time interpolation method of a single frame to be interpolated, corresponding to the instant T _j . For each frame to be interpolated also included in the interval (T _a , T _b ) the same steps are to be carried out. FIG. 3 does not mention the step of determining the speed vectors of the Dixels of the Dorteuse frame T _i because it is a step which is carried out only once for all the frames to be interpolated situated between the two known traces corresponding to the instants T _a and T _b . The frame for which the estimation of the speed vectors is carried out can be the frame T _a or the frame T _b or a frame whose luminance values are unknown but which corresponds to an instant T _i between T _a and T _b . For example, it can be constituted by one of the frames to be interpolated, the main thing being to use a speed estimation method making it possible to obtain an exact value of the components of the speed vector for each pixel of this frame. Many methods of speed estimation can be used. Those carrying out the determination by block of pixels or pixel by pixel are more suitable than those carrying out the determination by extraction of the characteristics of the frame, because they allow to obtain a denser velocity vector field and therefore greater precision for each of the pixels. For example, it is possible to apply the speed estimation method described by: NETRAVALL and ROBBINS in US patent 4,383,272.

As mentioned in FIG. 3, the method according to the invention comprises two main phases. A first phase consists in scanning the pixels of the carrier frame T _i and in determining a child pixel, if there is one, in the frame to be interpolated T _j , for each pixel of the carrier frame. When all the pixels of the carrier frame have been scanned, a second phase then consists in successively scanning all the pixels of the frame to be interpolated T _j ; to associate a speed vector to each pixel of this frame to be interpolated, as a function of the speed vector of the father pixel or of the parent pixels corresponding to the pixel to be interpolated considered, if there is at least one father pixel; determining, for each pixel to be interpolated, two base points of the temporal interpolation, in the two known frames T _a and T _b ; determining the luminance of these two base points by spatial interpolation in the two known frames T _a and T _b , when the base points are not the pixel centers of these known frames; and finally to determine an interpolated luminance value, for the pixel to be interpolated considered, by linearly combining the luminance values of the two base points in accordance with formula (2).

FIG. 4 represents a flowchart of the first phase of a first variant implementation of the method according to the invention. In this variant, the case where a pixel to be interpolated has several parent pixels is treated by selecting the parent pixel which is the best. The criterion is the value of a function called The DFD which is equal to the absolute value of the difference in the luminance values of the two base points A and B. The selection of the best parent pixel consists in comparing the values of the DFD function obtained by associating with the pixel to be interpolated the velocity vector of each of the parent pixels successively. The parent pixel considered to be the best, is the one whose speed vector gives a minimum value to the DFD function. On the other hand, according to this first variant, the case of an orphan pixel, that is to say having no father pixel in the frame carrying the movement, is treated by determining a speed vector interpolated from the vectors velocity associated with the neighboring pixels in the frame to be entered, insofar as they are not orphans. The value of the DFD function is used as a criterion to detect pixels which are orphaned. In the case where it is not possible either to associate a speed vector this variant consists in associating a zero speed vector.

The first phase of the first variant begins with an initialization of a memory M2 intended to store the minimum value of the DFD function determined for each pixel of the frame to be interpolated T _j . it is initialized with a constant value equal to 255 which corresponds to the maximum value of the luminance of the frames and therefore also of the DFD. The series of frames considered is in fact constituted by a series of interlaced frames, each frame being considered as an independent frame. Each frame comprises a number M of pixels per line and a number N of lines. Memory M2 has a capacity of MxN bytes.

Then the IY coordinate is initialized to zero. Then it is incremented by one and the coordinate IX is initialized to zero. Then the IX coordinate is incremented by one. The coordinates (IX, IY) designate the center C _i of a pixel of the frame T _i carrying the movement, and which can be a parent pixel. Then the vector IX, IY, T _i ), which is the speed vector of this pixel, is read in a memory MI at an address corresponding to the coordinates of this pixel. The memory MI is a memory having a capacity of M × N cells and which is loaded beforehand with the values of the components of the speed vector of all the pixels of the motion-carrying frame used for the interpolation of the missing frames in the interval T _a , T _b .

Then the coordinates of a point E _j corresponding to point C _j are determined according to the formula: = _d - V (IX, IY). (T _i -T _j ) (4)

Then the coordinates of the center F _j of a child pixel are determined by taking the integer part of the components of the vector: (0,5; 0,5) where (0,5; 0,5) makes it possible to find the integer value closest to the components, and not only the whole part of these; and or is a frame parity correction vector, having as components: (0.5; 0.5) if the frames T _i and T _j are frames having the same parity;

(0.5; 0) if the frame T _i is an odd frame and the frame T _j is an even frame;

(0.5; 1) if the frame T _i is an even frame and if the frame T _j is a pictorial frame.

The pixel to be interpolated centered on the point F _j is a child pixel of the pixel of coordinate (IX, IY) in the frame T _i . The method then consists in determining the coordinates of two basic interpolation points tem porelle, B and A, in the frames T _b and T _a respectively, according to the formulas:

In which the vector is a vector of frame parity correction, having as components:

(0; 0) if the frame T _a and the frame T _j are frames having the same parity;

(0; 0.5) if the frame T _a is an odd frame and if the frame T _j is an even frame;

(0; -0.5) if the frame T _a is an even frame and if the frame T _j is an odd frame.

The vector has analogous components as a function of the parity of the frame constituting the screen T _j and the parity of the frame constituting the frame T _b .

The first phase of the first variant then consists in calculating the luminance values,

L ( , T _a ) and L ( T _b ) at points A and B according to a calculation method which will be detailed later and which consists of a spatial interpolation combining the luminance values of four pixels closest to point A, respectively of point B, when these points do not correspond not at the pixel centers of frame T _a and frame T _b . Then it consists in calculating the value of the DFD function, which is equal to the absolute value of the difference in luminances, at points A and B determined for this speed vector V.

Then the value of the DFD function is compared with a fixed threshold value equal to 7 in this example. If the DFD function is less than the threshold value, this means that the coordinate pixel (IX, IY) in the carrier frame constitutes a parent pixel for the pixel to be interpolated centered on the point F _j . The process consists then to read in the memory M2 a value DFD 'at an address corresponding to the components of the vector _j . If the value DFD which has just been found for the parent pixel with coordinate (IX, IY) in the frame T _i , is less than the value DFD 'read in the memory M2, this means that the parent pixel which comes to be found is better because its speed vector makes the value of the DFD function smaller. Its speed vector is therefore chosen to be associated with the child pixel centered on the point F _j . This speed vector is written into a memory M1 at an address corresponding to the child pixel, that is to say corresponding to the components of the vector. The memory M1 is a memory of MxN cells intended to store the value of the components of an associated speed vector, for each pixel to be interpolated. In addition, the value of the DFD function which has just been calculated is written in the memory M2 at the address corresponding to the child pixel, in place of the previous DFD 'value which corresponds to another parent pixel or which is equal to the initialization value.

It should be noted that in the case where the child pixel did not already have one or more other parent pixels, the value of the DFD function read from the memory M2 is equal to the initialization value of the memory M2, c ' i.e. 255.

In the case where the calculated value of the DFD function is greater than 7, then the coordinate pixel (IX, IY) in the frame T _i is not considered as a parent pixel, and the first phase continues with a test of the value reached by the coordinate IX.

If the value of the calculated DFD function is greater than the value DFD 'read in the memory M2, this means that the pixel of coordinate (IX, IY) in the frame T _i is a father pixel which is not better than a father pixel found previously and having supplied the value DFD 'which has just been read in the memory M2, it is therefore to be rejected. The first phase continues with the test on the IX coordinate.

If the coordinate IX has not reached its maximum value which is M, the preceding operations are repeated from the incrementation of IX. If IX has reached its maximum value, the first phase continues with a test relating to the IY coordinate. If the latter has not reached its maximum value N, the preceding operations are repeated from the increment of IY, until IY reaches N. The first phase, of the first variant, is then completed, and then begins the second phase of this first variant.

FIG. 5 represents a flow diagram of the operations for calculating the luminance values L ( t, T _a ) and of

L ( PB, T _b ) by spatial interpolation in the frames T _a and T _b resoectively. Figure 6 illustrates this calculation method for the luminance value L , T _a ). In general, the point A does not coincide with the center of a pixel of the frame T _a but is located between the centers G1, G2, G3, G4 of four pixels. The spatial interpolation of the luminance value at point A firstly consists in determining the integer part of the components of the vector to constitute the coordinates (XG _a , YG _a ) of the point G1. The coordinates of the point G2 are then (XG _a + 1, YG _a ). The coordinates of point G3 are then (XG _a , YG _a +1). The coordinates of point G4 are then (XG _a + 1, YG _a +1). In this example of implementation, the point G4 constitutes the reference point for the interpolation. The method then consists in determining coordinates (ΔX _a , ΔY _a ) for point A, relative to the reference point G4. These relative coordinates are given by the formula (ΔX _a , ΔY _a ) = (XG _a , YG _a ) + (1.1) - (7)

The method then consists in reading from the memory MA luminance values: L (XG _a , YG _a ),

L (XG _a + 1, YG _a ), L (XG _a , YG _a +1), L (XG _a + 1, YG _a +1), corresponding to the pixels centered on the points G1 to G4, at corresponding addresses at the coordinates of these points G1 to G4. Then the luminance value L ( , T _a ) is calculated according to the formula:

The F, T _a ) = L (XG _a + 1, YG _a +1). (1-ΔX _a ) (1-ΔY _a ) + L (XG _a , YG _a +1) .ΔX _a . (1-ΔY _a )

+ L (XG _a + 1, YG _a ). (1-ΔX _a ) .ΔY _a (8)

+ L (XGa.YG _a ) .ΔX _a .ΔY _a . + 0.5

The calculation of the luminance value L ( , T _b ) is carried out according to a sequence of operations quite similar.

FIG. 7 represents the flowchart of the second phase of the first variant of the implementation of the method according to the invention. This second phase begins with an initialization at zero of a coordinate IY ', then an incrementation of one unit of this coordinate and an initialization at zero of a second coordinate IX'. The coordinates (IX ', IY') are the coordinates of the center of a pixel of the frame to be interpolated Tj. Then the method consists in incrementing the coordinate IX 'and in reading the value DFD (IX', IY ') stored in the memory M2 at an address corresponding to the coordinates (IX', IY '). The value read is compared with a threshold value equal to the value 255 having served to initialize the memory M2 and which is equal to the maximum value of the luminance.

If the value read is not equal to 255, this means that a value of the DFD function was previously calculated for this pixel to be interpolated, because it is the child pixel of a pixel of the motion-carrying frame. ment T _i . A speed vector was therefore associated with it previously, during the first phase of the process. This speed vector is read in the memory M1 at the address corresponding to the coordinates (IX ', IY') and is then used to calculate the vectors and according to equations (5) and (6). These vectors and determine two base points A and B for the temporal interpolation. They are used to calculate the luminance values L , T _a ) and L (, T _b ) by an interpolation spatial in the frames T _a and T _b , in accordance with the flow diagram represented in FIG. 5 and described previously.

If the value of the DFD function read from the memory M2 remained equal to the initialization value, 255, this means that the first phase of the process has never detected the pixel of coordinates (IX ', IY') as a pixel son of a father pixel located in the carrier frame of the movement T _i . The pixel to be interpolated, with coordinates (IX ', IY') is therefore an orphan pixel and it is necessary to associate a speed vector with it by another means. According to this first variant, a speed vector is calculated by interpolating between the speed vectors of four pixels neighboring the orphan pixel, insofar as they are not themselves orphan pixels. This computation is carried out by reading in the memory M2 the values of the function DFD and by reading in the memory M1 the components of the speed vector for the four neighboring points whose coordinates are: (IX ', IY'-1), (IX'-1, IY'), (IX '+ 1, IY'), (IX ', IY' + 1)

Then the value of the interpolated speed vector is calculated according to the formula:

V IX ', IY') = (A ₀₀ . V (IX ', IY') +

At ₀₁ . IX'-1, IY ') + A ₁₀ .V (IX' + 1, IY ') + A ₁₁ V (IX ', IY')) with A ₀₀ = 0 if DFD (IX ', IY'-1) = 255

A ₀₀ = 1 otherwise.

A ₀₁ : 0 if DFD (IX'-1, IY ') = 255

A ₀₁ = 1 otherwise. A ₁₀ = 0 if DFD (IX '+ 1, IY') = 255

A ₁₀ = 1 otherwise.

A ₁₁ = 0 if DFD (IX ', IY' + 1) = 255

A ₁₁ = 1 otherwise.

And V (IX ', IY') = (0,0) if A ₀₀ = A ₀₁ = A ₁₀ = A ₁₁ : 0, that is to say if the 4 neighboring pixels are also orphan pixels.

Vectors and are then calculated according to formulas (5) and (6) in which the vector consists of the interpolated vector Then the luminance values L ( A, T _a ) and L ( , T _b ) are then determined by a spatial interpolation in the frames T _a and T _b , in accordance with the flow diagram of FIG. 5 described above. Then 1 = value of the DFD function is calculated for the base points A and B thus determined.

If this value of the DFD function is less than the threshold value, 7, this means that the speed vector interpolated S constitutes a suitable estimate of the speed vector of the orphan pixel considered. It can therefore be associated with this orphan pixel. The method then consists in calculating the value of the interpolated luminance of the orphan pixel considered, L (IX ', IY', T _j ) according to formula (9).

If the value of the DFD function is greater than the threshold value, this means that the interpolated speed vector is not suitable and cannot be associated with the orphan pixel considered. As a last resort, a zero speed vector is associated with the orphan pixel. The two basic points of the temporal interpolation, A and B, then have the same coordinates (IX ', IY') as the center of the pixel to be interpolated, therefore the vectors and are both equal and have components (IX ', IY'). The luminance values L ( 1 _a ) and L ( T _b ) are then determined without spatial interpolation in the frames T _a and T _b , since IX 'and IY' are integer values. They are read in the memories MA and MB respectively at the addresses corresponding to (IX ', IY'). Then the interpolated luminance value

L (IX ', IY', T _j ) is calculated according to the formula

The _interpolated =

Whenever an interpolated luminance value L (IX ', IY', T _j ) has been calculated, this value is supplied, for example, to a memory of the frame storing the interpolated values of the frame T _j in awaiting use of these luminance values. Then the value of the coordinate IX 'is compared with its maximum value M to determine whether it has reached this maximum value. If it has not reached it, the previous operations are repeated from the increment of IX '. If it has reached its maximum value, the value of the second coordinate IY 'is compared with its maximum value N. If it has not reached its maximum value, the preceding operations are repeated from the incrementation of IY'. If it has reached its maximum value, this means that all the pixels to be interpolated have been scanned and that, consequently, the temporal interpolation processing of the frame T _j is finished.

FIG. 8 represents a flowchart of the first phase of the method according to the invention, according to a second variant of implementation, making it possible to simplify the calculations and to delete the memory M2 because this variant does not use the DFD function. The first pha begins with an initialization of the memory M1 with the value (255, 255) which constitutes the maximum value of the components of the speed vector associated with each pixel to be interpolated. In addition, the ordinate IY of the pixels of the frame carrying the movement T _i is initialized to zero. Then this ordinate IY is incremented by one, and the abscissa IX of the pixels of the frame carrying the movement T _i is initialized to zero. Then this abscissa is incremented by one. The value of the components of the speed vector IX, IY, T _i ) is read in the memory MI then is used to calculate the components of the vector and the components of the vector _j . Then this speed vector is written in the memory M1 at the address corresponding to the components of the vector _j , that is to say at the address corresponding to the coordinates of the child pixel.

Then the value of the coordinate IX is compared with its maximum value M. If this is not reached the preceding operations are repeated from the incrementation of IX. If it is reached, the value of the IY coordinate is compared with its maximum value N. If this is reached, the first phase is ended and a second phase begins. If IY has not reached its maximum value N, the preceding operations are repeated from the increment of IY.

FIG. 9 represents a flowchart of the second phase of the second variant of implementation of the method according to the invention. This second phase begins with an initialization at zero of the coordinate IY 'of the pixels to be interpolated. Then it consists of incrementing the coordinate IY 'and initializing the coordinate IX' to zero. Then it consists in incrementing the coordinate IX ′ by one and reading the speed vector V (IX ′, IY ′, T _j ) in the memory M1 at an address corresponding to the coordinates (IX ′, IY ′). The components of this speed vector are each compared with the value 255 which is the initial value loaded into the memory M1. If the components of the speed vector are equal to their initial value, this means that no speed vector has been associated with the pixel of coordinates (IX ', IY'). This is therefore an orphan pixel and it is necessary to determine in another way the speed vector to be associated with this pixel. An interpolated velocity vector T is calculated by an interpola spatial map from the speed vectors of four neighboring pixels of the orphan pixel. The speed vectors of these four neighboring pixels are read from memory M1 and then are combined linearly in accordance with equation (9) but taking: A ₀₀ = 0 if (IX'-1, IY ') = (255,255)

A ₀₀ = 1 otherwise.

At ₀₁ : 0 if (IX'-1, IY ') = (255,255)

A01 = 1 otherwise.

A ₁₀ = 0 if IX '+ 1, IY') = (255,255) A10 = 1 otherwise.

A ₁₁ = 0 if VIX ', IY' + 1) = (255,255)

A ₁₁ = 1 otherwise.

And 'IX', IY ') = (0,0) if A ₀₀ = A ₀₁ = A ₁₀ = A ₁₁ = 0, that is to say if the four neighboring pixels are also orphan pixels.

It should be noted that the absence of use of the DFD function does not prevent detection of whether a pixel to be interpolated is an orphan pixel or not, thanks to the initialization of the memory M1 with the value (255,255) which allows to discriminate the velocity vector values which do not exist. On the other hand, the first phase does not distinguish the best parent pixel when a pixel to be interpolated has several parent pixels, therefore the associated speed vector is that corresponding to the last found parent pixel, for a given pixel to interpolate, when it has several fathers pixels. When the velocity vector (IX ', IY', T _j ) has components other than 255, this means that the pixel to be interpolated has an associated velocity vector which was determined during the first phase. The method then consists in calculating the vectors B and A according to formulas (5) and (6) for the speed vector read in the memory M1 or for the speed vector interpolated, associated with the pixel (IX ', IY'). Then it consists in calculating the luminance values L ( , T _a ) and L , T _b ). When the speed vector is not zero, this calculation is carried out by spatial interpolation in the frames T _a and T _b respectively, according to the flow diagram represented in FIG. 5 and described previously. When the speed vector is zero, these two luminance values are read from the memories MA and MB at the address corresponding to (IX ', IY'). Then the method consists in calculating the interpolated luminance value L (IX ', IY', T _j ), according to formula (10). The interpolated luminance value is then supplied, for example, to a frame memory to be stored while waiting for use on the interpolated frame T _j .

Then the value of the coordinate IX 'is compared with its maximum value M. If the maximum value is not reached, the preceding operations are repeated from the incrementation of IX'. Otherwise the value of the coordinate IY 'is compared with its maximum value N. If this maximum value is not reached, the preceding operations are repeated from the incrementation of IY'. When IY 'has reached the maximum value N, the interpolation processing of the frame T _j is terminated. FIG. 10 represents a flow diagram of the first phase of the method according to the invention for a third variant of implementation of this method.

This third variant of implementation does not require no calculation of the DFD function and does not require an M2 memory, as does the second variant. Compared with the second variant, it has the advantage of allowing the interpolated luminance values to be stored without having an additional frame memory, since this third variant makes it possible to reuse the memory M1 for storing the interpolated luminance values. During the first phase, in this third variant, instead of storing in the memory M1 the speed vectors associated with the child pixels as the parent pixels of the frame carrying the motion are scanned, it consists of calculating and store in M1 the interpolated luminance values corresponding to these child pixels. Thus, during the second phase, all that remains is to calculate an interpolated luminance value for each orphan pixel only. The value of the components of the speed vectors of the neighboring pixels not being stored, it is not possible to use an interpolation to calculate a speed vector to be associated with each orphan pixel. The speed vector associated with the orphan pixel is then taken zero.

The first phase of this third variant begins with an initialization of the memory M1 with the value 255 which corresponds to the maximum value of the luminance and which is therefore easy to discriminate in order to detect the orphan pixels, these being discriminable by the fact that the corresponding value of the luminance, stored in the memory M1, remains equal to 255 at the end of the first phase of the process. It also begins with an initialization of the IY coordinate to zero. Then it consists of incrementing IY by one unit and initializing the coordinate IX to zero. It then consists of incrementing the IX coordinate by one, then reading the speed vector (IX, IY, T _i ) in the memory MI; then calculate the vectors according to formulas (4) to (6). Then it consists in calculating the luminance values L ( , T _a ) and L ( , T _b ) by a spatial interpolation in the frames T _a and T _b , in accordance with the flow diagram of FIG. 5 described above. Then it consists in calculating the interpolated luminance value L ( _j , T _j ), corresponding to the child pixel centered on the point F _j , according to formula (10); to write this luminance value in the memory M1 at an address corresponding to the components of the vector

The first phase then consists in testing the value of the coordinate IX to detect if it has reached its maximum value M. If the value M is not reached, it consists in repeating the previous operations from the incrementation of IX. When IX has reached the maximum value M, it consists in comparing the value of the coordinate IY with its maximum value N. If IY has reached the value N, the first phase is terminated. Otherwise, the previous operations are repeated from the increment of the IY coordinate.

FIG. 11 represents a flowchart of the second phase of the third variant of implementation of the method according to the invention. This second phase consists first of all in initializing to zero the coordinate IY 'of a pixel of the frame to be interpolated T _j ; then to increment this coordinate IY 'by one unit and to initialize to zero a second coordinate IX'. Then it consists in incrementing the coordinate IX 'of a unit; reading the luminance value stored in the memory M1 at the address corresponding to the coordinates (IX ', IY'); and to compare this luminance value with the value 255 which is the value loaded initially in the memory M1. If the luminance value is other than 255, this means that a luminance value was determined during the first phase, for the pixel to be interpolated with coordinates (IX ', IY'). If the luminance value is equal to 255, this means that this pixel to be interpolated has no parent pixel in the frame carrying the movement T _i . In this case, a zero speed vector is assigned to this orphan pixel, and the vectors F and have components equal to those of the vector carrying the center of the pixel to be interpolated, that is to say (IX ', IY'). Then the luminance values of the two basic points of the temporal interpolation, L ( A, T _a ) and L ( , T _b ) are read in the memories MA and MB at the address corresponding to (IX ', IY'). Then an interpolated luminance value L (IX ', IY', T _j ) is calculated according to formula (9). Finally, the interpolated luminance value is written into the memory M1 at the address corresponding to the coordinates (IX ', IY') to be stored there while waiting for the luminance values of the interpolated frame T _{j to be used} . At the end of the processing of all the pixels of the frame to be interpolated, the memory M1 will be filled with the interpolated luminance values. The memory M1 therefore plays the role of the memory of the additional frame which was necessary to store the interpolated luminance values according to the first and the second variant of implementation, described above.

Then this second phase consists in comparing the value of the coordinate IX 'with its maximum value M. If IX' has not reached its maximum value, the second phase then consists in repeating the previous operations starting from the incrementation of IX '. If IX 'has reached its maximum value, the second phase then consists in comparing the value of the coordinate IY' with its maximum value M. If IY 'has not reached its value N, the preceding operations are repeated from the increment of IY '. If IY 'has reached its maximum value, the processing of all the pixels to be interpolated is terminated. FIG. 12 represents the block diagram of a first part of a device for implementing the method according to the invention, in accordance with the first variant described above. This first part comprises: an input terminal 1; a multiplexing device 2; a motion estimation device 3; a sequencer 4; a memory 5 called memory MI; a device 7 for calculating the vector ; an address and data bus, 9; a computing device 10; a memory MA, for storing the luminance values of the frame corresponding to the instant T _a ; a memory MB for storing the luminance values of the frame corresponding to the instant T _b ; and calculation means 5a, for associating a speed vector with each pixel to be interpolated having at least one parent pixel. These calculation means 5a comprise: a device 6 for calculating the vector ; a device 8 for calculating the vector ; a device 10 for calculating

L (, T _a ); a device 11 for calculating L ( , T _b ); a subtractor 14; two comparators 15 and 16; two multiplexers, 17 and 18, with two inputs and one output; an AND logic gate 19; an initialization device 20; a memory M1 for storing the speed vectors associated with the pixels to be interpolated; and a memory M2 for storing the values of the DFD function.

The multiplexing device 2 has an input connected to the input terminal 1 of the interpolation device to receive a series of luminance values corresponding to a series of known frames constituted by frames of conventional television frames. These luminance values are provided, for example, by a television camera and an analog-to-digital converter. The device 2 has an output connected to the bus 9, which is a data, address and command bus, for transmitting the luminance values either to the memory MA or to the memory MB. In the example considered, the frame corresponding to time T _a is assumed to precede the frame corresponding to time T _b , but during the processing of a series of frames memories MA and MB are used alternately to store the frame the most recent.

The sequencer 4 supplies control signals to all of this exemplary embodiment and in particular to the device 2, in synchronism with the luminance values applied to the input terminal 1. The motion estimation device 3 has an input -output connected to bus 9 to enable it to control the reading of luminance values in the memories MA and MB, and the writing of the values of the components of the speed vector of the pixels of the frame T _i in the memory MI.

The device 3 is produced, for example, in accordance with the description of US Pat. No. 4,383,272. It provides an estimate of motion constituted by the components of a speed vector for each pixel of a frame corresponding to an instant T _i between the instants T _a and the instants T _b , as a function of the luminance values of the known frames at the instants T _a and T _b ,

The memory MI has an address input connected to an output of the sequencer 4 to receive an address constituted by the coordinates (IX, IY) of a pixel of the frame carrying the movement T _i . The sequencer 4 provides a series of coordinates corresponding to the usual scanning order of a frame of the television frame. This series of coordinates is also provided at a first input of the computing device 7. A second input of the computing device 7 is connected to a data output of the memory MI to receive the value of the components of the speed vector (IX, IY, T _i ) read in the MI memory at the address (IX, IY). The value of these components is also applied to an input of the computing device 6, to an input of the computing device 8, and to a first input of the multiplexer 18.

The calculation devices 6, 7, and 8 operate in parallel to determine the components of the vectors, respectively. _j PB. For each pixel of coordinates (IX, IY) in the frame T _i carrying the movement, the computing device 7 determines the center F _j of a child pixel, if there is one; the calculation devices 6 and 8 determine the coordinates of the base points A and B for a temporal interpolation, with a view to calculating a luminance value for the child pixel. Devices 6 and 8 respectively provide the components of the vector and vector to an input of the computing device 10 and to an input of the computing device 11.

The calculation device 10 has an output and an input connected respectively to an address input and to a data output of the memory MA, for controlling in the memory MA the reading of the luminance values necessary for a spatial interpolation from the pixels neighboring point A in frame T _a . Likewise, the device 11 has an output and an input linked respectively to an address input and to a data output of the memory MB to read there the luminance values necessary for a spatial interpolation between the pixels neighboring point B in the frame T _b . The luminance values L (PX, T _a ) and L ( Fïf, T _b ) calculated respectively by devices 10 and 11 are applied respectively to two inputs of the subtractor 14 to calculate a value of the DFD function. Subtractor 14 provides an algebraic value whose absolute value constitutes the value of the DFD function. This is applied to a first input of the multiplexer 17 and to first inputs of the comparators 15 and 16. A second input of the comparator 15 receives a value DFD 'of the DFD function read in the memory M2 at an address constituted by the components of the vector ^ ie corresponding to the child pixel being processed. The comparator 16 has a second input receiving a threshold value: 7. The comparators 15 and 16 each have an output connected to an input of the AND logic gate 19 to validate the latter when the value of the DFD function supplied by the subtractor 14 is less than the threshold value 7 and less than the value DFD 'read from the memory M2.

The door 19 has an output connected to a control input of the multiplexer 17 and to a control input of the multiplexer 18. The second input of the multiplexer 17 is connected to the data output the memory M2. The first and second input of the multiplexer 18 are respectively connected to the data output of the memory MI providing the vector (IX, IY, T _i ), and to an output of the memory M1 providing the value of the components of a speed vector read at the address constituted by the components _j

The memories M1 and M2 each have an address input connected to the output of the computing device 7 to receive the value of the components of the vector. PF _j . Firstly, the sequencer 4 controls in the memories M1 and M2, a reading at the address constituted by the components of the vector PF _j . This reading allows to know the value DFD 'of the DFD function and the value of the components of the speed vector previously found if the pixel centered on the point F _j has at least one parent pixel having been determined previously. If the value of the DFD function provided by the subtractor 14, for the parent pixel with coordinates (IX, IY), is less than 7 and less than the value DFD 'determined for another parent pixel found previously, then the signal supplied by the output of the gate 19 controls the multiplexers 17 and 18 to transmit respectively the value of the DFD function supplied by the subtractor 14 and the speed vector supplied by the output of the memory MI.

If the value of the DFD function provided by the output of the subtractor 14 is greater than 7 or is greater than the value DFD 'read in the memory M2, then the multiplexers 17 and 18 respectively transmit the value DFD' provided by the memory M2 and the speed vector supplied by the memory M1. The output of the multiplexer 17 and the output of the multiplexer 18 are respectively connected to data inputs of the memories M2 and M1.

In a second step, the sequencer 4 commands a write in these memories to write the value of the DFD function transmitted by the multiplexer 17 and the speed vector transmitted by the multiplexer T8. The content of memories M2 and M1 is therefore renewed when a better parent pixel is detected.

An initialization device 20 has two outputs connected respectively to the address input and to the data input of the memory M2 to provide it with a series of address values making it possible to write the value 255 at all the addresses during an initialization phase preceding the processing of each frame. During this initialization phase, the device 20 and the memory M2 are controlled by the sequencer 4 to carry out this writing. The address input and the data input of the memory M2 are connected on the other hand to an input terminal 26 and to an input terminal 28 which are connected to the second part of this exemplary embodiment. The data output of the memory M2 and the data output of the memory M1 are connected respectively to two output terminals 27 and 30, connected to the second part of this exemplary embodiment. The address input and the data input of the memory M1 are connected respectively to the input terminal 28 and to an input terminal 29, connected to the second part of this exemplary embodiment. FIG. 13 represents a more detailed block diagram of the calculation devices 6, 7, 8, 10 and 11. The device 6 comprises a multiplier 61, a device 62 for correcting parity of frames, and an adder 63. The input of the device 6, receiving the value of the components of the vector (IX, IY, T _i ), is connected to a first input of the multiplier 61. A second input of the multiplier 61 permanently receives the value - (T _j -T _a ). An output of the multiplier 61 is connected to an input of the device 62 to provide it with the value of the components of a vector: -V. (T _j -T _a ).

The device 62 adds to the value of these components, the value of the components of the vector Z which is a function of the parity of the frame being interpolated and of the frame T _a . The device 62 is connected, by links not shown, to outputs of the sequencer 4 supplying control signals which are functions of the parity of these two frames. The device 62 can consist of adders and a read only memory storing the value of the components of the vector Z, this read-only memory receiving an address constituted by the control signals supplied by the sequencer 4. The adder 63 has: an input connected to the output of the device 62; another input connected to the output of the computing device 7 to receive the value of the components of the vector ; and an output constituting the output of the device 6, output providing the value of the components of the vector

The calculation device 8 has a structure very similar to that of the calculation device 6. It includes a multiplier 81, a device for correcting parity of frames, and an adder 83. The multiplier 81 receives the value (T _b -T _j ) instead of the value - (T _j -T _a ), and the device 82 adds the value of the components of the vector _b , instead of the value of the components of the vector _a The value of the components of the vector T is a function of the parity of the frame T _j and the frame T _b . The device 82 is connected to the sequencer

4 by links not shown, providing control signals as a function of the parity of these frames. The calculation device 7 comprises a multiplier 71, a subtractor 72, a device 73 for parity correction of frames, and a device 74 for calculating the nearest whole value (VEPP) of a value. A first input of the multiplier 71 constitutes the input of the device 7 receiving the value of the components of the vector 7 (IX, IY, T _i ). A second input of the multiplier 71 permanently receives the value (T _i -T _j ). An output of the multiplier 71 is connected to a first input. of the subtractor 72 and provides it with the value of the components of the vector V. (T _i -T _j ). A second input of the subtractor 12 receives the coordinates (IX, IY) supplied by the sequencer 4 and constituting the components of the vector PC _j . An exit from the subtractor 72 supplies an input to the device 73 with the value of the components of the vector calculated in accordance with the formula (4).

The device 73 adds to the value of these components the value of the components of the vector which is function of the parity of the frame T _j and the frame T _i . The device 73 can be constituted like the devices 62 and 82. It is connected to outputs of the sequencer 4 by links not shown providing it with control signals depending on the parity of the frames T _j and T _i . An output of the device 73 is connected to an input of the device 74.

The device 74 adds 0.5 to each of the components of the vector supplied by the output of the device 73 and then takes the whole part of the sum obtained. The device 74 can consist of two adders each adding the constant value 0.5. The whole part is extracted by setting the bits making up the fractional part to 0. An output of the device 74 constitutes the output of the device 7 and provides the value of two components constituting the vector j The value of these components is used in particular as address value for the memories M1 and M2, during the writing in these memories of the value of the function DFD and of the components of the speed vector, corresponding to the child pixel centered on the point F _j .

The calculation devices 10 and 11 have a similar structure, but the device 10 is coupled to the memory MA while the device: 1 is coupled to the memory MB. The device 10 comprises: a device

101 for calculating the integer value closest to the value of the components of a vector; an adder

102; a subtractor 103; an address calculating device 104; and a computing device 105 which puts using formula (8) and which will be described in more detail with reference to FIG. 14.

An input of the device 101 constitutes the input of the device 10, and is also connected to a first input of the subtractor 103. An output of the device 101 is connected to a first input of the adder 102 and to an input of the computing device 104 addresses, to provide them with the value of the components (XG _a , YG _a ) obtained by taking the integer part of the value of the components of the vector The device 101 can be constituted like the device 74.

The adder 102 has a second input permanently receiving the value of the components of a vector (1,1), and has an output connected to a second input of the subtractor 103. An output of the subtractor 103 is connected to a terminal d input 106 of the calculation device 105 to provide it with the value of the relative coordinates of the point A: (ΔX _a , ΔY _a ). The calculation device 105 has a second input terminal 108 connected to a data output from the memory MA and has an output terminal 107 constituting the output of the device 10 and connected to the first input of the subtractor 14 to supply it with the value of luminance L (PA, T _a ). The address calculation device 104 has an input connected to the output of the device 101 for receiving the coordinates (XG _a , YG _a ) and has an output connected to an address input of the memory MA. The device 104 supplies four successive addresses to the memory MA under the action of a clock signal supplied by the sequencer 4 which commands four successive readings in the memory A to read the luminance values stored at addresses constituted by the coordinates : (XG _a + 1, YG _a +1); (XG _a , YG _a +1); (XG _a + 1, YG _a ); (XG _a , YG _a ). The device 103 can consist of four registers, four adders, and a multiplexer, controlled by control signals supplied by the sequencer 4. The calculation device 11 comprises: a device 111 for calculating the most integer value close to the components of a vector; a device 112 for calculating addresses; an adder 113; a subtractor 114; and a computing device 115, having functions analogous to the functions of elements 101 to 105 of the computing device 10.

FIG. 14 represents the block diagram of an exemplary embodiment of the calculation device 105. It comprises: six registers, 120 to 123 and 133, 135; eight multipliers, 124 to 131; an adder 132; and two subtractors 134 and 136. The device 105 implements formula (8). The input terminal 106 is connected to inputs of registers 120 to 123 which respectively store the four pairs of coordinates applied to the input terminal 106. The input terminal 108 is connected to inputs of registers 133 and 135 which store the values ΔX _a and ΔY _{a respectively} . The outputs of registers 120 to 123 are respectively connected to first inputs of multipliers 124, 126, 128, 130. The output of register 133 is connected to: a first input of subtractor 134, a second input of multiplier 126, and a second , input of the multiplier 130. The output of the multipliers 124, 126, 128, 130 is connected respectively to a first input of the multiplier 125, of the multiplier 127, of the multiplier 129, and of the multiplier 131. The output of the register 135 is connected to a second input of multiplier 129, a second input of multiplier 131, and a first input of subtractor 136. The subtractor 134 has a second input permanently receiving the value 1 and has an output connected to a second input of multiplier 124 and to a second input of multiplier 128. Subtractor 136 has a second input permanently receiving value 1 and has an output connected to a second input of the multiplier 125 and to a second input of the multiplier 127. The outputs of the multipliers 125, 127, 129, 131, are respectively connected to four inputs of the adder 132. A fifth input of the adder 132 receives permanently the value 0.5. An output of the adder 132 is connected to the output terminal 107.

Subtractors 134 and 136 respectively supply the values ΔX _a -1 and ΔY _a -1. Multipliers

124 and 125 calculate the first term of formula (8). Multipliers 126 and 127 calculate the second term. Multipliers 128 and 129 calculate the third term. Multipliers 130 and 131 calculate the fourth term. The adder 132 adds up these four terms and the constant term 0.5.

FIG. 15 represents the block diagram of the second part of the exemplary embodiment of a device for the implementation of the first variant of the method according to the invention. This second part corresponds to the second phase of the process. It notably comprises calculation means 5b for determining a speed vector interpolated from the speed vectors of the pixels neighboring the pixel to be interpolated when the latter has no parent pixel or else when its parent pixels have not been validated.

It further comprises: calculation means 49 for associating a zero speed vector with the pixel to be interpolated when the interpolated speed vector is not validated, and to determine the luminance values L ( PA, T _a ) and L ( PB, T _b ) of the two base points A and B, as a function of the speed vector associated with the pixel to be interpolated; and calculating means 50 for calculating an interpolated luminance value, as a function of the luminance values of the two base points.

The calculation means 5b include: an address generator 31; two registers, 32 and 33; a comparator 34; a multiplexer 35, with two inputs and one output; and a calculating device 36 for calculating an interpolated speed vector

An output of the address generator 31 and an output terminal 58 of the computing device 36 are connected to the input terminal 28 of the computing means 5a, which is connected to the address inputs of the memories M1 and M2. The address generator 31 provides a series of addresses constituted by coordinates (IX ′, IY ′) corresponding to a systematic scanning of all the pixels to be interpolated from the frame T _j . Each of these addresses makes it possible to read a DFD value (IX ', IY') of the DFD function in the memory M2 and a speed vector (IX ', IY') in the memory M1, corresponding to a pixel to be interpolated. A data input from register 32 and an input terminal 57 of the calculation device 36 are connected to the output terminal 27 of the calculation means 5a, which receives the value DFD (IX ', IY') read in the memory M2 . A data input from register 33 and an input terminal 56 of the calculation device 36 are connected to the output terminal 30 of the calculation means 5a, which supplies the value of the components of the speed vector V (IX ', IY') read in memory M1.

The value DFD (IX ', IY') stored in the register 32 is compared with a threshold value, 255, by the comparator 34. An output of the comparator 34 is connected to a control input of the multiplexer 35. An output of the register 33 is connected to a first input of the multiplexer 35. An output terminal 59 of the calculation device 36 supplies the value of the components of a speed vector interpolated to a second input of the multiple xer 35. An output of the multiplexer 35 constitutes an output of the means 5b and is connected to a first input of the calculation means 49 to supply a speed vector Vt When the value of the DFD function supplied by the register 32 is less than the threshold value 255, the comparator 34 controls the multiplexer 35 so that it transmits the value of the components of the vector stored in register 33, because this speed vector is suitable. When the value of the DFD function is equal to 255, this means that the pixel to be interpolated has no parent pixel, therefore it is necessary to determine a speed vector by interpolation from the speed vectors of the neighboring pixels. The comparator 34 then controls the multiplexer 35 so that it transmits the value of the components of the interpolated speed vector T supplied by the calculation device 36.

A block diagram of the computing device 36 is shown in FIG. 16. It comprises: an address generator 150; ten registers, 151 to 154 and 165 to 168; five comparators 156 to 159; four multipliers, 161 to 164; two adders 170 and 171; a divider 172; and a multiplexer 173, with two inputs and one output. The address generator 150 has an input connected to the input terminal 60 to receive the sequence of coordinates (IX'.IY ') of each pixel of the frame to be interpolated T _j , and it has an output connected to the terminal 58 to successively supply four values coordinates (IX '', IY '') for each pixel to be interpolated. Each pair of coordinates (IX '', IY '') corresponding to a pixel close to the pixel to be interpolated, on the same column or on the same line of the frame, and constitutes a reading address for reading the vectors from the memory M1 speed associated with these neighboring pixels, if they have an associated speed vector, and to read in the memory M2 the values of the DFD function corresponding to these neighboring pixels and making it possible to know whether they have a parent pixel or not.

The registers 151 to 154 each have a data input connected to the input terminal 56 for receiving and storing the values of the DFD function read in the memory M2 and corresponding to the four neighboring points respectively. The registers 165 to 168 each have a data input connected to the input terminal 57 to receive the value of the components of the speed vectors read in the memory M1 and corresponding respectively to the four neighboring points. The registers 151 to 154 have outputs connected respectively to first inputs of the comparators 156 to 159. The comparators 156 to 159 have second inputs receiving a constant value equal to 255 and have outputs connected respectively to first inputs of the multipliers 161 to 164 and respectively to four inputs of the adder 170. The comparators 156 to 159 each supply a logic signal of value 1 when the value applied to their first input is less than 255 and a value of zero when the value applied to their first input is equal to 255. The values of these logic signals are added by the adder 170. The output of the adder 170 is connected to a first input of the divider 172 and to a control input of the multiplexer 173. The outputs of registers 165 to 168 are respectively connected to second inputs of the multipliers 161 to 164. The multipliers 161 to 164 multiply the values of the components of the speed vectors by the values of the logic signals supplied respectively by the comparators 156 to 159, and provide the result on outputs connected respectively to four inputs of the adder 171. The output of the adder 171 is connected to a second input e of the divider 172. An output of the divider 172 provides the value of the components of an interpolated speed vector according to formula (10). An output of the multiplexer 173 is connected to the output terminal 59 of the calculation device 36. When the sum calculated by the adder 170 is not zero, the value which it provides on its output controls the multiplexer 173 so that it transmits the value of the components of the interpolated vector. If the value of the sum calculated by the adder 170 is equal to 0, that is to say if the four neighboring pixels do not have validated parent pixels, the value 0 applied to the control input of the multiplexer 173 commands the transmission of a zero vector to constitute the vector T supplied by the computing device 36. The computing means 49, of the second part of the exemplary embodiment, the block diagram of which is shown in FIG. 15, include two halves calculating in parallel the luminance values of the base points A and B and the value of the DFD function, respectively for the speed vector vt provided by the calculation means 5b and for a zero speed vector. These two calculations in parallel make it possible to save time when finally it turns out that the speed vector provided by the calculation means 5b is not validated because the corresponding DFD has a value greater than threshold 7.

These calculation means 49 comprise: a calculation device 37 for calculating the vector for the velocity vector T a calculating device 38 for calculating a vector for the velocity vector ; a calculating device 41 for calculating a luminance value L ( T _a ) for the velocity vector ; a calculating device 42 for calculating a luminance value L ( PBT _b ) for the velocity vector; a device 43 reading L ( PA, T _a ) at the address (IX ', IY') in the memory MA and a device 44 for reading L ( , T _b ) at the address (IX ', IY') in the memory MB, which corresponds to a zero speed vector; a subtractor 45 and a subtractor 46 for determining respectively the value of the DFD function for the speed vector and for a zero speed vector; a comparator 48; and a multiplexer 47 with eight inputs and four outputs.

The calculation device 37 is similar to the calculation device 6 described above and shown in FIG. 13. The calculation device 38 is similar to the calculation device 8 described above and shown in FIG. 13. The calculation devices 41 and 42 are respectively similar to the calculation devices 10 and 11 described above and represented in FIG. 13. The reading devices 43 and 44 each have an input-output connected to the bus 9. They have a very simple structure made up of buffer registers controlled by signals supplied by sequencer 4, and routed by links not shown.

The calculation devices 37 and 38 each have their input which is connected to the first input of the calculation means 49 to receive the value of the components of the vector Tj. The outputs of the calculation 37 and 38 are respectively connected to the inputs of the calculation devices 41 and 42. The output of the device 41 is connected to a first input of the subtractor 45 and to an input marked ai of the multiplexer 47. The output of the device 42 is connected to a second input of the subtractor 45 and to an input denoted b1 of the multiplexer 47. The output of the device 43 is connected to a first input of the subtractor 46 and to an input denoted a2 of the multiplexer 47. The output of the device 44 is connected to a second input of the subtractor 46 and to an input denoted b2 of the multiplexer 47.

The output of the subtractor 45 is connected to a first input of the comparator 48 and to an input marked d1 of the multiplexer 47. The output of the subtractor 46 is connected to an input marked d2 of the multiplexer 47. The multiplexer 47 also has: an input marked d connected to the first input of the calculation means 49 to receive the vector Υ an input denoted c2 which receives a zero speed vector; four outputs denoted a, b, c, d constituting respectively a first, a second, a third and a fourth output of the calculation means 49; and a control input connected to the output of comparator 48.

Subtractor 45 calculates the value of the DFD function for the pixel to be interpolated, with the vector Tt as associated velocity vector. The value of the DFD function is compared with the threshold value, 7, by the comparator 48. If the value of the DFD function is less than or equal to 7, the multiplexer 47 transmits the information applied to the inputs a1, b1, c1, d1, respectively to outputs a, b, c, d. If the value of the DFD function corresponding to the vector V is greater lower than the threshold value, 7, the multiplexer 47 is controlled to transmit the information present on inputs a2, b2, c2, d2 respectively to outputs a, b, c, d.

Thus the first and the second output of the calculation means 49 respectively provide a luminance value L , T _a ), L ( , T _b ), corresponding to the speed vector selected to be associated with the pixel to be interpolated; the second and the third output of the calculation means 49 respectively supply the speed vector which is used to constitute the vector V (IX ′, IY ′, T _i ) associated with the pixel to be interpolated, and the value of the DFD function corresponding to this pixel to be interpolated (IX ', IY', T _i ).

The third and fourth outputs of the calculation means 49 are respectively connected to the input terminals 26 and 29 of the calculation means 5a for recording the associated speed vector and the value of the DFD function in the memories M1 and M2 at the address constituted by the coordinates (IX ', IY'). This speed vector and this value of the DFD function will thus be available later to calculate interpolated speed vectors for orphan pixels.

The calculation means 50 have two inputs connected respectively to the first and to the second output of the calculation means 49 to receive the two luminance values of the base points A and B corresponding to the pixel to be interpolated. The calculation means 50 comprise two multipliers 51 and 52 and an adder 53. The multiplier 51 has: a first input connected to the first output of the means 49; a second input receiving a constant value equal to T -T; and an output connected to a first input of the adder 53. The multiplier 52 has: a first input connected to the second output of the means 49; a second input receiving a constant value equal to and one output connected to one second input of the adder 53. The adder 53 has an output connected to the output terminal 51 of the time interpolation device to provide it with an interpolated luminance value L (IX ′, IY ′, T _i ) corresponding to the pixel at interpolate, with coordinates (IX ', IY', T _i ), and calculated according to formula (9). These luminance values can be used in real time, as they are calculated, or else be stored in a memory of the frame, not shown.

The invention is not limited to the embodiments described above, many variants are within the reach of ordinary skill in the art. In particular, it is within the reach of those skilled in the art to modify the embodiment described above to implement the second and third variants of the method, described above.

The invention is applicable in real time, in particular to devices for changing the standard for television frames, and for the restitution of frames transmitted at very low bit rate with the suppression of a certain number of frames.

Claims

1. A method of temporal interpolation of images, to determine an interpolated luminance value for each pixel of an image (T _j ) said image to be interpolated, from the luminance values of the pixels of a first and of a second image called known images (T _a ,

T _b ), taken at times T _a and T _b respectively, T _j being between T _a and T _b ; consists in :

- associate the values of the components of a speed vector with each pixel (H) of the image to be interpolated (T _j ), said pixel to be interpolated;

- determine in the two known images (T _a ,

T _b ) the coordinates of two points (A, B) called base points of the temporal interpolation, each of these two points corresponding to the center (F _j ) of the pixel to be interpolated (H), by a translation in the plane of one of the known images (T _a , T _b ), translation whose vector is a function of the speed vector associated with the pixel to be interpolated (H), multiplied respectively by the durations T _j -T _a and T _b -T _j ; - determine the luminance value of each of the two base points (A, B) by spatial interpolation between neighboring pixels of each of these base points (A, B), respectively in the two known images (T _a , T _b ); - determine a luminance value for the pixel to be interpolated (H), by calculating a linear combination of the two luminance values of the base points

(A, B), weighted by the durations T _j -T _a and T _b -T _j respectively; characterized in that, to associate the components of a speed vector with each pixel to be interpolated (H), it consists in: - estimate by a conventional process the components of a speed vector for each pixel (G) of a single image (T _i ), called the carrier of the movement, corresponding to an instant T _i between T _a and T _b , or being T _a or T _b ;

- determine, successively for each pixel

(G) of the carrier image (T _i ), said father pixel, the coordinates of the center (F _j ) of a pixel to be interpolated (H), said child pixel, corresponding to the center (C _j ) of the father pixel by a translation whose vector is a function of the speed vector i of the parent pixel, multiplied by the duration

T _i -T _j ;

- associate, successively with each pixel to be interpolated (H), a speed vector constituted by the speed vector its parent pixel (G) or one of its parent pixels, if there is at least one parent pixel for this pixel to be interpolated (H).

2. Method according to claim 1, characterized in that to determine the coordinates of the center (F _j ) of a child pixel (H) corresponding to a given father pixel (G), it consists in:

- determine the coordinates of a point (E _j ) of the image to be interpolated (T _j ) corresponding to the center (C _i ) of the parent pixel (G), by a translation whose vector is equal to the speed vector of the parent pixel multiplied by the duration T _i -T _j ;

- then to determine the pixel (H) of the image to be interpolated (T _j ), whose center (F _j ) is closest to the point (E _j ) thus determined, this pixel then constituting the child pixel of the parent pixel given (G).

3. Method according to claim 1, applied to images consisting of television image frames, characterized in that to determine the coordinates of the center (F _j ) of a child pixel corresponding to the center (C _j ) of a given parent pixel, it consists of: - determine the coordinates of a point (E _j ) of the frame to be interpolated (T _j ) corresponding to the center (C _i ) of the father pixel (G), by a translation whose vector is equal to the speed vector of the multiplied father pixel by the duration T _i -T _j , by shifting one of the coordinates of the point (E _j ) of the image to be interpolated corresponding to the center (C _j ) of the parent pixel (G), by a value equal to 0 or -0.5 or +0.5 pixels, depending on the parity of the frame constituting the image to be interpolated (T _j ) and the parity of the frame constituting the motion-carrying image (T _i );

- then to determine the pixel (H) of the image to be interpolated (T _j ), whose center (F _j ) is closest to the point (E _j ) thus determined, this pixel then constituting the child pixel of the parent pixel given (G); and in that in order to determine in the two known images (T _a , T _b ) the coordinates of the two base points (A, B), it also consists in shifting one of s coordinates of these two points (A, B) of 0 or -0.5 or +0.5 pixels, depending on the parities of the two frames constituting the two known images (T _a , T _b ), and the parity of the frame constituting the image to be interpolated

(T _j ).

4. Method according to one of claims 1 to 3, characterized in that for each pixel to be interpolated (H) having at least one parent pixel, it also consists in validating the speed vector of the parent pixel (G), by calculating the value, called DFD of the pixel to be interpolated, of the difference in the luminances of the two base points (A, B) of the temporal interpolation; and verifying that the DFD is less than a fixed threshold value; and in that, if the speed vector of no parent pixel is not validated, or if the pixel to be interpolated has no parent pixel, it also consists: - Associating with the pixel to be interpolated an interpolated speed vector as a function of the speed vectors associated with the neighboring pixels of the pixel to be interpolated, if the DFD of the pixel to be interpolated, calculated for the interpolated speed vector, is less than a fixed threshold value;

- otherwise, to associate with the pixel to be interpolated a zero speed vector.

5. Method according to claim 4, characterized in that, when a pixel to be interpolated (H) has several parent pixels (G), the speed vector associated with this pixel to be interpolated is constituted by the speed vector of the parent pixel such that the DFD of the pixel to be interpolated is minimal.

6. Method according to claim 1, characterized in that, when a pixel to be interpolated (H) has several parent pixels (G), the speed vector associated with this pixel to be interpolated is constituted by the speed vector of the parent pixel having been determined last.

7. Method according to claim 1, characterized in that it further consists, for each pixel to be interpolated (H) having no parent pixel, to associate with it a speed vector interpolated as a function of the speed vectors associated with the neighboring pixels of the pixel to be interpolated, if there are neighboring pixels having an associated speed vector; otherwise, to associate it with a zero speed vector.

8. Method according to claim 1, characterized in that it also consists, for each pixel to be interpolated (H) having no parent pixel (G), to associate therewith a zero speed vector.

9. Device for temporal interpolation of images, for implementing the method according to claim 1, receiving a series of luminance values representing respectively the pixels of a first known image (T _a ) taken at an instant T _a and the pixels of a second known image (T _b ) taken at an instant T _b ; receiving a series of values representing respectively the components of the speed vector of the pixels of a third image (T _i ), called the image carrying motion, corresponding to an instant T _i between T _a and T _b or which can be T _a or T _b ; and providing a series of interpolated luminance values representing the luminance of the pixels of an image to be interpolated (T _j ) corresponding to an instant T _j between T _a and T _b ; characterized in that it comprises:

- a first and a second memory (MA, MB) for storing respectively the luminance values of the first and of the second known image (T _a , T _b ); - first calculation means (7) for calculating, for each pixel of the carrier image (T _i ), called the father pixel (G), the coordinates of the center (F _j ) of a pixel, called the child pixel (H ), if killed in the image to be interpolated (T _j ) and whose center (F _j ) corresponds to the center (C _j ) of the parent pixel by a translation whose vector is a function of the speed vector of the parent pixel (G), multiplied by the duration T _i -T _j ;

- second calculation means (5a, 5b) for associating with each pixel (H) of the image to be interpolated (T _j ), said pixel to be interpolated, a speed vector which is the speed vector of its parent pixel (G) or one of its parent pixels, if there is at least one;

- third means (49) for calculating the coordinates, in the two known images (T _a , T _b ) respectively, of two points (A, B) called base points of the temporal interpolation, each of these two points ( A, B) corresponding to the center of the pixel to be interpolated (H), by a translation whose vector is a function of speed vector associated with the pixel to be interpolated, multiplied respectively by the durations T _j -T _a and T _b -T _j ; - and fourth means (50) for calculating the luminance values of the two base points (A, B) and deducing therefrom a luminance value for the pixel to be interpolated (H), by calculating a linear combination of two luminance values of the two base points, weighted by the durations T _j -T _a and T _b -T _j respectively.

10. Device according to claim 9, for implementing the method according to claim 2, characterized in that the first calculation means (7) comprise:

- a multiplier (71) for calculating a vector by multiplying the speed vector of the parent pixel (G), by the duration T _i -T _j ;

- a subtractor (72) for subtracting, from the vector calculated by the multiplier (71), a vector having as components the coordinates (IX, IY) of the center (C _j ) of the parent pixel (G);

- a device (74) for calculating the integer value closest to each of the components of the vector calculated by the subtractor (72); each constituting a coordinate of the center (F _j ) of the child pixel (H).

11. Device according to claim 10, for implementing the method according to claim 3, characterized in that it further comprises a device (73) for parity correction of frames, for adding a correction vector to the calculated vector by the subtractor (72), interposed between the subtractor (72) and the device (74) to calculate the integer value closest to each component; the correction device (73) being controlled as a function of the parity of the frames corresponding to the instants T _i and T _j .

12. Device according to claim 9, for implementing the method according to claim 4, characterized in that the second calculation means (5a, 5b) comprise: - two first calculation devices (6, 8) to calculate the coordinates, in the two known images (T _a , T _b ) respectively, of two points (A, B) called base points of the temporal interpolation, starting coordinates of the pixel to be interpolated (H), by a translation function of the speed vector of a parent pixel

(G) of the pixel to be interpolated, if there is at least one parent pixel, and as a function of the durations T _j -T _a and T _b -T _j respectively; - two second calculation devices (10, 11) for calculating the luminance values of the two base points (A, B) by spatial interpolation respectively in the two known images (T _a , T _b );

- a subtractor (14) for calculating the absolute value, called the DFD of the pixel to be interpolated of the difference, of the two luminance values calculated by the second calculation devices (10, 11);

- a first comparator (16) for comparing the value of DFD with a predetermined threshold value, and validating the speed vector of the father pixel considered if the value of DFD is less than the threshold value;

- a first memory (M1) for storing an associated speed vector, for each pixel to be interpolated;

- calculation means (5b) for associating with a pixel to be interpolated, a speed vector interpolated from speed vectors associated with neighboring pixels, and read in the first memory (M1), when the pixel to be interpolated has no parent pixel whose velocity vector is validated.

13. Device according to claim 12, for implementing the method according to claim 5, characterized in that the second calculation means (5a, 5b) further comprise:

- a second memory (M2) for storing a DFD value for each pixel to be interpolated; - a second comparator (15) for comparing each calculated DFD value for a pixel to be interpolated, with a previously stored DFD value for the same pixel to be interpolated; - means (4, 18) for writing into the first memory (M1) the speed vector of a father pixel (G) of a pixel to be interpolated (H), if the DFD value calculated for this speed vector is lower to that stored in the second memory (M2) at the address corresponding to this pixel to be interpolated (H);

- Means (4, 17) for writing into the second memory (M2) the value of DFD calculated for this speed vector, at the address corresponding to the pixel to be interpolated (H).