EP0492702B1 - Correlation device - Google Patents

Description

• un corrélateur qui détermine une corrélation entre des symboles d'un bloc courant et des symboles ayant des positions identiques dans une suite de blocs à traiter issus de ladite configuration,
• et des moyens de sélection qui sélectionnent parmi les symboles dudit bloc courant un nombre limité de symboles de référence sur lesquels porte la corrélation.

The invention relates to a correlation device operating on input signals representing symbols organized in blocks in a regular configuration, comprising:

• a correlator which determines a correlation between symbols of a current block and symbols having identical positions in a series of blocks to be processed from said configuration,
• and selection means which select from the symbols of said current block a limited number of reference symbols to which the correlation relates.

Regular configuration of input signals can be one-dimensional or two-dimensional. he can s act for example of a one-dimensional configuration where correlations can occur on signal blocks acoustic input for sound processing. A configuration symbol is then a value coded under binary form. Preferably it may be a question of two-dimensional configurations where correlations operate on images, for example animated, on which we determines motion vectors of the animated sequence between successive images. A configuration symbol is then a pixel. These applications are often undertaken to reduce the data rate. We meet them for example in the field of television (high speed) or in the field of videophone and video conference (speed low).

The app for estimating moving image movement in television presents the particularity of revealing very severe constraints in particularly by the need to operate a large amount of high throughput calculations. This application will therefore be taken here as an example without limiting the invention.

For motion motion estimation, the technique that seems the most interesting is this so-called "block to block correspondence" called "block matching" in English language. It consists of cutting the image into blocks of pixels. For a given block of the image we determine in a previous image the block closest to it in order to determine the movement of the animated sequence represented in the image. It is this movement information which is transmitted, for each block, to the receiving agency located at the end of the transmission channel.

The principle of treatment is to follow the evolution of the content of images on successive images. This technique takes advantage of the fact that the content informational varies little from one image to another. Move of the information contained in a block is generally limited a few neighboring blocks. This monitoring is carried out by identifying certain parts which may be the most significant or who are chosen voluntarily. This may concern the all blocks of the image.

A current reference block (expression later simplified in "current block") may be found located differently in images either earlier or posterior. It is this displacement that must be determined. But this displacement remains limited and for that we define a window inside which said current block is likely to be found.

The estimated movement is the translation between the position in the image of the current block and the position in the anterior / posterior image of the anterior / posterior block which has the highest correlation. This allows determining a displacement vector relative to the current block. The correlation will therefore be determined pixel by pixel similarly position for all possible positions of the current block at inside the corresponding window. The same operations are repeated for the other current blocks selected. The correlation is generally evaluated by summing for all block, the absolute values of the differences in characteristics from pixel to pixel block to block. The pixel characteristic is generally its luminance.

Two main obligations appear for a material realization of a motion estimator: sound ability to operate in real time, its material complexity. Different strategies have been developed, but some algorithms that are less demanding in number of operations are less regular and therefore more complex to implement artwork. The regularity of the calculation phases is a condition that must be taken into account in particular for the realization of dedicated integrated circuits. This explains why the strategy usually implemented is research exhaustive covering all the pixels of all the blocks of the window. Scanning techniques to operate successively the analysis of each pixel of the current block in the window can reduce the hardware complexity of implementations.

We therefore sought to develop methods for reduce the complexity of motion estimation processing without affecting its ability to operate in real time. So we knows the article entitled "Motion-compensated interframe coding for video conferencing "by T. KOGA et al. National Telecommunications Conference 29 November-3 December 1981 NEW ORLEANS LOUISIANE. This document determines a vector motion estimation using different manipulations including a subsampling which allows take into account only a limited number of pixels per block current i.e. one pixel out of two in a line for one first case and which even extends to every other line in one second case. The hardware complexity is reduced but at the expense of the accuracy of motion estimation. The inter-image differential coding, which is an application privileged motion estimation, consists of pass a prediction error, the latter being in the present case the block of the previous image chosen in the window. When motion estimation is done correctly the prediction error is small and therefore the length of the code transmitted by a transmission channel is scaled down. We can therefore see that the degradation of the precision of motion estimation has an impact on the efficiency of the coding. Thus in the prior art cited for a rate of 1.5 Mbits / s transmission the prediction error rate can increase by 50%. This requires an increase of throughput to achieve acceptable image quality.

The problem is therefore to design a correlation device which allows an achievement simplified material (compared to a realization usual) and who can operate in real time with a rate reduced prediction errors even for television image processing.

• soit en fonction du contenu informationnel dudit bloc courant,
• soit selon une sélection aléatoire,
• soit pour une première partie prioritairement en fonction du contenu informationnel et pour une seconde partie selon une sélection aléatoire.

This goal is achieved with a correlation device, the selection means of which select, in said current block, one (or more) reference symbol (s) by choosing its (their) position:

• either according to the information content of said current block,
• either according to a random selection,
• either for a first part primarily according to the informational content and for a second part according to a random selection.

It is thus possible to adopt two strategies which can be combined.

In the case of an image for example, advantageously according to the first strategy only the pixels carrying "distortion" information in the block current can be chosen for correlation calculation.

According to the second strategy, we ignore the content of the current block, and the reference pixels selected are distributed randomly in the current block.

These two strategies break the regularity of the selection which is made when eliminating a pixel on two or even a line on two. The regularity of this selection has the disadvantage of deleting the high frequency part of the information spectrum and cause an unacceptable error rate at high throughput (see the document already cited). On the other hand according to the invention the part high frequency of the information spectrum is not suppressed and its importance can even be controlled by intervening on the limited number of symbols selected per current block. In combining strategy with informational content and strategy by random selection we keep this absence of regularity of the position of the symbols in the current block and the error rate is further reduced by closely following the content of information. In the latter case the symbols of reference are selected in priority according to the content informational and their number is supplemented by a selection random, up to the predetermined limited number.

When a block is made up of pixels (respectively binary coded values) the selection consists in taking pixels with reference (respectively binary coded values) having a certain position in the current block.

To these pixels which have a determined position in the current block correspond to other pixels which have the same position in blocks of the window to analyze. The correlation which is calculated between two blocks actually takes place between the pixels of the two blocks that have the same position. The operation is continued for all possible positions of the current block in the window. The treatment is then continued with another current block associated with another search window. It is possible that the same selection random be used for multiple common blocks for select the position of the reference symbol (s) randomly selected.

In case the selection is made taking account of the information content of the current block, the position reference symbols in the current block is determined from a distortion function calculated for each symbol of the current block vis-à-vis the surrounding symbols, the selected reference symbol (s) being chosen among the symbols whose values of the function of distortion most differ from those relating to symbols surrounding.

This distortion function can, for example, take into account the variation of a characteristic such as the luminance or others between the neighboring pixels of the same block current. To determine the most representative pixels of a current block, the distortion function can be determined by calculating a Laplacian pertaining to the characteristic chosen for the input signal. More generally we can take into account neighboring pixels which surround a given pixel isotropically. But so that in a current block the reference pixels are not concentrated in an overly localized part of the current block, it is possible to impose a distribution constraint in subdividing the current block into several zones for example in the form of a regular grid or not or any other mesh form. In this case we impose that all zones or at least some of them are represented by at least one pixel. This pixel by area can be chosen with the same strategies than those already described for a current block. So when a current block is subdivided into several zones, the selection means operate in isolation on each zone, the correlation always operating on all the pixels of reference of the current block. Preferably each zone of a current block has the same number of reference pixels. When the reference pixel (s) of the zones is (are) selected from the determination of a function of distortion, preferably the reference pixel (s) is (are) selected in each zone so that the values of the distortion function which characterize it, are alternately maximum and minimum for zones contiguous.

According to the invention, one does not keep in a block current that a limited number of reference symbols for operate the correlation calculation. This limited number can be chosen in several ways. It can be fixed a priori after preliminary tests which make it possible to establish a calculation of correlation with an acceptable error rate. This can be achievable when the current block contains a reduced number pixels for example 4 x 4.

When it comes to a strategy operates a distortion function taking into account the content informational, this limited number can be chosen from bounds imposed on the values that this function can present distortion.

More generally when the block size current becomes large, it is possible to take into account the informational content of the entire image and not only that of the current block for determining the limited number of symbols. For this we establish a matrix of characteristic correlation of all input signals constituting the image. By the so-called "analysis method in principal components ", we calculate the eigenvalues of this correlation matrix and we determine the variances among which the number of highest variances necessary for good representation of the entire image. This number is then chosen as a limited number of reference symbols.

According to the invention, in this case, therefore, depend on the limited number of content reference symbols of the image. In a large number of situations this number limited remains mainly related to the type of images, i.e. to the application itself. Nevertheless it is possible enslave more finely said number limited to the content of the image by determining it regularly and adaptively on lots of images. This can be achieved using a correlation device which includes a neural network which implements the so-called "component analysis" method principal "and which selects the number of variances higher. Cyclically this number of variances is introduced into the selection means to constitute said limited number in order to operate as already described.

A neural network is a known device for signal processing which can, after a step of programming or a learning step, perform treatments determined for example a "component analysis main "distribution of input signals. Such neural network can for example be that described in the document: "Optimal unsupervised Learning in single-layer linear Feedforward Neural networks "T.D. SANGER Neural Networks, vol.2, 1989, p.459-473.

As noted earlier, the fact of only carry the correlation calculation on certain reference symbols having a certain position in the block current makes it possible to simplify the material realization, increase the processing speed while maintaining the error rate at low and controllable values.

• un réseau systolique de registres qui se transfèrent l'un à l'autre des données des pixels de la fenêtre,
• une pluralité de processeurs, chaque processeur réalisant un calcul de la corrélation d'au moins un vecteur de déplacement,

le dispositif mettant en oeuvre une première étape de calcul au cours de laquelle :

a) chaque processeur opère un calcul de corrélation successivement sur les n/p symboles de référence sélectionnés d'une première zone, ledit calcul portant successivement sur tous les vecteurs de déplacement que doit calculer ledit processeur,

b) des moyens de mémorisation stockent des résultats intermédiaires délivrés par les processeurs à l'issue de a),

c) des moyens de transfert transfèrent le contenu des registres opérant sur une zone vers des registres suivants opérant sur une zone suivante,

ladite première étape de calcul étant ensuite réitérée pour les p-1 zones suivantes,
des moyens de sommation déterminant la somme des résultats intermédiaires relatifs à chaque bloc à traiter pour délivrer les corrélations de ladite suite de blocs.In addition, the mechanisms for selecting the reference symbols make it possible, in certain situations, to further simplify the complexity of the hardware implementation. In particular when the current block is subdivided into zones, this advantageously makes it possible to obtain either a gain in processing speed or a gain in complexity of the hardware. This situation appears when the correlation device operates a correlation calculation with current blocks of data subdivided into p zones with n / p reference symbols selected by zone, the correlator comprising:

• a systolic network of registers which transfer data from the window pixels to each other,
• a plurality of processors, each processor performing a calculation of the correlation of at least one displacement vector,

the device implementing a first calculation step during which:

a) each processor operates a correlation calculation successively on the n / p selected reference symbols of a first area, said calculation relating successively to all the displacement vectors that said processor must calculate,

b) storage means store intermediate results delivered by the processors at the end of a),

c) transfer means transfer the content of the registers operating in one zone to the following registers operating in a following zone,

said first calculation step then being repeated for the following p-1 zones,
summation means determining the sum of the intermediate results relating to each block to be processed in order to deliver the correlations of said series of blocks.

A particularly interesting case arises for image processing, when said areas contain several pixels of which only one is selected to operate the correlation calculation. It is then possible to obtain a gain additional and to choose, for the correlator, between a architecture that favors speed or architecture that favors the simplicity of the hardware that is to say that reduces the surface for an integrated realization.

In the case of architecture favoring speed, the correlator includes substantially as many processors that there are possible displacements in the window and it operates on a reduced number of pixels of reference which increases its execution speed compared to a use of all the pixels of the current block. A particularly interesting case is that where each area does contains only 4 pixels, with a single reference pixel selected by area. In this case the correlator operates on a two-dimensional pixel window with current blocks subdivided into areas containing 4 pixels each with a single reference pixel selected by area, a matrix two-dimensional registers storing, at a given time, the pixels of said window according to the same order, four registers being assigned to the treatment of an area, rows of said registers communicating with each other, in the two directions of the matrix, even order pixels of the window and other rows of registers communicating between them odd order pixels of the window this communication being unidirectional in a direction of the window and being bidirectional in the other direction of the window, a processor being assigned to the processing of each displacement vector and operating with four registers containing even and odd pixels from the window, the order numbers of said pixels taken two by two being consecutive in each direction, a selection signal common operating processors with data from registers corresponding to the selected reference pixels in the current block.

In the case of architecture favoring reduction in hardware complexity, a case particularly interesting is the one where the correlator operates on a two-dimensional pixel window with blocks streams subdivided into rectangular areas containing k pixels with a single reference pixel selected per area, a two-dimensional matrix of registers storing, at an instant given, the pixels of said window according to the same scheduling, k registers being assigned to the processing of a zone, rows of said registers communicating with each other bidirectionally window pixels in both directions of the matrix, a processor being assigned to successive processing of k displacement vectors by operating with k registers containing pixels belonging to the same zone, the sequence numbers of said pixels being consecutive in both directions, a selection signal making operate each processor, for each shift, with the register data corresponding to the reference pixel selected from the current area.

The invention is not limited to the case of two-dimensional symbol configurations. It applies also in the case of one-dimensional configurations. The symbols can thus be binary coded values representing a one-dimensional configuration of signals acoustic. The correlation device then operates on current blocks containing several of said values from which are selected, in limited number, and in position, reference values.

figures 1A, 1B : une représentation d'une image dans laquelle se trouve une fenêtre renfermant des blocs à traiter sur lesquels doit porter la corrélation et celle d'un bloc courant dans une image postérieure,

figure 2 : un schéma-bloc d'un dispositif de corrélation,

figures 3A, 3B : une représentation d'un bloc courant de 8 x 8 pixels indiquant les positions des symboles de référence sélectionnés selon l'art antérieur,

figures 3C, 3D, 3E, 3F : à titre d'exemple, une représentation d'un bloc courant de 8x8 pixels indiquant les positions des symboles de référence sélectionnés selon l'invention,

figures 4A, 4B, 4C, 4D : une représentation des pixels environnant un pixel donné pour le calcul de la fonction de distorsion,

figure 5 : un schéma d'une partie des moyens de sélection permettant de calculer une fonction de distorsion selon la représentation de la figure 4A,

figure 6 : un schéma détaillé d'une unité de sélection comprenant un organe de calcul d'un Laplacien et un organe de tri,

figure 7 : un schéma-bloc d'un dispositif de corrélation comprenant un réseau de neurones pour déterminer le nombre de symboles de référence,

figure 8 : un schéma d'un corrélateur selon l'art antérieur,

figures 9A, 9B : une représentation de blocs à traiter d'une fenêtre et d'un bloc courant subdivisé en zones,

figure 10 : un schéma d'un processeur entouré des organes de base nécessaires à son fonctionnement,

figure 11 : un schéma d'un corrélateur privilégiant la simplicité du hardware opérant sur des blocs courants subdivisés en zones de 4 symboles parmi lesquels est choisi un symbole de référence.

figure 12 : un schéma d'un corrélateur privilégiant la vitesse opérant sur des blocs courants subdivisés en zones de 4 symboles.

The invention will be better understood using the following figures given by way of non-limiting examples which represent:

FIGS. 1A, 1B: a representation of an image in which there is a window containing blocks to be processed to which the correlation must relate and that of a current block in a posterior image,

FIG. 2: a block diagram of a correlation device,

FIGS. 3A, 3B: a representation of a current block of 8 x 8 pixels indicating the positions of the reference symbols selected according to the prior art,

FIGS. 3C, 3D, 3E, 3F: by way of example, a representation of a current block of 8 × 8 pixels indicating the positions of the reference symbols selected according to the invention,

FIGS. 4A, 4B, 4C, 4D: a representation of the pixels surrounding a given pixel for the calculation of the distortion function,

FIG. 5: a diagram of part of the selection means making it possible to calculate a distortion function according to the representation of FIG. 4A,

FIG. 6: a detailed diagram of a selection unit comprising a member for calculating a Laplacian and a member for sorting,

FIG. 7: a block diagram of a correlation device comprising a neural network for determining the number of reference symbols,

FIG. 8: a diagram of a correlator according to the prior art,

FIGS. 9A, 9B: a representation of blocks to be processed from a window and from a current block subdivided into zones,

FIG. 10: a diagram of a processor surrounded by the basic organs necessary for its operation,

FIG. 11: a diagram of a correlator favoring the simplicity of the hardware operating on common blocks subdivided into zones of 4 symbols from which a reference symbol is chosen.

Figure 12: a diagram of a correlator favoring speed operating on common blocks subdivided into areas of 4 symbols.

FIG. 1A represents an image 10 and FIG. 1B represents an image 20 which may be earlier but preferably after image 10 on which is performed the correlation calculation.

Image 20 is made up of pixels which are analyzed in the form of current blocks, for example blocks 21 ₁ , 21 ₂ , 21 ₃ . In image 20, for any current block, for example current block 22, the processing will consist in determining which block of the previous image 10 it is correlated with. For this around a block 12 which has, in image 10, the same coordinates as those of the current block 22 in image 20, a window 13 is determined which surrounds block 12 in order to limit the number and the position blocks that should be searched for as being the most likely to contain the correlated block. The size and shape of this window depends on the extent of the search that one wishes to operate and therefore ultimately on the processing time that it is possible to devote to examining a window and also on the complexity that we admit for the material realization which results from it. Thus the current block 22 (image 20) is compared with all the blocks of window 13 of image 10 obtained by moving the block in successive steps of at least 1 pixel.

At the end of this treatment another current block is chosen in image 20 to which another will correspond window in image 10 and the same processing is carried out, and and so on. The current blocks chosen in image 20 may be limited in number but preferably all blocks undergo the same treatment. Depends on position of the current block in image 20 and that of the block correlated in image 10 we deduce each time a vector movement of the current block. It is this vector of displacement as well as the pixel characteristic deviation (by luminance example) which is encoded and transmitted through the transmission in order to operate data compression.

The correlation device which implements the processing is shown in Figure 2. It includes a correlator 25, a selection unit 26 which determines on which pixels of the current block 22 should carry the correlation, a scanning unit 27 which determines the order in which the pixels of block 12 and by extension of the window must be analyzed. At the output of the correlator 25 a unit of decision 28 determines the block of window 13 for which the tion is maximum. The current block BL to be processed is introduced in the selection unit 26 and in the correlator 25. This last also receives the extent of the FEN window which can be slaved to the speed of execution necessary to the application.

Correlation consists in comparing successively two blocks together comparing the pixels having in each block of identical respective positions. It is common to carry out a comprehensive treatment covering all block pixels. But according to the document already cited it is possible to reduce the importance of treatment by not whereas only a limited number of pixels per current block.

Figures 3A and 3B show two possibilities implemented in the cited document. They represent a current block of 8 x 8 = 64 pixels on which we hatched the reference pixels which are selected to operate the correlation. In Figure 3A we selects every other pixel, which divides the processing by 2. In FIG. 3B we also ignore a complete line of pixels out of two which still divides the processing by 2. These limitations on the number of reference pixels may apply easily for example by intervening on the durations of sweep clocks. But the great regularity that follows of these modes of limitation has the consequence that the part high frequency of the spectrum of an information block is truncated and a significant error rate appears.

To remedy this defect according to the invention, does not systematically select regularly the reference pixels inside a current block.

A first strategy shown in the figure 3C as an example consists of a random selection at competition from a predetermined limited number of pixels of reference. So in Figure 3C we hatched 16 pixels of reference randomly selected in the current block.

A second strategy shown in the figure 3E as an example takes into account the informational content contained in the current block. So suppose that a curve fictitious 31 delimits in the current block two parts of different luminances. According to the invention, the pixels of reference selected to operate the correlation are going to be determined taking into account the informational content in determining a distortion function. Briefly we can indicate that the selection will be made in the parts where the characteristic (here the luminance) undergoes variations by compared to his close neighbors. All pixels in the current block are thus examined. The selection unit 26 (FIG. 2) operates this determination. Figures 4A, 4B, 4C, 4D indicate some possible ways of examining information content with isotropic distributions.

FIG. 4A represents a preferential mode which consists for a given pixel (hatched) in comparing its variation in luminance with its neighbors having coordinates which are individually distant from ± 1. This amounts to calculating a Laplacian Δ such that: Δ = 4I (x, y) - (I (x + 1, y) + I (x, y + 1) + I (x-1), y) + I (x, y-1)) For Figure 4B the distortion function is: Γ = 4I (x, y) - (I (x + 1, y + 1) + I (x + 1, y-1) + I (x-1, y + 1) + I (x-1, y -1)) For Figure 4C the distortion function is: Λ = 4I (x, y) - (I (x + 2, y) + I (x-2, y) + I (x, y + 2) + I (x, y-2)) For Figure 4E the distortion function is: Ψ = 8I (x, y) - (I (x + 1, y + 2) + I (x + 1, y-2) + I (x-1, y + 2) + I (x-1, y -2) + I (x + 2, y + 1) + I (x-2, y + 1) + I (x + 2, y-1) + I (x-2, y-1)).

Other distributions of surrounding pixels are possible even if they are not isotropic.

When the distortion function has been thus calculated for all the pixels of the current block, the unit of selection determines the pixels having the extreme values in order to select them to represent said current block.

According to the two strategies which have just been described it appears that certain parts of a current block may not be represented or may not be represented. To limit the error on the correlation which risks result it may be desirable that said parts be at least partially represented. For that more particularly according to the invention, the block is subdivided running in areas that may or may not be regular. Preferably, the current block is subdivided into zones identical having the same number of pixels. We do then no longer bring random selection to block level current but at the level of the area. So the pixels of references are randomly selected by area. this is shown in Figure 3D where 16 reference pixels randomly selected are represented by hatching. Same random selection of this type can be used for several current blocks to operate the calculation of correlation. Although the random selection takes place at the level of the zone, the correlation calculation remains performed at the level of the current block.

Figure 3F shows the situation where the two strategies can be combined. For this a first part of the reference pixels is selected primarily according to the information content and a second part is selected from a selection random by area. This second part is limited to competition than all of the reference pixels selected reaches the planned limited number.

The diagram in Figure 5 represents a unit of selection 26 determining the position of the points in a block current from the calculation of a distortion function (previous Δ, Γ, Λ or Ψ functions) operated in an organ of calculation 50. The results obtained are sorted in a sorting 51 according to a sorting algorithm in order to limit the number of reference symbols per current block. The addresses of retained reference symbols are stored in a register of addresses 52 to be used by the correlator. Unity selection 26 operates under the control of the scanning unit 27.

• d'une première ligne à retard 60 et d'une seconde ligne à retard 63 qui retardent chacune les données de la durée d'une ligne moins celle d'un pixel,
• d'un premier registre 61 et d'un second registre 62 qui retardent chacun les données de la durée de balayage d'un pixel.

A detailed diagram of the calculating member 50 and the sorting member 51 is shown in FIG. 6 in the case where the distortion function is the Laplacian Δ defined above. This distortion function corresponds to the representation of FIG. 4A where the letters N, S, E, W respectively represent the cardinal points North, South, East, West. These letters are shown in the diagram in FIG. 6, the letter C representing the central pixel. The situation concerns the case where the scanning applies to all the pixels of the current block according to a line after line scan. In Figure 6 is shown a chain of delays consisting:

• a first delay line 60 and a second delay line 63 which each delay the data for the duration of a line minus that of a pixel,
• a first register 61 and a second register 62 which each delay the data by the scanning duration of a pixel.

la donnée N est à la sortie de la ligne à retard 63 et

la donnée S est à l'entrée de la ligne à retard 60.

Thus after three scanning lines, the data of the pixels N, S, E, W are distributed as shown in FIG. 6:

data N is at the output of delay line 63 and

data S is at the input of delay line 60.

An adder 64 and an adder 65 operate the additions S + E and W + N respectively. The two results are added in an adder 66. This last result must be subtracted from 4 times the value of the central data C. This occurs at C between the two registers 61 and 62. To multiply it by 4, just shift it by 2 bits at the input of subtractor 67 which receives the output of the adder 66. The output of the subtractor 67 delivers the Δ distortion function which is calculated for all pixels of each current block. Among all these pixels, select a limited number to constitute the pixels of reference using the sorting unit 51. According to one mode preferential we sort in the set of all the pixels of a block current the pixels for which the function of distortion Δ is either maximum or minimum and we apply this sorting in case of zoning by making only two zones contiguous have positive alternating Δ functions and negative.

• un sélecteur de signe 70 qui est contrôlé par l'unité de balayage 27,
• une batterie de registres 71 stockant des valeurs maximum/minimum pour chaque zone,
• un comparateur 72 qui détermine pour chaque zone la valeur à conserver soit maximale soit minimale selon le signe requis.

For this, the sorting member 51 includes:

• a sign selector 70 which is controlled by the scanning unit 27,
• a battery of registers 71 storing maximum / minimum values for each zone,
• a comparator 72 which determines for each zone the value to be kept either maximum or minimum according to the required sign.

The addresses of the selected reference pixels are stored in the address register 52 which receives the ADP pixel addresses and unit ADZ area addresses scan 27.

The scanning unit 27 gives the rate of signs alternating + and - assigned to each zone. When the sign positive is assigned to an area, the result from the organ computational 50 is taken with its sign and then compared to the value stored for the same area in the battery of registers 71. If the last value is the highest it replaces the old value in the register bank 71. Simultaneously the address of the corresponding pixel ADP is stored in address register 52.

When the negative sign is assigned to another zone, the result from the calculation unit 50 is taken into account reversing its sign. It is compared to the previous value stored for the same other zone in the register bank 71. Because of the sign inversion, the same comparator operates in the same way in both cases by determining a maximum.

At the end of the scan, the address register 52 contains the sorting result, that is, for each zone, the address of the selected pixel, having the maximum distortion or minimum area.

The diagram in Figure 6 corresponds to the calculation of the distortion function relating to FIG. 4A. In applying the same method, the skilled person can without difficulty establishing the diagram of the calculation unit operating the calculation of the examples of distortion functions in FIGS. 4B, 4C and 4D.

To achieve a compact implementation and regular according to this aspect of the invention it is interesting that the correlation device operates on blocks of N x N pixels, with N = 16, from which n are selected reference pixels, with n = 8.

Until now the number of symbols of reference in a current block was limited and predetermined to advance, the selection is made on the position of the pixels selected in the current block. The number itself can be determined in many ways. When the number of pixels per current block is low it is possible perform preliminary tests depending on the type of images (therefore of the configuration) to process. We reduce to each test the number of selected reference pixels, we calculates an error function between the initial image and the restored image and we determine this number according to the level acceptable errors. Same number can apply to several types of images.

When the number of reference pixels per block current increases the method by preliminary tests can become tedious. It is therefore preferable to analyze the images based on characteristics that are representative of the pixel distribution. In this case the invention recommends using the technique called "analysis in main components "of said distribution (taken here in the mathematical sense of the term). We calculate the values own of the correlation matrix relating to this distribution and the corresponding variances among which we determine the number of the highest variances sufficient to correctly represent the image. It's that number of variances which is chosen as a limited number of reference pixels. This calculation can be done beforehand for standard images then used in the correlation. But it is also possible to perform it adaptively on the images supplied to the correlation. So dynamically the limited number of pixels of reference can be adapted to the images to be processed in order to operate more precise treatment. This calculation is performed at a rate which depends on the images to be processed by a neural network which implements the "component analysis" technique ". This is shown in the diagram in Figure 7 which presents the same elements as figure 2 with in addition a neural network 29 which receives the blocks of images to process and which provides the selection unit 26 with the number limited reference pixels to select.

Figure 8 shows the diagram of a structure classic of a correlator. It consists of networks systolic of a.b cells where a and b are the dimensions of the search window. Each cell contains an R register which stores the value of a sample and a processor elementary P, linked to the register R, which calculates the function of correlation. The correlator has as many cells as the window contains pixels to analyze. Each cell has a determined position in relation to the window, therefore a its own displacement vector.

Figure 8 shows a 9 cell correlator. A first row contains cells R ₁₁ / P ₁₁ to R ₁₃ / P ₁₃ , a second row contains cells R ₂₁ / P ₂₁ to R ₂₃ / P ₂₃ , a third row contains cells R ₃₁ / P ₃₁ to R ₃₃ / P ₃₃ . Each register is connected to its four closest neighbors by 3 receiver links and 3 transmitter links. Thus, for example, the register R22 receives data from the registers R ₁₂ , R ₂₁ , R ₃₂ . The same register R ₂₂ delivers data to the registers R ₁₂ , R ₂₃ , R ₃₂ . Each register successively contains the data relating to all the pixels of a block to be analyzed. The processor assigned to said register calculates each time the correlation function between this datum and the ECH value of the current sample which is distributed to all the processors. If c and d are the dimensions of the block to be analyzed, cd cycles are required for each processor to accumulate the total of the distortion function relating to a block to be analyzed. At each cycle, the value of the sample of the current block is broadcast to all the processors and the contents of the registers R are shifted by one step so that for all processors there is always the same displacement D between the data in the register R and the data broadcast. Additional members (not shown) make it possible to determine the processor delivering the best correlation and to deduce therefrom the displacement vector by the position of this best processor in the network. Ab cells operating on cd cycles are therefore required.

In the structure of figure 8 a processor is assigned to a register to process all the pixels of a current block. According to the invention this number of pixels to process is limited hence arrangements are possible for get a gain either on speed for a size of substantially analogous hardware, i.e. on the size of the hardware for a substantially constant speed.

Substantial advantages are obtained when the current block is subdivided into zones. FIGS. 9A, 9B schematically represent the processing which is carried out. FIG. 9B represents a current block 22 which is subdivided into 6 zones, for example zone 22 ₁ , themselves formed of pixels. A zone contains 9 pixels in this example from which the reference pixels are selected.

Furthermore, in an image 10, a window to be processed 13 is represented. The processing of the entire image consists of performing successive treatments on a series of such windows. As previously described, the correlation calculation consists in correlating pixel by pixel the current block 22 with each block to be treated constituting the window, for example block 12. All the blocks to be treated included in the window are obtained by sliding a step of 1 pixel successively in both directions. Block 12 slides one pixel successively according to block 12 ₁ (dots) then according to block 12 ₂ (cross). They are slightly offset vertically to be able to represent them. In the case where the current block 22 is subdivided into zones, the correlation calculation with the blocks to be processed takes place at two levels.

We first consider the reference pixels of a specific area, for example those of area 22 ₁ of the current block (top left).

We calculate the correlation for the pixels of each block to be treated (in a window) having the same relative position in the block, i.e. for the same area top left in the block. This produces results intermediaries concerning a particular area for all blocks. The calculation then relates to the following area according to the same process.

• première zone :
  • premier bloc
  • deuxième bloc
  • . . . . . . . . . . . . . . .
• deuxième zone :
  • premier bloc
  • deuxième bloc
  • . . . . . . . . . . . . . .
• puis zones suivantes
• puis fenêtres suivantes

The treatment is schematized as follows:

• first zone:
  • first block
  • second block
  • . . . . . . . . . . . . . . .
• second area:
  • first block
  • second block
  • . . . . . . . . . . . . . .
• then next areas
• then next windows

The flow of calculations in this order has the advantage of reducing data exchange between registers which simplifies the architecture and reduces calculation time.

Figure 10 shows the basic organs necessary for the development of correlation calculations. The diagram represents only a processor and a register but the situation is the same for other processors and other registers.

Any register 40 is connected to other neighboring registers, for example in the four directions N, S, E, W by bidirectional or unidirectional buses depending on the architecture used. These data transfers between the registers are controlled by control means 71 common to all the registers. A processor 72 is connected to several of these registers by connections I ₁ , I ₂ , I ₃ ... which transmit the data of the registers.

This number of connections also depends on the architecture used. The processor 72 operating the calculations in successive stages must temporarily store intermediate results relating to the zones themselves subdivided according to the blocks. These intermediate results are stored in storage means 73.

When all the intermediate results have been obtained they are added in summation means 74 to deliver all the results for each block intermediaries relating to all the zones of said block. We thus obtains a series of correlation values relating to the different blocks from which is selected the one which, having the best correlation, constitutes the block whose position constitutes the new position of the block running in the picture.

FIG. 11 represents the diagram of part of a correlator according to the invention which favors a reduction in the number of cells. In this case, a processor P is no longer associated with a single displacement vector, but it is made to operate for several displacement vectors, for example f displacements (therefore f registers). In this case there are ab registers R and ab / f processors. When this value f is equal to the number of points per zone, ie f = cd / n, we can take advantage of some of the arrangements already presented. In this case each register is linked to its nearest neighboring registers and any processor (for example P ₈₁ ) receives the data stored in the surrounding registers (respectively R ₈₁ , R ₈₂ , R ₉₁ , R ₉₂ ). With, for example, areas of 4 pixels (FIG. 11) from which 1 reference pixel is selected, to process a window of 4 x 4 pixels, 16 registers and 4 processors are then required. All processors receive a selection signal SEL which indicates the reference pixel selected in the area. This architecture is easily generalized to the case of zones having k pixels from which a single reference pixel per zone is selected. Each processor delivers the results of its calculation on its own output, for example S81 for the processor P81. More generally when a and b are the dimensions of the window, c and d the dimensions of the current block with n reference pixels selected by zone the correlator has ab registers, ab / f processors and operates on cd cycles.

FIG. 12 represents the diagram of a repetitive part of a correlator according to the invention which favors the gain in speed in the case of a division of the current blocks into zones. This diagram corresponds to areas of 4 pixels from which a single reference pixel is selected to operate the correlation. The processing which is to be carried out for a particular zone will be carried out by the members located inside the discontinuous reference 90. There are four registers R ₅₁ , R ₅₂ , R ₆₁ , R ₆₂ all connected to the same processor P ₅₁ . Each register is no longer directly connected to the register which precedes it and to that which follows it as in FIG. 8 but the connections both in rows and in columns skip every other register. This arrangement follows from the fact that by selecting a single pixel out of 4 pixels per zone, one pixel out of 2 must be skipped (Figure 3D) in both directions.

A SEL selection signal allows operating the processor on one of the four registers that surround it in which are stored at a given time the values of 4 pixels from the area. In a more general case when the block contains c.d pixels from which n pixels of reference so c.d / n connections between each processor are required and neighboring registers. Sample ECH value is simultaneously broadcast to all processors for calculate the correlation. With a window of a.b blocks, it a correlator of a.b cells operating on n cycles is required. Each processor delivers the result of its calculation on a own output, for example S51 for the P51 processor.

Claims (13)

1. A correlation device acting on input signals which represent symbols organized in blocks in accordance with a regular configuration, comprising :

   a correlator (25) determining a correlation between the symbols of a current block (22) and symbols (12) having identical positions in a sequence of blocks to be processed, coming from said configuration,

   and selection means (26) selecting, from the symbols of said current block, a limited number of reference symbols to which the correlation relates, this number being limited to a value which is smaller than the number of symbols per block, characterized in that the selection means (26) select, in said current block, n reference symbols with n being at least equal to 1, the position of a symbol being chosen:

   either as a function of the data content of said current block,

   or in accordance with a random selection,

   or for a first portion preferably as a function of the data content and for a second portion according to a random selection.
2. A device as claimed in Claim 1, characterized in that the same random selection is used for a plurality of current blocks for selecting the position of the n randomly selected reference symbols in each block.
3. A device as claimed in Claim 1, characterized in that said position adapted to the data content is determined from a distortion function calculated for each symbol of the current block with respect to the surrounding symbols, the n selected reference symbols being chosen from the symbols of the current block whose values of the distortion function differ most from those related to the surrounding symbols.
4. A device as claimed in Claim 3, characterized in that said distortion function is determined by calculating a laplacian relating to a characteristic suited to the input signal.
5. A device as claimed in any one of Claims 1 to 4, characterized in that the current block is subdivided into several zones, the selection means acting separately on each zone, the correlation acting on the totality of the selected reference symbols of the current block.
6. A device as claimed in Claim 5, characterized in that the reference symbols of a zone are selected in such a way that the distortion function is alternately maximal and minimal for contiguous zones.
7. A device as claimed in any one of Claims 1 to 6, characterized in that it includes a neural network which adaptively determines the number of reference symbols as a function of the data content by calculating suitable values of the correlation matrix and the variances, said number of symbols being equal to the number of highest variances which are characteristic of the set of input signals.
8. A device as claimed in any one of Claims 1 to 7, characterized in that the symbols organized in a regular configuration are pixels constituting an image, the correlation device acting on blocks of NxN reference pixels, with N being larger than 1, from which the n reference pixels are selected with n < N², the correlations being determined between a current block and blocks of a window taken in the image, the current block being obtained from a preceding or a subsequent image to determine the displacement vectors of the image blocks.
9. A device as claimed in any one of Claims 1 to 7, characterized in that the symbols are binary encoded values representing a unidimensional acoustic signal configuration, the correlation device acting on current blocks enclosing several of said values from which reference values are selected in a limited number and in position.
10. A device as claimed in any one of Claims 1 to 9 which effect a displacement vector calculation related to a sequence of data blocks to be processed contained in a window, characterized in that the correlator effects a correlation calculation using current data blocks subdivided into p zones with n/p selected reference symbols per zone, the correlator comprising :

    a systolic network of registers in which pixel data of the window are transferred from one register to the other,

    a plurality of processors, each processor effecting a calculation of the correlation of at least one displacement vector,

    the device making operative a first calculating stage during which :

    a) each processor performs a correlation calculation successively on the n/p selected reference symbols of a first zone, said calculation successively bearing on all the displacement vectors which said processor is to calculate,

    b) storage means store the intermediate results supplied by the processors at the end of a),

    c) transfer means transfer the content of the registers acting on a zone to subsequent registers acting on a next zone,

    said first calculation step being thereafter reiterated for the subsequent p-1 zones,
    summing means determining the sum of the intermediate results related to each block to be processed to supply the correlations of said sequence of blocks.
11. A device as claimed in Claim 10, characterized in that the symbols organized in a regular configuration are pixels constituting an image, the correlator acting on a bidimensional pixel window having current blocks subdivided into rectangular zones containing k pixels, with a single reference pixel being selected per zone, where k>1, a bidimensional matrix of registers storing, at a given instant, the pixels of said window in accordance with the same lay-out, k registers being assigned to the processing of a zone, rows of said registers exchanging pixels of the window in the two directions of the matrix, a processor being assigned to the consecutive processing of k displacement vectors while operating with k registers containing pixels belonging to the same zone, the serial numbers of said pixels being consecutive in the two directions, a selection signal causing each processor to operate, for each displacement, with the register data corresponding to the selected reference pixel of the current zone.
12. A device as claimed in Claim 10, characterized in that the correlator acts on a bidimensional pixel window having current blocks (22) subdivided into zones (22₁) containing 4 pixels each, with a single reference pixel being selected per zone, a bidimensional matrix of registers storing, at a given instant, the pixels of said window in accordance with the same lay-out, four registers (R₅₁, R₅₂, R₆₁, R₆₂) being assigned to the processing (90) of a zone, rows of said registers exchanging even pixels of the window in both directions of the matrix and further register rows exchanging odd pixels of the window, this exchange being unidirectional in one direction of the window and bidirectional in the other direction of the window, a processor (P₅₁) being assigned to the processing of each displacement vector and operating with the four registers containing even and odd pixels of the zone, the serial numbers of said pixels taken two by two being consecutive in each direction, a common selection signal (SEL) causing the processors to operate with the data of the registers corresponding to the selected reference pixels in the current block.
13. A method of processing images formed from pixels organized in blocks in accordance with a regular configuration, the method comprising the following steps :

    determining (25) a correlation between the pixels of a current block (22) and pixels (12) having identical positions in a sequence of blocks to be processed, coming from said configuration,

    selecting (26), from the pixels of said current block, a limited number of reference pixels to which the correlation relates, this number being limited to a value which is smaller than the number of pixels per block, the method being characterized in that it comprises the following step :

    selecting (26), in said current block, n reference pixels with n being at least equal to 1, the position of a pixel being chosen:

    either as a function of the data content of said current block,

    or in accordance with a random selection,

    or for a first portion preferably as a function of the data content and for a second portion according to a random selection.

Images