WO1997008654A1

WO1997008654A1 - Method of assessing the movement of an object in a sequence of images

Info

Publication number: WO1997008654A1
Application number: PCT/DE1996/001493
Authority: WO
Inventors: Luc De Vos
Original assignee: Siemens Aktiengesellschaft
Priority date: 1995-08-28
Filing date: 1996-08-08
Publication date: 1997-03-06
Also published as: AR003370A1

Abstract

Described is a method of assessing the movement of an object in an image sequence by examining the object, certain sets of image dots being scanned and compared with each other. The invention calls for a set of image dots to be used which is limited by the outline (101) of the object (10) itself so that, compared with prior art methods, the number of image dots scanned is reduced and it is possible to carry out the method in real time. Also described are VLSI architectures based on the invention.

Description

description

Method for estimating a movement of an object in a sequence of images

The invention relates to a method for estimating a movement of an object in a sequence of images according to the preamble of claim 1.

In the case of an already proposed method of this type, in the predetermined image, in which the pixels are arranged in a matrix in rows and columns, a block-shaped pixel set is selected which contains the pixels and pixels representing the object and which contains a rectangular or square surrounding the object Show outline. Correspondingly, the pixel set defined by the selected pixel set and each further pixel set of the other image are block-shaped and have a rectangular border made of pixels.

To determine the determined deviation value, the pixels of the selected pixel set of the predetermined image and the pixels of the defined pixel set and each further set of the other image are scanned and the pixel values of all the pixels of each of these sets are determined.

The scanning of the pixels is carried out in a meandering manner in each pixel set. The scanning can be carried out column by column or row by row in such a way that all the pixels of each column or row, including the pixels representing the rectangular border, are detected and the pixels of respectively adjacent columns or rows are scanned in opposite directions to one another. The deviation value for the defined and each further pixel set of the other image is determined in such a way that for each pixel of the selected pixel set of the predetermined image and the reversibly uniquely assigned pixel of the relevant pixel set of the other image, the difference between the pixel values is The two pixels are determined and the difference amounts of all pairs of pixels of these two sets of pixels assigned to one another are summed.

To reduce the number of difference amounts to be determined, the rectangular border is selected so that it encloses the object, which often differs greatly from the rectangular shape, as closely as possible.

This is an advantage of the methods of the type mentioned at the beginning, which can be referred to as object-based motion estimation methods, over known other motion estimation methods, which are called "block matching methods" or block-based methods.

In the block-based method, unlike the object-based method, a simplified subdivision of the images is used, a subdivision into adjoining block-shaped pixel sets having a fixed size being established.

Similar to the object-based pixel sets of the type specified above, in the block-based method, due to a reversibly unique assignment between the pixels of an image and the pixels of every other image, each block-like pixel set of the one image in each other image has a corresponding block-like one assigned amount of pixels defined.

A deviation value is determined for each pair of block-shaped picture element sets of two pictures assigned to one another, in that, similarly to the proposed object-based method, for each pair of picture elements of these two block-shaped picture element sets assigned to each other, the difference between the picture element values of these two picture elements is determined and the difference amounts of all pairs of picture elements of these two picture element sets assigned to one another are summed.

The pair of pixel sets with the relatively smallest deviation value denotes the position of the selected object in the other image, into which this object has moved from its position in the one image.

A disadvantage of the block-based method lies in the fact that the boundaries of the block-shaped pixel sets can only approximate the boundaries of the object. A block-shaped set of pixels can contain partially pixels of the moving object and partially pixels of the background or can even consist of parts of different moving objects. In such cases, a block-based method delivers incorrect results for some of the pixels.

For these reasons, preference is given to object-based motion estimation methods. In particular for new coding methods that try to get by with extremely low transmission rates (eg for the MPEG standard), a significant coding gain can be expected from the transition from block-based to object-based motion estimation methods.

The object-based method according to the invention as claimed in claim 1 has the advantage over the object-based method already proposed that, in order to determine the deviation value for the defined and each further pixel amount of the other image, as a rule significantly fewer pixels have to be scanned . This advantageously enables the object-based movement estimation method to be carried out in real time. In the method according to the invention, the selected pixel set advantageously coincides with the object itself, ie the pixels of the selected pixel set are the pixels of the object itself and no further pixels of the predetermined image, as is the case with the object-based method already proposed is when the block-shaped number of pixels and the object contained therein differ.

In the method according to the invention, to determine the deviation value, essentially only the pixels of the selected pixel set of the predetermined image and the pixel sets of the other image determined by this quantity have to be scanned. Only under certain circumstances, for example in the case of very irregular objects, can it occur that a few pixels have to be scanned which are outside the relevant pixel set and which are to be masked out, i.e. they must not be used to determine the deviation value.

Preferred and advantageous refinements of the method according to the invention emerge from claims 2 to 14.

Preferred and advantageous devices for carrying out the method according to the invention emerge from claims 15 and 16, wherein, as described later, the device according to claim 15 accumulates locally and the device according to claim 16 globally. These devices advantageously allow the method according to the invention to be carried out in real time and, moreover, can be implemented as a VLSI architecture. Preferred and advantageous configurations of these devices and methods for their operation are evident from further subclaims. Current CMOS technologies can advantageously be used to implement efficient VLSI implementations for real-time execution of the method according to the invention.

The invention is advantageously suitable for both local and global accumulation and can be used, inter alia, with MPEG coding. In general, the invention can be applied to image coding with low, preferably extremely low transmission rates.

Preferred and advantageous refinements of the device according to the invention emerge from claims 14 to 18.

The invention is explained in more detail in the following description using the figures as an example. Show it:

1 shows a schematic representation of a temporal sequence of images composed of pixels with a selected number of pixels, the position of which changes from image to image and which represents a selected, moving object,

FIG. 2 shows an enlarged, detailed schematic illustration of a predetermined image of the sequence according to FIG. 1, in which the selected set of pixels is highlighted,

3 shows, in the same representation as in FIG. 2, another image of the sequence according to FIG. 1, in which the selected amount of pixels offset from the predetermined image is also highlighted,

FIG. 4, in the same representation as in FIG. 3, the other image, in which the defined number of pixels is clarified and a search area is highlighted, FIGS. 5 to 8, each in the same representation as in FIG. 4, the other image, the search area being highlighted in each case and a different further set of pixels being illustrated in each figure,

FIG. 9 shows, in the same representation as in FIG. 1, the predetermined image with a meandering scanning path, on which the selected amount of pixels is scanned, with pixels to be masked being emphasized, and

FIG. 10 shows a block diagram of a device for carrying out the method according to the invention in a schematic illustration.

In Figures 1 to 9, the temporal sequence of images is designated 1. The images themselves are generally designated 2, the pixels generally 20, the selected object 10 and the outline of the object 101.

The predetermined picture is labeled 2 -] _ and the other picture is labeled 2 ^. The number of pixels selected in the predetermined image 2- _] _ and representing the selected object 10 is designated by 21 and its pixels by 20ι_ and 20n. The defined pixel set of the other image 22 is designated 22 and its pixels with 2O2 and 2O21.

The search area pixel set in the other image 22 is designated with 23 and its pixels with 2O3.

The further picture element sets of the other picture 22 are with 24, 25, ... 20j, ... 20 _n and their picture elements with 20 ₄ and 204 _! , 20 ₅ and 2θ5 _{1 (} ... 20j and 20j ₊₁ , ... or 20 _n and 20 _{n + 1} respectively. The term "each pixel set of the other image 22" is used in the following not only includes every further pixel set 24, 25, ... 2j, ... or 2n of the other image 22, but also their defined pixel set 22 .

The pixels 20 _u , 20 ₂ ι, 20 ₄₁ , 20 ₅₁ , ... 20j _{1 #} ... 20 _nl each designate the outline pixels defining the outline 101 of the relevant pixel set 21, 22, 24, 25, ... 2j , ...respectively. 2n belonging to this set.

The temporal sequence 1 of images 2 shown in FIG. 1, which is strictly a section of a larger sequence of such images, begins, for example, with image 2 at time tg. This first image 2 is successively followed at times tτ_, t2, t _ß , ... t _n more pictures 2.

Each of these images 2 can be selected as the predetermined image 2 --_ ge. Each image 2 following the predetermined image 2- _j _ or preceding the predetermined image 2 _] _ can in principle be selected as the other image 22. In FIG. 1, for example and without restricting generality, the image 2 at the time t-j_ is selected as the predetermined image 2 -] _ and for example the image 2 at the next time t2 as the selected image 22.

The pixels 20 of each of two images 2 must be reversibly clearly assignable to each other, i. H. , in each of these two images 2 only the same number of pixels 20 can be considered. For the sake of simplicity, however, it should not necessarily be assumed that each image 2 has the same number of pixels 20.

It is also expedient if the pixels 20 are arranged in the same way in each image 2, for example and as shown in FIGS. 1 to 9, in a matrix form in horizontal rows 3 - and _ and vertical columns 32, the images 2 preferably each having the same number of rows 3- _j and the same number of columns.

For the sake of clarity, the pixels 20 in FIG. 1 are arranged in a smaller number and in a larger grid dimension from one another. At each intersection of a line 3 -] _ represented by a horizontal line with a column 32 represented by a vertical line there is a pixel 20, of which only a few by small ones in the images 2 of FIG Circles are illustrated.

In FIGS. 2 to 9, the pixels 20 are shown in larger numbers and in a smaller grid size and are therefore more practical (for example, a field of a television picture of the PAL system has 312x864 or 313x864 pixels). In these figures, each of the adjoining small squares each represents a pixel 20.

Each pixel 20 in each image 2 has a pixel value, which can be a brightness and / or color value, for example, and can differ from pixel to pixel. The distribution of the pixel values over the pixels 20 in an image 2 depends on the objects shown and can change from image 2 to image 2 of sequence 1, which is particularly the case with moving objects.

The pixel values of the individual pixels 20 are not shown in FIGS. 1 to 9 for the sake of clarity. If picture elements 20 are highlighted or clarified in relation to other picture elements 20, this serves only to indicate picture element quantities and not picture element values.

In the method according to the invention, the moving object 10 is selected in the predetermined image 2- _] _ by a

Pixel set 21 is selected, the pixels 20τ_ and 20H of this object 10 and its outline. The outline of the selected object 10 is not a rectangle surrounding the object 10, as in the object-based motion estimation method already proposed and mentioned above, but according to the invention a predetermined or predetermined rectangle of the selected pixel set that directly delimits the object 10 and ₁₁ 21 defined outline 101 of this object 10 itself. In FIG. 2, the pixels 20-j_ of the pixel set 21 including their outline pixels 20π are highlighted by narrow hatching, the outline pixels 20H additionally being highlighted by a central point.

The selected object 10 with its outline 101 can be determined by a suitable segmentation algorithm, according to which the pixels 20-j_ and outline pixels 20π of the selected pixel set 21 are defined.

It is assumed that the object 10 is moving, i.e. that its position in the images 2 of the time sequence 1 changes, e.g. B. from image 2 to image 2, for example, as shown in Figure 1.

The position of the object 10 in an image 2 can be determined, for example, by image-related horizontal and vertical coordinates of a particular pixel of the pixel set representing the moving object 10, for example by a leftmost pixel of this pixel set. The coordinate origin is, for example, the lower left corner 0 of each image 2 shown in FIGS. 1 to 9.

For example, as shown in FIG. 2, in the predetermined image 2- _j _ this leftmost pixel 20 _1:] _ of the pixel set 21 has the coordinate h - * _ in the horizontal direction and the coordinate in the vertical direction v -] _. In the other image 22, as shown in FIG. 3, the leftmost pixel of the set of pixels representing object 10 has the horizontal coordinate Ϊ12 and the vertical coordinate V2, with 1-2> hη_ and V2> vι_ being assumed, for example . The latter means that the object 10 has shifted both horizontally and vertically by a certain amount and therefore has a different position in the other picture 22 than in the picture 2-j_

Although the object 10 itself can change in shape and size as well as in the distribution of brightness and / or color over its pixels 20 during its movement and the invention can also be used with such variable objects 10, it is assumed for the sake of simplicity that that the object 10 does not change even during its movement.

If the object 10 itself does not change, when it moves from image 2 to image 2 only the pixel sets representing the object 10 change in the sense that the pixel sets representing the object 10 of different images 2 do not at least partially have the same pixels .

Due to the reversible unambiguous association between the pixels 20 of the predetermined image 2 _j _ and Bildpunk¬ th 20 of the other image 22 is the point set by the selected Bild¬ 21 of the predetermined image 2-j_ in the other image 22, to be exact in each of predetermined Image 2 -] _ various image 2, each defining a pixel set 22, the pixels 2O2 and 2O2 _{1 of} the object 10 and its outline 101 each in the same shape, size and position as in the predetermined image 2 --_, as a rule, however, not in the distribution of the pixel values over the pixels 2O2 and 2O2 _{1 of the} defined set 22 in the selected pixel set 21 of the predetermined image 2- _j _ and determined by the object 10. This defined pixel set 22 is shown in FIG. 1 and in FIG. 4 only for the other image 22, the pixels 202 and outline pixels 202 ₁ being illustrated in FIG. 4 by small central circles and the outline pixels 2021 additionally by reinforced circles are highlighted.

In the other image 22, the search area pixel set 23 is defined from the pixels 2O3 not contained in the defined pixel set 22 of the other image 22, which is highlighted in FIGS. 4 to 9 by relatively wide hatching and preferably that defined by the outline pixels 2O2 _{1 of} the defined ones Pixel set 22 completely surrounds certain outline 101 of this pixel set 22.

In addition, in the other image 22, in addition to the defined pixel set 22, one or more different further pixel sets different from the defined pixel set 22 are determined such that the pixels of each additional pixel set are reversibly uniquely assigned to the pixels of the defined pixel set 22 and one or contain several pixels that coincide with pixels 20 _{3 of} the search area pixel set 23, and that each of these further pixel sets each has the selected object 10 and its outline 101 in the same shape and size as in the predetermined image 2-j_, but offset into the search area pixel set 23.

Such different further pixel sets are shown by way of example in FIGS. 5 to 8 and are illustrated by larger central circles in the pixels in comparison with the small circles of pixels 2O2 and outline pixels 2O2 _{1 of} the defined pixels 22 in FIG again the outline pixels are highlighted by reinforced circles. The illustrated in Figure 5 further set of pixels 24 with the pixels 20 _4, and the outline 101 representing Umriß¬ pixels 2O41 has the h of the coordinates --_ and v --_ the defined set of pixels 22 different coordinate H.4 and V _4, where, for example, I14 <h] _ and V ₄ <v -] _ is assumed, ie the further pixel set 24 is offset from the defined pixel set 22 in the direction obliquely to the bottom left into the search area pixel set 23.

The further pixel set 25 shown in FIG. 6 with the image pixels 205 and the outline 101 representing outline pixels 205 ₁ has the coordinates hs and V5, where, for example, I15 <I1 ₄ <h ^ and V ₄ <V5 <v -] _ is assumed, ie the further pixel set 25 is wider than the defined pixel set 22 in the horizontal direction and less in the vertical direction than the further pixel set 24.

The shown in the Figure 7 more pixel amount 2j with the view points 20j, and the outline 101 representing Umriß¬ pixels 20ji has, for example, the coordinates hj and VJ, wherein for example hj> _hj _ and VJ> v ^ _ is believed that this further Pixel set 2j is offset from the defined pixel set 22 in the direction obliquely to the top right into the search area pixel set 23. This only applies to a specific j, which otherwise denotes any natural number between 5 and the natural number n.

The further pixel set 2n according to FIG. 8, which was assumed to be the last, with the image pixels 20 _n and the outline 101 representing the outline pixels 20 _n - _] _ has the coordinates h _n and v _n , where, for example, h _n > hι_ and v- _j _ <v _{n is} assumed, ie this last further pixel set 2n is offset obliquely to the right into the search area pixel set 23 compared to the defined pixel set 22. In practice, the procedure is such that as many directions as possible from the defined set of pixels 22 are detected by further sets of pixels 24, 25,... 2j,... 2n in order to cover the entire environment of the defined set of pixels 22 to scan evenly.

The outline 203 of the search area pixel set 23 defined by outline pixels 203 is generally not rectangular, as shown in FIGS. 4 to 9, because it outlines the combination of the number of further pixel sets 24 to 2n which can be freely selected and is thus determined by the mostly irregular shape of the object 10 itself. The search area pixel set 23 can within its limits by specifying a maximum offset Δh of the defined pixel set 22 horizontally to the left and right from its original position in the other image 22 and a maximum offset Δv of the defined pixel set 22 vertically up and down from its original Location in the other picture 22 can be characterized.

In the method according to the invention, the pixels 20 --_ and outline pixels 20n of the selected pixel set 21 and the pixels 2O2, 20 ₄ , 2O5, ... 20j, ... or 20 _n and outline pixels 20 ₂ ι, 20 ₄₁ , 205 -j_, ... 20j -] _, ... or 20 _{nl of} each pixel set 22, 24, 25, ... 2j, ... or 2n of the other image 22 are scanned for their pixel values, whereby these Scanning at each of these pixel sets 21, 22, 24, 25, ... 2j, ... or 2n essentially does not exceed the outline pixels 20H, ²⁰ ₂₁ , ²⁰ 4 ₁ » ²⁰ 51 '••• ²⁰ jl '••• or 20 _{nl of} the respective pixel set 21, 22, 24, 25, ... 2j, ... or 2n defined outline 101 of the object 10 is made.

For each pixel set 22, 24, 25, ... 2j, ... or 2n of the other image 2 ₂ , a specific deviation value Δ ₂ Δ ₄ , Δ ₅ , ... Δ _j , ... or Δ is used _{n is} determined, which is a measure of a deviation of the pixel values of this set 22, 24, 25, ... 2j, ... or 2n of the pixel values of the selected pixel set 21 of the predetermined image 2 -] _.

The relatively smallest deviation becomes from the deviation values Δ2 Δ ₄ , Δ5, ... Δj, ... and Δ _n determined for each pixel set 22, 24, 25, ... 2j, ... and 2n of the other image 22 ¬ chung value, which is given, for example, by Δj with a specific j, and the pixel set 2j of the other image 22 belonging to this relatively smallest deviation value Δj is defined as the position of the selected object 10 in the other image 22, into which this object is expressed its position in the predetermined image 2 -] _ has moved.

To determine the deviation value Δ2 _t Δ ₄ , Δ5, ... Δj, ... or Δ _n , the procedure is preferably such that for each of a pixel 20- _] _ or 20n the selected pixel set 21 and this pixel 20j_ or 20H associated pixel 2O2 or 2O2 ₁ , 2O4 or 204 _! , 2O5 or 205 ₂ , ... 20j or 20jι, ... or 20 _n or 20 _n ι of a pixel set 22, 24, 25, ... 2j, ... or 2n of the other image 22 existing pixel pair each a value which is dependent on the two pixel values of this pixel pair is determined in a predetermined manner, and that the value for all pixel pairs of these two sets 21 and 22, 21 and 24, 21 and 25, ... 21 and 2j, .... or 21 and 2n determined values for a deviation value Δ2 _# Δ ₄ , Δ5, ... Δj, ... and Δ _{n of} these two quantities 21 and 22, 21 and 24, 21 and 25, 21 and 2j, ... or sum totaling 21 and 2n.

The specific value dependent on the two pixel values of a pixel pair can, for example, | ab | ^m , where a is the pixel value of the relevant pixel of the selected pixel set 21 of the predetermined image 2- _j _, b the pixel value of the pixel assigned to this pixel of the relevant pixel set 22, 24, 25, ... 2j, ... or 2n of the other image 22 and m is an arbitrary number, which, however, applies to all pixel pairs of all pixel sets 21, 22 and 24 to 2n is the same. Preferably, however, m = 1 and thus the difference | ab | between the two pixel values a and b of a pixel pair is used as the specific value dependent on these two pixel values a, b.

The pixels 20 _! and 20π of ^the selected pixel set 21 of the predetermined image 2-j_ and the pixels 202 and 2O21, 2O4 and 2O4 ₁ , 2O5 and 205 assigned to these pixels 20- _] _ and 20-j_ι !, ••• ²⁰ j ^{and 20} jl '• •• • ^Dzw - ²⁰ n ^{and 20} nl of each pixel set 22, 24, 25, ... 2j, ... resp. 2n of the other image 22 are preferably scanned one after the other in one and the same predetermined sequence, so that the pixels 20-j_ and 20H ki ^{3 20} n ^{and 20} n _{l of} all pixel sets 21 to 2n are scanned in the same predetermined sequence .

The predetermined order in which the pixels 20 --_ and

20χι, 202 and 2O21, ²⁰ 4 ^{and 20} 41 » ²⁰ 5 ^{and 20} 51- • • • ²⁰ j ^and ² 0jχ, ... or 20 _n and 20 _{nl of} a pixel set 21, 22, 24, 25, ... 2j,... Or 2n are preferably scanned by a meandering scanning of all the pixels of this set.

Such a meandering scanning should expediently begin at a most outer pixel of the relevant pixel set and continue from there in a certain continuation direction through this pixel set, the successive scanning of the pixels of this set alternating in the continuation direction in one Scanning direction is transverse to the continuation direction and then in the scanning direction opposite to this scanning direction.

For example, the meandering scanning can be carried out horizontally from the left at a leftmost outline pixel of the relevant pixel set with a continuation direction start to the right and, for example, the first scanning direction from bottom to top or top to bottom. A meandering scan with a continuation direction horizontally from right to left or in a vertical or oblique direction from top to bottom or bottom to top could just as well be used, for example

FIG. 9 shows an example of a meandering scan of a pixel set using the example of the defined pixel set 22. The outline pixels 2021 of this defined pixel set 22 are highlighted by small central circles.

In this example according to FIG. 9, the meandering scanning of the pixels 2O2 and 2O21 of the defined pixel set 22 begins at the individual leftmost outline pixel 2O2 ₁ with the coordinates h-ι_ and v-j_ and starts from there in the first scanning direction upwards to the immediately adjacent pixel 203 of the search area pixel set 23, from there in the continuation direction to the right to the immediately adjacent outline pixel 2O2 _{1 of} the defined pixel set 22, from there in the opposite scanning direction downwards to an outline pixel 2O2 ₁ the defined pixel set 22, from there in the continuation direction to the right to the immediately adjacent outline pixel 2O2 _{1 of} the defined pixel set 22, from there in the first scanning direction upwards to an outline pixel 2O21 of the defined pixel set 22 and via this outline pixel 2O21 to an immediately neighboring picture pu 2O3 of the search area pixel set 23, from there in the continuation direction to the right to an outline pixel 2O2 _{1 of} the defined pixel set 22, from there in the opposite scanning direction down to an outline pixel 2O21 of the defined pixel set 22 and above this outline pixel 2O2 ₁ to an immediately adjacent pixel 2O3 of the search area pixel set 23, from there in the continuation direction to the right to an un- indirectly adjacent outline pixel 2O2 _{1 of} the defined pixel set 22, from there in the first scanning direction up to an outline pixel 2O2 _{1 of} the defined pixel set 22, from there in the continuation direction to the right to an immediately adjacent outline pixel 2O2 _{1 of} the defined pixel set 22, thence in the opposite scanning direction down to an outline pixel 2O2 _{1 of} the defined pixel set 22, from there in the continuation direction to the right to an immediately adjacent pixel 203 of the search area pixel set 23, from there in the first scanning direction upwards to an immediately adjacent outline pixel 2O21 of the defined pixel set 22 and further up to the next outline pixel 2O2 _{1 of} the defined pixel set 22, from there in the continuation direction to the right to an immediately adjacent pixel 2O3 of the search area pixel set 23, from there in the en opposite scanning direction down to an immediately adjacent outline pixel 20 2 _{1 of} the defined pixel set 22 and further down to the next outline pixel 20 02 of the defined pixel set

22, from there in the continuation direction to the right to an immediately adjacent outline pixel 2O2 _{1 of} the defined pixel set 22, from there in the first scanning direction up to an outline pixel 2O2 _{1 of} the defined pixel set 22, from there in the continuation direction to the right to one immediately adjacent outline pixel 2021 of the defined pixel set 22, from there in the opposite scanning direction down to an outline pixel 202 _{1 of} the defined pixel set 22, from there in the continuation direction to the right to an immediately adjacent outline pixel 202 of the defined pixel set 22 , from there in the first scanning direction up to an outline pixel 2O2 _{1 of} the defined pixel set 22, from there in the continuation direction to the right to an immediately adjacent pixel 2O3 of the search area pixel set 23 and from there in the opposite scanning direction downwards to an outline pixel-2O2 ₁ of the defined set of pixels 22 and further down to an immediately adjacent outline pixel 20 2 _{1 of} the defined pixel set 22, at which the meandering scanning ends.

In this example it is assumed that in the first scanning direction there are only one and not two or more or the most outer pixels of the relevant pixel set, at which the meandering scanning begins. If, on the other hand, there are two or more such outermost pixels in the first scanning direction, it is expedient to start with the first of these outermost pixels in the first scanning direction.

If, for example, there were two or more left-most outline pixels 2O2 ₁ with the same horizontal coordinate h- _] _ but different vertical coordinates v- _] _, based on the example according to FIG. 9, the manned shape would be the given one in the first scanning direction vertically from bottom to top at the bottom left-most outline pixel 2O2 ₁ with the horizontal coordinate h-j_ and the relatively smallest vertical coordinate V _] _.

As can be seen from FIG. 9, during the meandering scanning of the pixels 2O2 and 2O2 _{1 of} the defined pixel set 22, some pixels 2O3 of the search area pixel set 23 are also sampled so that all pixels 2O2 and 2O21 of the defined pixel set 22 are detected. These co-sampled pixels 2O3 of the search area pixel set 23, which lie outside the outline 101 of the defined pixel set 22 and do not belong to this set 22, are highlighted by a relatively fine hatching.

The pixel values of pixels 20 that are also sampled but not belonging to a pixel set 22, 24, 25,... 2j,... Or 2n, be it that they are outside or inside the area. cracks 101 of this quantity are ignored when determining the deviation values. In the method according to the invention, advantageously only relatively few such pixels and generally only in the vicinity of the outline pixels of the relevant pixel set occur, where ignoring their pixel values is not complex and does not cause any problems. Pixels are expediently ignored by masking these points during processing

It is expedient if the specific values formed from the two pixel values of each pair of mutually assigned pixels of the predetermined pixel set 21 and a pixel set 22, 24, 25, ... 2j, ... or 2n of the other image 22 are summed in the same predetermined order as the pixels of these two sets are scanned.

The size of the object 10 determines the size of a memory for storing the scanned pixel values and / or certain values formed therefrom. In the event that a sufficiently large memory is not available or is not expedient, for example with regard to real-time processing of the pixel values and values derived therefrom, the procedure can advantageously be such that the selected pixel set 21 of the predetermined one Image 2 --_ is subdivided into one or more disjunct image sub-sets such that the image points of all these sub-sets result in all image points of the selected image point set.

Each pixel subset of the selected pixel set 21 of the predetermined image 2 ^ means, owing to the reversible, unambiguous association between the pixels 20 of the predetermined image 2 --_ and the pixels 20 of each pixel set 22, 24, 25,. ..2j, ... or 2n of the other image 22 in this pixel set 22, 23, 24, 25, ... 2j, ... or 2n of the other image 22 each defines an associated subset of pixels.

For each pair of a pixel subset of the selected pixel set 21 of the predetermined image 2- _j _ and the pixel subset assigned to this pixel subset of each pixel set 22, 23, 24, 25, ... 2j, ... or In the other image 22, a subset deviation value is determined in each case by scanning the pixels of this pair of pixel subsets, which is a measure of a deviation between the pixel values of this pair of pixel subsets.

Finally, for each pixel set 22, 23, 24, 25, ... 2j, ... or 2n of the other image 22, the measure of the deviation between the pixel values of this pixel set 22, 23, 24, 25,. .. 2j, ... 2n of the other image 22 and the pixel values of the selected pixel set 21 of the predetermined image 2 -] _ forming deviation value Δ2, Δ ₄ , Δ ₅ , ... Δj, ... and Δ _n formed by summation of all subset deviation values determined for this pixel set 22, 23, 24, 25, ... 2j, ... or 2n of the other image 22.

An intermediate result memory can be provided for the subset deviation values, in which these subset deviation values can be saved as intermediate results for further processing.

FIG. 2 shows an example of a subdivision of the selected pixel set 21 of the predetermined image 2 --_ into two disjoint pixel subsets, which when combined again result in the selected pixel set 21. The division is carried out by a horizontal line 32. All above this line 32 lying pixels 20- _j _ _ ₁₃ and 20 of the selected set of pixels 21 form an upper pixel subset and all lying below the horizontal line 32 pixels 20 --_ and ^the 20π selected pixel set 21 form a lower pixel subset. The upper pixel subset of the selected pixel set 21 is each assigned an upper pixel subset in the defined pixel set 22 and each further pixel set 22, 23, 24, 25, ... 2j, ... or 2n.

The associated top pixel subset of the defined set of pixels 22 consisting of Figure 4 from all ober¬ half of the horizontal line 32 lying pixels 2O22 ^and ² θ2i- the associated top pixel subset of the further set of pixels 24 consists of all above the hori¬ zontal line 35 Pixels 2O ₄ and ²⁰ ₄₁ - ^D: - ^e assigned to the upper pixel subset of the further pixel set 25 in FIG. 5 consist of all pixels 20 ₅ and lying above the horizontal line 36 in FIG

2O5 ₁ . The assigned upper pixel subset of the further pixel set 2j consists of all pixels 20j and 20jι lying above the horizontal line 37 in FIG. 7. Finally, the assigned upper pixel subset of the further pixel set 2n consists of all above the horizontal line 38 in FIG 8 pixels 20 _n and 20 _n ι.

Correspondingly, the assigned lower pixel subset of the defined pixel set 22 consists of all pixels 2O2 and 2O2 ₁ lying below the horizontal line 32 in FIG. 4, the assigned lower pixel subset of the further pixel set 24 consists of all below the line 35 pixels 2O ₄ and 2O ₄₁ in FIG. 5, the assigned lower pixel subset of the further pixel set 25 from all the pixels 2O5 and 2O5 ₁ lying below the horizontal line 36 in FIG. 6, the assigned lower pixel subset of the further pixel set 20H from all the pixels 20j and 20j ^ below the horizontal line 37 in FIG. 7 and the assigned lower pixel subset of the further pixel set 2n from all the pixels 20 _n and 20 _nl lying below the horizontal line 38 in FIG. 8.

The upper pixel subsets are scanned separately from the lower pixel subsets. In the exemplary meandering scanning shown in FIG. 9, this means that two separate meandering scans are carried out which do not exceed the horizontal line 32, so that only the pixels lying above the horizontal line 32 and separately from them only the un ¬ are sampled meandering terhalb the horizontal line 32 lying pixels 2O2 and 2O2. ₁

For each pixel set 22, 24, 25, ... 2j, ... or 2n, a subset deviation value is given for the upper pixel subset and separately for the lower pixel subset a subset deviation value in the determined in the manner described above. The deviation value for this pixel set 22, 23, 24, 25, ... 2j, ... or 2n results from the sum of the two subset deviation values determined for this set.

A device for carrying out the method according to the invention with local accumulation is advantageously designed such that for each pixel set 22, 24, 25, ... 2j, ... or 2n of the other image 22, one of these pixel sets 22, 24, 25, ... 2j, ... or 2n is assigned to the processor element, to which the pixel values of the selected pixel set 21 and the pixel values of the pixel set 22, 24, 25, ... 2j, ... or 2n of the other image 22 zuführ¬ bar and from the input pixel values, the average deviation Δ2, Δ ₄ Δ5, ... Aj, Δ _n between the pixel values of the selected set of pixels 21 of the voted vorbe¬ image _2] _ and Pixel values of the pixel set 22, 24, 25, ... 2j, ... or 2n of the other image 2 _{2 are} calculated. Local accumulation means that with each cycle with which the pixel values of the pixels of each pixel set of the other image 22 are processed in the assigned processor elements, only one pixel value of each pixel set of the other image 22 together with the assigned one

Pixel value of the selected pixel set of the predetermined image 2- _] _ is processed, in contrast to a global accumulation, in which at each cycle all pixel values of only one pixel set of the other image 22 together with the assigned pixel values of the selected one th pixel amount of the predetermined image 2 --_ are processed simultaneously, a processor element being provided for each pixel value of the selected pixel amount of the predetermined image 2j_ or each pixel amount of the other image 22. It should be noted that local and global accumulation has already been described in connection with the "block matching method" in Luc de Vos et al. : "Parameterizable VLSI Architectures for the Füll-Search Block-Matching Algorithm", IEEE Transactions on Circuits and Systems, Vol. 36 No. 10, October 1989, pp. 1309 to 1316. In this document, the "Type-1 Array" stands for global and the "Type-2 Array" for local accumulation.

If, in a device with local accumulation described in more detail above, a processor element array of processor elements arranged in a matrix in P rows and Q columns, i.e. Using a processor element array with PxQ processor elements, PxQ pixel sets of the other image 22, including the defined pixel set of this image, can be treated. The case P = 1 or Q = 1 is also included, i.e. a processor element field which in the former case consists of a single column having Q processor elements and in the second case consists of a single row having P processor elements.

Each processor element is preferably only in communication with the immediately adjacent processor elements, so that the communication between the processor elements is local and there is a so-called systolic processor element field. In this case, the data supply to the processor element field can only take place from the edge of the field.

In the figure 10, such a device with local accumulation is shown schematically by way of example, arranged a sy¬ stolisches processor element array 201 of a matrix form in spielsweise four lines 30- _j _ and, for example, four columns 302 and having processor elements 200 shown in each case by a square, so that a total of 16 processor elements 200 are present and therefore, in addition to the deviation value Δ2 for the defined pixel set 22, the deviation values Δ ₄ , Aς, ... Δj, ... Δ _n for fifteen further pixel sets 24, 25, .. .2j, ... 2n are determined, where n = 18 applies. In practice, however, the number of further pixel sets and thus also the number n and the number of processor elements 200 in the processor element field 201 are generally much larger.

In FIG. 10, for the sake of better distinction and only as an example in each of the sixteen processor elements 200, the reference number for that pixel set of the sixteen pixel sets 22, 24, 25, 26, 27, 28, 29, 210, 211, 212, 213, 214, 215, 216, 217 and 218 = 2n of the other image 22, which should be reversibly uniquely assigned to this processor element 200.

This exemplary assignment does not take into account the fact that in a systolic processor element field with local accumulation in neighboring processor elements, neighboring pixel sets of the other image 22 are treated, whereby neighboring pixel sets are each two pixel sets that result from one another by a defined minimal shift relative to one another. The exemplary assignment given in FIG. 10 between the processor elements 200 and the pixel sets 22, 24 to 2n = 218 des other image 22 does not correspond to the standard configuration of a systolic processor element field 201 used for this purpose.

_N for calculating the deviation values Δ2 to Δ in ei¬ nem systolic processor element array 201 at local Akku¬ advantageously the pixels 20ι and 20 _{1 mulation:]} _ of the selected set of pixels 21 of the predetermined image 2 --_ and _j these pixels 20 _ and 20H assigned pixels 2O2 and 2O21, 2O4 and 2O41, 2O5 and 205 !, ... 20j and 20jι, ... or 20 _n and 20 _n ι the defined and each further set of pixels 22, 24, 25 , ... 2j, ... or 2n each sequentially sampled in the same order.

An example of such a temporally successive scanning of the pixels of each pixel set of the predetermined and other image in a specific sequence is the meandering scanning according to FIG. 9 with respect to the defined pixel set 22 of the other image 22 shown there.

With such a scan, it is expedient for each pixel set 22, 24, 25,... 2j,... Or 2n of the other image 22 to have the pixel values of each pair of pixels that consist of a pixel 20 _] _ or 20H of the selected one Pixel set 21 of the predetermined image 2- _j _ and that pixel 20 -] _ or 20 _1: L reversibly uniquely assigned pixel 2O2 or 2O2 ₁ , 20 ₄ or 2θ4 _1; 2O5 or 2O51, ... 20j or 20j _l7 ... or 20 _n or 20 _{nl of} this pixel set 22, 24, 25, ... 2j, ... or 2n of the other image 22, correspondingly in pairs to be made available to one another and according to the predetermined specific sequence one after the other in the processor element 200 assigned to this pixel set 22, 24, 25, ... 2j, ... or 2n of the other image 22 for the relevant deviation value Δ ₂ , Δ ₄ , Δ ₅ , ... Δj, ... or Δ _n to be processed. For this purpose, in each processor element 200 the difference between the two pixel values of this pair is successively formed for each pair of pixel values assigned to one another for each pair of processor elements 200 assigned to this processor element, and successively in this order Summed up deviation value. Processor elements 200 that achieve this are known and will not be described further.

Thereafter, the sum obtained in each processor element 200 after addition of all difference amounts defines the deviation value Δ ₂ , Δ ₄ , Δ ₅ , ... Δj, ... or Δ _{n of} the pixel set 22, 24, 25, .. .2j, ... or. 2n of the other image 22, which is assigned to this processor element 200.

The calculation of the deviation values must be a real-time calculation for each pixel set 22, 24 to 2n of the other image 22, before the appearance of the image 2 next to the other image 22 in question, in this case before the time t _ß , be completed.

The pixel values of the pixels of each pixel set 22, 24, 25,... 2j,... Or 2n of the other image 22 are used to calculate the difference amounts or other specific values to form the deviation values for each of these pixel sets in a special purpose provided memory 202, from which these pixel values can be transferred to processor elements 200 in such a way that processing takes place in all processor elements 200 simultaneously.

This memory 202 is preferably designed in such a way that the pixel value data are read into it in series via one or a few gates 202-LO and are read out from it in rows or columns. It is expedient, starting from a first pixel set of the other image 22, for example the defined pixel set 22, to form the remaining pixel sets 24, 25, ... 2j, ... 2n of the other image 2 _{2 in} succession in such a way that each the following pixel set corresponds to a shift of a previous pixel set in a certain direction by a certain amount, for example by a single pixel. In the sense defined above, two sets of image points which follow one another in this way are adjacent sets of image points which are treated in adjacent processor elements of the systolic processor element field 201.

Such neighboring pixel sets always have a number of pixels in common, and since the pixel values of these common pixels already determined for the previous of these two neighboring pixel sets are then already available for the subsequent pixel set, it is advantageous to increase the computing speed and save storage capacity, the pixel values of the first pixel set of the other image 22 and for each subsequent pixel set of this other image 22, temporarily store the pixel values of the pixels of this following pixel set common to the previous pixel set in a memory 203, referred to below as an overlap memory.

The pixel values stored in the overlap memory 203 for each of these pixel sets of the other image 22 are available to the processor element 200 assigned to this bil point set for further processing, and only those pixel values of this pixel set that are not yet available and to be determined by scanning have to be available for a pixel set on this pixel set following pixel amount can be read into this processor element 200 and in the overlap memory 203. Only those pixel values not yet available for a pixel set have to be stored in the memory 202 and read from there into the processor elements 200 and the overlap memory 203. For this reason, the memory 202 need not have such a large storage capacity that the pixel values of all the pixels 203 of the search area pixel set 23, including the pixels 202 and 202 _{1 of} the defined pixel set 22 of the other image 22, can be stored, but it suffices Part of it.

With this procedure and this structure, the storage capacity can be considerably reduced and the computing speed for determining the deviation values can be increased considerably.

The processor element array 200 is preferably integrated as a VLSI architecture (VLSI stands for Very Large Scale Integration) on a substrate 100, the memory 202 and the overlap memory 203 advantageously also being integrated on the substrate 100. An intermediate result storage mentioned above, but not shown, can also be integrated on the substrate 100.

The overlap memory 203 preferably consists of a register field consisting of individual registers 203, each represented by an elongated rectangle, which, to facilitate communication with the processor elements 200, preferably consists of two mutually opposite sides 201 _] , 2012 of the processor element field 201 arranged register field sections 203 and 2032.

The memory 202 is arranged opposite a third side 20I3 of the processor element field 201 lying between the two opposite sides 201 - * _, 2012 and has, for example, a multiplexer 202 --_ on its output side facing this third side 20I3. The Loading of the memory 202 with data is preferably carried out from a memory which is not integrated on the substrate 100 and which is not shown.

FIG. 10 shows communication lines between the multiplexer 202- _] _ of the memory 202 and registers 203-j_o of the register field sections 203-j_ and 2032 ^{as well} as processor elements 200 of the processor element field 201, between processor elements 200, for the transmission of the pixel value data to be compared of the processor element field 201 and registers 203 dero of ^the register field sections 203 --_ and 2032, between processor elements 200 themselves and between registers 203ιo themselves each represented by a solid connecting line 5 between these components, an arrow on each of these lines 5 indicating the direction of communication and each of these lines 5 has a certain bit width, for example a width of 8 bits.

Communication lines for supplying the pixel value data of the individual pixels 20- _j _, 20- _] _ι of the selected pixel set 21 of the predetermined image 22 to the process elements 200 ε are represented by dash-dotted lines 6, with an arrow on each line also here 6 indicates the direction of communication. The pixel value data of the individual pixels 20χ, 20-j _ - _ of the selected pixel amount 21 of the predetermined image 2 ₂ do not require a memory on the substrate 100 in the case of local accumulation, since in each clock cycle only the pixel value of a single pixel 20- _j _ or 20H ^ er selected pixel set 21 of the predetermined image 2- _] _ is required for the entire processor element field 201, which can be loaded from an external memory (not shown).

In addition, communication lines for transmitting the deviation values determined in the processor elements 200 from the processor elements 200 to an input 300 --_ a comparator preferably integrated on the substrate 100 are device 300, which determines the smallest deviation value by comparison and outputs it at an output 3002, represented by dashed lines 7. Here too, an arrow indicates the direction of communication.

It is also preferable to provide a control unit, not shown, which is integrated on the substrate 100 and which has to carry out the controls for memory accesses and for a calculation control and which also has to process the existing information about the outline 101 of the object 10 .

The size of the overlap memory 203 determines the maximum object size that can be treated in one run. If the object is larger, the pixel sets determined by this object can be subdivided and processed in the manner described above.

It is advantageous to select the memory 202 as large as possible. In this way, the probability is increased that even for larger objects in which several scanning passes through the processor element field 201 are necessary, data of the search area pixel set 23 only has to be loaded from external memories.

From a certain object size or a certain maximum shift out of the defined pixel set 22, however, an external reloading of the data cannot be avoided. The calculations with the data present in the memory 202 are then expediently organized in such a way that a part of this memory 202 can be released as quickly as possible. In this way, the reloading of new data into this memory 202 can already be started, while the remaining existing data in this memory 202 is still being used. This can significantly reduce the number of idle cycles. A complete search has hitherto been used as a basis, ie all the neighboring pixel sets of the other image 22 defined above are treated. Since the processor element field 201 consists of as many processor elements 200 as there are such neighboring pixel sets of the other image 22, a complete search means that each processor element 200 of the processor element field 201 calculates its deviation value.

As with the known "block matching method", an incomplete, i.e. effort-reduced search can be carried out. This means that not all pixel sets of the other image 22 are taken into account, i.e. the processor elements 200 assigned to such pixel sets not to be taken into account do not have to carry out any calculations. Such processor elements 200 are masked during an incomplete search during the calculation.

There are two types of masking. The first type is the masking of the entire process element field 201 at specific times, with which pixels which do not belong to the object are masked out. The second type of masking is the constant masking of individual processor elements 200, which are assigned to unnecessary pixel sets of the other image 22.

As an alternative to local accumulation, global accumulation can also be used in the invention. Also in the case of global accumulation, similar to local accumulation, a processor element field can be used, for example, that consists of processor elements arranged in a matrix in rows and columns and whose size determines the size of the object to be treated or the size to adjust the object. In the global accumulation, the pixel values of all the pixels of a single pixel set of the other image 22 are processed simultaneously. A processor element of the processor element field is assigned to each pixel of this pixel set. The object 10 with its outline 101 determines the processor elements that are used for the calculation. These are the processor elements which are assigned to a picture element or outline picture element of each picture element set of the predetermined picture element 2- _] _ and other picture elements 22 determined by this object 10 and its contour 101. All other processor elements which are assigned to a pixel lying outside of this pixel set have to be continuously masked during the calculation. On the other hand, all processor elements which are assigned pixels to a set of pixels and which are not to be taken into account due to an incomplete search must currently be masked.

For a complete search for any objects, the processor element field with local accumulation constantly offers better utilization than a processor element field with global accumulation.

For any search for any objects, the processor element field for local accumulation also appears to be the cheaper one. In fact, if the masking of individual processor elements is dispensed with, the results are retained or even improved, whereby the utilization of the processor element field can increase to almost 100%. However, in the case of processor elements for global accumulation, the masking of individual processor elements for the definition of the object outline cannot be dispensed with. At the same time, however, this means that the maximum utilization of the processor element field is strongly dependent on the shape of the object. The basic architecture is identical for one-dimensional and two-dimensional processor element fields. If a one-dimensional processor element field provides sufficient computing power, taking into account the achievable mean utilization, the advantages of one-dimensional compared to two-dimensional processor element fields can also be exploited.

A one-dimensional processor element field is considered that, for example, consists only of the processor elements of the leftmost column 302 of the field 201 according to FIG. 10. Since in such a one-dimensional process element field, a direct connection between the memories 202 and 203 integrated on the substrate 100 and each processor element 200 is possible, since all processor elements 200 lie on the edge of the processor element field, no more conditions arise through local communication, but instead data can be read column by column or row by row from these memories in each clock cycle. A meandering flow of data is not necessary. Accordingly, the masking type, in which the entire processor element array is masked in individual cycles, is also omitted, since this type of masking was only caused by the local communication in the two-dimensional εyεtolic processor elements.

If a one-dimensional processor element field with, for example, P processor elements with local accumulation is used, P horizontal or vertically adjacent pixel sets of the other image 22 can be treated in a calculation run P without load losses for any objects. A marking can be used for individual process elements if the associated amount of pixels does not have to be considered. If the masking is not carried out, however, only the fact that more pixel sets than necessary are examined. This can be taken into account when designing the algorithm. If a one-dimensional processor element field with global accumulation is used, a line or a column of pixels of a pixel set is treated in one system cycle. The masking of individual processor elements in accordance with the object shape cannot be dispensed with here, since it is predetermined by the object shape.

Claims

claims

1. Method for estimating a movement of an object in a temporal sequence (1) of images (2) composed of pixels (20), wherein

- In each image (2) each pixel (20) has a pixel value defined by this image (2) and

- The pixels (20) of two images (2) are reversibly uniquely assigned to each other, wherein - in a predetermined image (2- _] _) a pixel set (21) is selected, the pixels (20 ^, 20 _1; -_) represent a selected object (10) and a selected border surrounding the object (10), wherein

- due to the reversible unambiguous association between the said predetermined image (2j _{_)} and the Bild¬ pixels (20) points (20) jedeε other Bildeε (2) of the predetermined image {2j) in each of the selected set of pixels (21) another image (2) each has a set of pixels (22) whose pixels (202, 2021) represent this object (10) and this border each in the same shape, size and position as in the predetermined image (2 ^), in which

- In at least one other image (22) a search area pixel set (23) other than the defined pixel set

(22) of this other image (22) containing pixels (2O ₃ ) is defined,

- In this other image (22), in addition to the defined pixel set (22), one or more different additional pixel sets (24, 25, ... 2j, ... 2n) different from the defined pixel set (22) it is determined that - the pixels (2O4, 2O ₄₁ ; 20 ₅ , 20 ₅₁ ; ... 20j, 20j ₁ , ... 20 _n , 20 _n ι) of every further pixel set (24, 25, ... 2j, ... 2n) reversibly clearly assigned to the pixels (2O2, 2O2 ₁ ) of the defined pixel set (22) and contain one or more pixels (2O3) of the search area pixel set (23), and

- Each further set of pixels (24, 25, ... 2j, ... 2n) each the selected object (10) and the selected border in this shows another image (2) in the same shape and size as in the predetermined image (2 - * _) but offset into the search area pixel set (23), whereby

- For each pixel set (22, 24, 25, ... 2j, ... 2n) of the other image (22) including the defined pixel set (22), a certain deviation value (Δ ₂ , Δ ₄ , Δ ₅ , ... Δ- _j , ... Δ _n ), which is a measure of a deviation of the pixel values of this set (22, 24, 25, ... 2j, ... 2n) from the pixel values of the selected pixel set ( 21) of the predetermined image (2-j_) is determined,

- from the determined deviation values (Δ2, Δ ₄ , Δ5, ... Δ j, ... Δ _n ) the relatively smallest deviation value (eg Δj) is determined and

- The pixel quantity (2j) of the other image (22) belonging to the relatively smallest deviation value (Δj) is defined as the position of the selected object (10) in the other image (22), into which this object (10) is located from its position in vor¬ certain image (2- _] _) has moved, characterized in that - as a border of the selected object (10) a predetermined or predetermined outline (101) of this object (10) itself is used, which the object (10) directly limited and defined by outline pixels (20n) belonging to the pixels (20 -] _, 20H) of the selected pixel set (21) of the predetermined image (2 _] _), where

- Each pixel set (22, 24, 25, ... 2j, ... 2n) of the other image (22) has this outline (101), which in this pixel set (22, 24, 25, ... 2j , ... 2n) by outline image points

(2O2 ₁ , 2O ₄₁ , 2θ5 _1; ... 20jι, .. -20 _nl ) is defined, which corresponds to the pixels (2O2, 2O21; 20 ₄ , 20 ₄₁ ; 2O5, 20 ₅₁ ; ... 20j,

20j- _] _; ... 20 _n , 20 _nl ) of this set (22, 24, 25, ... 2j, ... 2n) heard that

- The pixels (20-j_, 20H) of ^the selected pixel set

(21) of the predetermined image (2 -] _) and the pixels (2O2, 20 ₂ ι; 20 ₄ , 20 ₄₁ ; 20 ₅ , 20 ₅₁ ; ... 20j, 20j- ^• _; ... 20 _n , 20 _nl ) one

Set of pixels (22, 24, 25, ... 2j, .., 2n) of the other image

(22) are scanned for their pixel values that - this scan does not substantially superior to the contour pixels _(# 20 ₁₁ - ι 20 _2; 20 _41; 20 _51; ... 20j _i; ... 20 _nl) of the respective set of pixels (21, 22, 24, 25, ... 2j, ... 2n) defined outline (101) of the object (10) is made.

2. The method according to claim 1, characterized by

Using the same arrangement of the pixels (20) in different images (2).

3rd Process according to claim 1 or 2, g e k e n n e e i c h n e t d u r c h

Using the same number of pixels (20) in different images (2).

4. The method according to any one of the preceding claims, characterized by

Use of a search area pixel set (23) which completely surrounds the outline (101) of the defined pixel set (22) of the other image (2 ₂ ).

5. The method according to any one of the preceding claims, characterized in that

- for each pixel (20-j_, 20H) ^ selected pixel set (21) and the pixel assigned to this pixel (20 -] _, 20 --_ - _) (2O2, 2O2 ₁ ; 20 ₄ , 20 ₄₁ ; 20 ₅ , 2O5 ₁ ; ... 20j, 20jι; ... 20 _n , 20 _nl ) of a pixel set (22, 24,

25, ... 2j, ... 2n) of the other image pair (22) consisting of a pair of pixels - a value dependent on the two pixel values of this pair of pixels is determined in a predetermined manner, and that

- the values determined for all pixel pairs of these two sets (21, 22; 21, 24; 21, 25, -... 21, 2j; 21, 2n) - for one the deviation value (Δ2, Δ ₄ , Δ5, .. .Δj ... Δ _n ) of these two amounts (21, 22; 21, 24; 21, 25; ... 21, 2j; 21, 2n) forming a sum.

6. The method according to claim 5, characterized by

Using the difference between the two pixel values of a pair of pixels as the specific value dependent on these two pixel values.

7. The method according to Anεpruch 5 or 6, characterized in that - the pixels (20- _] _, 20H) of the selected pixel set (21) of the predetermined image (2-j_) and

- The pixels (202, 2θ2i; 20 ₄ , 20 ₄₁ ; 2O5, 2θ5i; ... 20j, 20j ₁ ; ... 20 _n , 20 _nl ) assigned to these pixels (20-j_, 20-π) of a pixel set ( 22, 24, 25, ... 2j, ... 2n) of the other image (2 ₂ )

- Scanned one after the other in one and the same predetermined sequence.

8. The method according to claim 7, characterized in that in each pixel set (22, 24, 25, ... 2j, ... 2n) of the other image (22) whose pixels (2O2, 2O2 ₁ ; 20 ₄ , 20 ₄₁ ; 2O5, 2O5 ₁ ; ... 20j, 20jι; ... 20 _n , 20 _n3 _) can be scanned in the same predetermined sequence.

9. The method according to claim 7 or 8, characterized in that the predetermined order in which the pixels (20 _1; 20 _11; - 2O2, 2O2 ₁ ; 20 ₄ , 20 ₄₁ ; 20 ₅ , 20 ₅₁ ; .. .20j, 20j • -_; ... 20 _n , 20 _nl ) of a pixel set (21, 22, 24, 25, ... 2j, ... 2n) can be scanned by meandering scanning of all pixels ( 20- _j _, 20 _lι; 20 ₂ , 20 ₂ ι; 20 ₄ , 20 ₄₁ ; 20 ₅ , 20 ₅₁ ; ... 20j, 20j --_; ... 20 _n , 20 _nl ) of this quantity (21 , 22, 24, 25, ... 2j, ... 2n) is defined.

10th Method according to claim 7, 8 or 9, characterized in that the determined values formed from the two pixel values of each pair of pixels of the predetermined pixel set (21) and a pixel set (22, 24, 25, ... 2j, ... 2n) of the other image (22) in the same predetermined order are formed and summed up as the pixels of these two sets are scanned.

11. The method according to any one of the preceding claims, characterized in that when scanning the pixels of a pixel set (21, 22,

24, 25, ... 2j, ... 2n) a pixel value of a sampled, but outside the outline (101) of this set (21, 22, 24,

25, ... 2j, ... 2n) and thus not belonging to this set (21, 22, 24, 25, ... 2j, ... 2n) during the determination of this set ( 21, 22, 24, 25, ... 2j, ... 2n) relevant deviation values is ignored.

12. The method according to any one of the preceding claims, characterized in that - the selected pixel set (21) of the predetermined image (2- _j _) is subdivided into one or more subjoint pixel subsets such that

- The pixels of all these subsets yield all pixels (20-j_, 20] _ι) of the selected pixel set (21), whereby

- by each pixel subset of the selected pixel set (21) of the predetermined image (2 -] _) due to the reversibly unique assignment between the pixels (20) of the predetermined image (2- _] _) and the pixels (20 ) each other image (2 ₂ ) in the defined one and each further one

Set of pixels (22, 23, 24, 25, ... 2j, ... or 2n) of this other image (22)

- Each associated pixel subset defines that

- For the defined and each further pixel set (22, 23, 24, 25, ... 2j, ... or 2n) of the other image (22) per pixel subset of the selected pixel set (21) of the predetermined image (2- _j _) and that of this pixel subset assigned pixel subset of this pixel set (22,

23, 24, 25, ... 2j, ... resp. 2n) of the other image (2 ₂ ) by scanning the pixels of these two pixel subsets, one subset deviation value each, which is a measure of a deviation of the pixel values of this assigned pixel subset of this pixel set (22, 23, 24, 25, ... 2j, ... or 2n) of the other image (22) from the pixel values of this pixel subset of the selected pixel set (21) of the predetermined image (2- _j _) is determined, and that for the defined and each further pixel set (22, 23,

24, 25, ... 2j, ... resp. 2n) of the other image (22), which is the measure for the deviation between the pixel values of this pixel set (22, 24, 25, ... 2j, ... 2n) of the other image (22) and the pixel values of the selected pixel set (21) of the predetermined image (2 --_) forming deviation value (Δ2, Δ ₄ , Δ5, ... Δj ... Δ _n ) by summation of all for this pixel set (22, 24, 25, ... 2j, ... or 2n) of the other image (22) determined subset deviation values is formed.

13. The method according to any one of the preceding claims, characterized in that the defined and all further pixel sets (22, 24, 25, ... 2j, ... 2n) deε other Bildeε (22) together the search area pixel set ( 23) form.

14. The method according to claim 13, characterized in that the deviation values are determined only for a part of the pixel sets (22, 24, 25, ... 2j, ... 2n) of the other image (22) forming the search area pixel set (23) and the relatively smallest deviation value is determined from this incomplete number of deviation values.

15. Device for carrying out a method according to one of the preceding claims, characterized in that for each set of pixels (22, 24, 25, ... 2j, ... 2n) of the other

A processor element (200) is provided for each image (22),

- To which the pixel values of the selected pixel set (21) and the pixel values of the pixel set (22, 24,

25, ... 2j, ... 2n) of the other image (22) can be fed, and

- that from the supplied pixel values of the deviation value (Δ2, Δ ₄ Δ5, ... Aj, ... Δ _n) zwiεchen the pixel values of the auεgewählten set of pixels (21) deε predetermined image _(2-] _) and the pixel values of the pixel amount (22, 24, 25, ... 2j, ... 2n) of the other image (22) is calculated.

16. Device for carrying out a method according to one of the preceding claims, characterized in that for each pixel (20-j_, 20H) of the selected pixel set (21) of the predetermined image (2 - J, one process element each (200 ) is provided,

- the two pixel values for each pixel set (22, 24, 25, ... 2j, ... or 2n) of the other image (22)

Pixel (20] _, 20π) of the selected pixel set (21) and the pixel assigned to this pixel (2O2, 2O2 ₁ ; 20 ₄ , 20 ₄₁ ; 20 ₅ , 2O5 ₁ ; ... 20j, 20jι; ... or 20 _n , 20 _nl ) of this pixel set (22, 24, 25, ... 2j, ... or 2n) of the other image (2 ₂ ) can be fed, and that

- All of these processor elements (200) together for this pixel set (22, 24, 25, ... 2j, ... or 2n) of the other image

⁽ 2 ₂ ⁾

- from the pixel values supplied to them the deviation value (Δ2, Δ ₄ , Δ5, ... Δj, ... or Δ _n ) between the pixel values of the selected pixel quantity (21) of the predetermined image (2 --_ ) and the pixel values of this pixel set

(22, 24, 25, ... 2j, ... or 2n) of the other image (22).

17. The apparatus of claim 15 or 16, characterized in that the processor elements (200) are arranged in a matrix in rows (30-j_) and columns (302).

18. The apparatus according to claim 15 or 16, characterized in that the processor elements (200) are arranged in a single row (30 - * _ or 3O2).

19. The apparatus of claim 17 or 18, characterized in that each processor element (200) is only in communication with immediately adjacent processor elements (200).

20. Device according to one of claims 15 to 19, characterized in that the processor elements (200) are integrated on a common substrate (100).

21.Device according to claim 20, characterized in that on the substrate (100) a memory (202) for storing pixel values of sampled pixels (202 / 2O2 ₁ ; 2O ₄ , 20 ₄₁ ; 20 ₅ , 2O51; ... 20j, 20j-j_; ... 20 _n , 20 _n ι) of a pixel set (22, 24, 25, ... 2j, .., 2n) of the other image (2 ₂ ) is integrated.

22. Apparatus according to claim 20 or 21, characterized in that an overlap memory (203) is integrated on the substrate (100), in which the pixel values of the pixels scanned from the first scanned pixels are referenced with respect to two time-sequentially scanned pixel sets Point set that this pixel set has in common with the subsequent pixel set can be stored for determining the deviation value of the subsequent pixel set.

23. The method according to claim 14 for operating a device according to claim 15 or claim 15 and one of claims 17 to 22, characterized in that the deviation values successively by successive scanning of all the pixel sets (22, 24) forming the search area pixel set (23) , 25, ... 2j, ... 2n) of the other image (22) are determined, wherein - each processor element (200), which is assigned to a pixel set, its deviation value is to be used to determine the relatively smallest deviation value , calculates the deviation value of this pixel set, and

- Each processor element (200) that is assigned to a pixel set whose deviation value cannot be used to determine the relatively smallest deviation value does not carry out any calculations.

24. The method according to claim 14 for operating a device according to claim 16 or claim 16 and one of claims 17 to 22, characterized in that the deviation values are successively scanned by successively scanning all the pixel sets (22, 24) forming the search area pixel set (23) , 25, ... 2j, ... 2n) of the other image (22) can be determined, whereby

for each pixel set whose deviation value can be used to determine the relatively smallest deviation value, the processor elements (200) assigned to the pixels of this pixel set calculate the deviation value of this pixel set, and

- For each pixel set whose deviation value cannot be used to determine the relatively smallest deviation value, the processor elements (200) assigned to the pixel of this pixel set do not perform any calculations.

25. A method for operating a device according to claim 21 or claim 21 and claim 22, characterized in that during a reading of pixel values of a pixel set stored in the memory (202) from the memory (202), pixel values of a pixel set following this pixel set in the memory (202) can be read.