WO1997008654A1 - Procede pour evaluer le mouvement d'un objet dans une sequence d'images - Google Patents

Procede pour evaluer le mouvement d'un objet dans une sequence d'images Download PDF

Info

Publication number
WO1997008654A1
WO1997008654A1 PCT/DE1996/001493 DE9601493W WO9708654A1 WO 1997008654 A1 WO1997008654 A1 WO 1997008654A1 DE 9601493 W DE9601493 W DE 9601493W WO 9708654 A1 WO9708654 A1 WO 9708654A1
Authority
WO
WIPO (PCT)
Prior art keywords
pixel
pixels
image
pixel set
values
Prior art date
Application number
PCT/DE1996/001493
Other languages
German (de)
English (en)
Inventor
Luc De Vos
Original Assignee
Siemens Aktiengesellschaft
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Siemens Aktiengesellschaft filed Critical Siemens Aktiengesellschaft
Publication of WO1997008654A1 publication Critical patent/WO1997008654A1/fr

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/254—Analysis of motion involving subtraction of images
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras

Definitions

  • the invention relates to a method for estimating a movement of an object in a sequence of images according to the preamble of claim 1.
  • a block-shaped pixel set which contains the pixels and pixels representing the object and which contains a rectangular or square surrounding the object Show outline.
  • the pixel set defined by the selected pixel set and each further pixel set of the other image are block-shaped and have a rectangular border made of pixels.
  • the pixels of the selected pixel set of the predetermined image and the pixels of the defined pixel set and each further set of the other image are scanned and the pixel values of all the pixels of each of these sets are determined.
  • the scanning of the pixels is carried out in a meandering manner in each pixel set.
  • the scanning can be carried out column by column or row by row in such a way that all the pixels of each column or row, including the pixels representing the rectangular border, are detected and the pixels of respectively adjacent columns or rows are scanned in opposite directions to one another.
  • the deviation value for the defined and each further pixel set of the other image is determined in such a way that for each pixel of the selected pixel set of the predetermined image and the reversibly uniquely assigned pixel of the relevant pixel set of the other image, the difference between the pixel values is The two pixels are determined and the difference amounts of all pairs of pixels of these two sets of pixels assigned to one another are summed.
  • the rectangular border is selected so that it encloses the object, which often differs greatly from the rectangular shape, as closely as possible.
  • each block-like pixel set of the one image in each other image has a corresponding block-like one assigned amount of pixels defined.
  • a deviation value is determined for each pair of block-shaped picture element sets of two pictures assigned to one another, in that, similarly to the proposed object-based method, for each pair of picture elements of these two block-shaped picture element sets assigned to each other, the difference between the picture element values of these two picture elements is determined and the difference amounts of all pairs of picture elements of these two picture element sets assigned to one another are summed.
  • the pair of pixel sets with the relatively smallest deviation value denotes the position of the selected object in the other image, into which this object has moved from its position in the one image.
  • a disadvantage of the block-based method lies in the fact that the boundaries of the block-shaped pixel sets can only approximate the boundaries of the object.
  • a block-shaped set of pixels can contain partially pixels of the moving object and partially pixels of the background or can even consist of parts of different moving objects. In such cases, a block-based method delivers incorrect results for some of the pixels.
  • the object-based method according to the invention as claimed in claim 1 has the advantage over the object-based method already proposed that, in order to determine the deviation value for the defined and each further pixel amount of the other image, as a rule significantly fewer pixels have to be scanned .
  • the selected pixel set advantageously coincides with the object itself, ie the pixels of the selected pixel set are the pixels of the object itself and no further pixels of the predetermined image, as is the case with the object-based method already proposed is when the block-shaped number of pixels and the object contained therein differ.
  • the deviation value essentially only the pixels of the selected pixel set of the predetermined image and the pixel sets of the other image determined by this quantity have to be scanned. Only under certain circumstances, for example in the case of very irregular objects, can it occur that a few pixels have to be scanned which are outside the relevant pixel set and which are to be masked out, i.e. they must not be used to determine the deviation value.
  • the invention is advantageously suitable for both local and global accumulation and can be used, inter alia, with MPEG coding.
  • the invention can be applied to image coding with low, preferably extremely low transmission rates.
  • FIG. 1 shows a schematic representation of a temporal sequence of images composed of pixels with a selected number of pixels, the position of which changes from image to image and which represents a selected, moving object,
  • FIG. 2 shows an enlarged, detailed schematic illustration of a predetermined image of the sequence according to FIG. 1, in which the selected set of pixels is highlighted,
  • FIG. 3 shows, in the same representation as in FIG. 2, another image of the sequence according to FIG. 1, in which the selected amount of pixels offset from the predetermined image is also highlighted,
  • FIG. 4 in the same representation as in FIG. 3, the other image, in which the defined number of pixels is clarified and a search area is highlighted
  • FIGS. 5 to 8 each in the same representation as in FIG. 4, the other image, the search area being highlighted in each case and a different further set of pixels being illustrated in each figure
  • FIG. 9 shows, in the same representation as in FIG. 1, the predetermined image with a meandering scanning path, on which the selected amount of pixels is scanned, with pixels to be masked being emphasized, and
  • FIG. 10 shows a block diagram of a device for carrying out the method according to the invention in a schematic illustration.
  • the temporal sequence of images is designated 1.
  • the images themselves are generally designated 2, the pixels generally 20, the selected object 10 and the outline of the object 101.
  • the predetermined picture is labeled 2 -] _ and the other picture is labeled 2 ⁇ .
  • the number of pixels selected in the predetermined image 2- ] _ and representing the selected object 10 is designated by 21 and its pixels by 20 ⁇ _ and 20n.
  • the defined pixel set of the other image 22 is designated 22 and its pixels with 2O2 and 2O21.
  • the search area pixel set in the other image 22 is designated with 23 and its pixels with 2O3.
  • the further picture element sets of the other picture 22 are with 24, 25, ... 20j, ... 20 n and their picture elements with 20 4 and 204 ! , 20 5 and 2 ⁇ 5 1 ( ... 20j and 20j +1 , ... or 20 n and 20 n + 1 respectively.
  • the term "each pixel set of the other image 22" is used in the following not only includes every further pixel set 24, 25, ... 2j, ... or 2n of the other image 22, but also their defined pixel set 22 .
  • the pixels 20 u , 20 2 ⁇ , 20 41 , 20 51 , ... 20j 1 # ... 20 nl each designate the outline pixels defining the outline 101 of the relevant pixel set 21, 22, 24, 25, ... 2j , ...respectively. 2n belonging to this set.
  • Each of these images 2 can be selected as the predetermined image 2 --_ ge.
  • Each image 2 following the predetermined image 2- j _ or preceding the predetermined image 2 ] _ can in principle be selected as the other image 22.
  • the image 2 at the time t-j_ is selected as the predetermined image 2 -] _ and for example the image 2 at the next time t2 as the selected image 22.
  • each of two images 2 must be reversibly clearly assignable to each other, i. H. , in each of these two images 2 only the same number of pixels 20 can be considered. For the sake of simplicity, however, it should not necessarily be assumed that each image 2 has the same number of pixels 20.
  • the pixels 20 are arranged in the same way in each image 2, for example and as shown in FIGS. 1 to 9, in a matrix form in horizontal rows 3 - and _ and vertical columns 32, the images 2 preferably each having the same number of rows 3- j and the same number of columns.
  • the pixels 20 in FIG. 1 are arranged in a smaller number and in a larger grid dimension from one another.
  • a line 3 -] _ represented by a horizontal line
  • a column 32 represented by a vertical line
  • each of the adjoining small squares each represents a pixel 20.
  • Each pixel 20 in each image 2 has a pixel value, which can be a brightness and / or color value, for example, and can differ from pixel to pixel.
  • the distribution of the pixel values over the pixels 20 in an image 2 depends on the objects shown and can change from image 2 to image 2 of sequence 1, which is particularly the case with moving objects.
  • the pixel values of the individual pixels 20 are not shown in FIGS. 1 to 9 for the sake of clarity. If picture elements 20 are highlighted or clarified in relation to other picture elements 20, this serves only to indicate picture element quantities and not picture element values.
  • the moving object 10 is selected in the predetermined image 2- ] _ by a
  • Pixel set 21 is selected, the pixels 20 ⁇ _ and 20H of this object 10 and its outline.
  • the outline of the selected object 10 is not a rectangle surrounding the object 10, as in the object-based motion estimation method already proposed and mentioned above, but according to the invention a predetermined or predetermined rectangle of the selected pixel set that directly delimits the object 10 and 11 21 defined outline 101 of this object 10 itself.
  • the pixels 20-j_ of the pixel set 21 including their outline pixels 20 ⁇ are highlighted by narrow hatching, the outline pixels 20H additionally being highlighted by a central point.
  • the selected object 10 with its outline 101 can be determined by a suitable segmentation algorithm, according to which the pixels 20-j_ and outline pixels 20 ⁇ of the selected pixel set 21 are defined.
  • the position of the object 10 in an image 2 can be determined, for example, by image-related horizontal and vertical coordinates of a particular pixel of the pixel set representing the moving object 10, for example by a leftmost pixel of this pixel set.
  • the coordinate origin is, for example, the lower left corner 0 of each image 2 shown in FIGS. 1 to 9.
  • this leftmost pixel 20 1:] _ of the pixel set 21 has the coordinate h - * _ in the horizontal direction and the coordinate in the vertical direction v -] _.
  • the leftmost pixel of the set of pixels representing object 10 has the horizontal coordinate ⁇ 12 and the vertical coordinate V2, with 1-2> h ⁇ _ and V2> v ⁇ _ being assumed, for example .
  • the latter means that the object 10 has shifted both horizontally and vertically by a certain amount and therefore has a different position in the other picture 22 than in the picture 2-j_
  • the object 10 itself can change in shape and size as well as in the distribution of brightness and / or color over its pixels 20 during its movement and the invention can also be used with such variable objects 10, it is assumed for the sake of simplicity that that the object 10 does not change even during its movement.
  • the object 10 itself does not change, when it moves from image 2 to image 2 only the pixel sets representing the object 10 change in the sense that the pixel sets representing the object 10 of different images 2 do not at least partially have the same pixels .
  • the search area pixel set 23 is defined from the pixels 2O3 not contained in the defined pixel set 22 of the other image 22, which is highlighted in FIGS. 4 to 9 by relatively wide hatching and preferably that defined by the outline pixels 2O2 1 of the defined ones Pixel set 22 completely surrounds certain outline 101 of this pixel set 22.
  • one or more different further pixel sets different from the defined pixel set 22 are determined such that the pixels of each additional pixel set are reversibly uniquely assigned to the pixels of the defined pixel set 22 and one or contain several pixels that coincide with pixels 20 3 of the search area pixel set 23, and that each of these further pixel sets each has the selected object 10 and its outline 101 in the same shape and size as in the predetermined image 2-j_, but offset into the search area pixel set 23.
  • FIGS. 5 to 8 Such different further pixel sets are shown by way of example in FIGS. 5 to 8 and are illustrated by larger central circles in the pixels in comparison with the small circles of pixels 2O2 and outline pixels 2O2 1 of the defined pixels 22 in FIG again the outline pixels are highlighted by reinforced circles.
  • the illustrated in Figure 5 further set of pixels 24 with the pixels 20 4, and the outline 101 representing Umriß ⁇ pixels 2O41 has the h of the coordinates --_ and v --_ the defined set of pixels 22 different coordinate H.4 and V 4, where, for example, I14 ⁇ h] _ and V 4 ⁇ v -] _ is assumed, ie the further pixel set 24 is offset from the defined pixel set 22 in the direction obliquely to the bottom left into the search area pixel set 23.
  • the further pixel set 25 shown in FIG. 6 with the image pixels 205 and the outline 101 representing outline pixels 205 1 has the coordinates hs and V5, where, for example, I15 ⁇ I1 4 ⁇ h ⁇ and V 4 ⁇ V5 ⁇ v -] _ is assumed, ie the further pixel set 25 is wider than the defined pixel set 22 in the horizontal direction and less in the vertical direction than the further pixel set 24.
  • the further pixel set 2n according to FIG. 8, which was assumed to be the last, with the image pixels 20 n and the outline 101 representing the outline pixels 20 n - ] _ has the coordinates h n and v n , where, for example, h n > h ⁇ _ and v- j _ ⁇ v n is assumed, ie this last further pixel set 2n is offset obliquely to the right into the search area pixel set 23 compared to the defined pixel set 22.
  • the procedure is such that as many directions as possible from the defined set of pixels 22 are detected by further sets of pixels 24, 25,... 2j,... 2n in order to cover the entire environment of the defined set of pixels 22 to scan evenly.
  • the outline 203 of the search area pixel set 23 defined by outline pixels 203 is generally not rectangular, as shown in FIGS. 4 to 9, because it outlines the combination of the number of further pixel sets 24 to 2n which can be freely selected and is thus determined by the mostly irregular shape of the object 10 itself.
  • the search area pixel set 23 can within its limits by specifying a maximum offset ⁇ h of the defined pixel set 22 horizontally to the left and right from its original position in the other image 22 and a maximum offset ⁇ v of the defined pixel set 22 vertically up and down from its original Location in the other picture 22 can be characterized.
  • the pixels 20 --_ and outline pixels 20n of the selected pixel set 21 and the pixels 2O2, 20 4 , 2O5, ... 20j, ... or 20 n and outline pixels 20 2 ⁇ , 20 41 , 205 -j_, ... 20j -] _, ... or 20 nl of each pixel set 22, 24, 25, ... 2j, ... or 2n of the other image 22 are scanned for their pixel values, whereby these Scanning at each of these pixel sets 21, 22, 24, 25, ... 2j, ...
  • n is determined, which is a measure of a deviation of the pixel values of this set 22, 24, 25, ... 2j, ... or 2n of the pixel values of the selected pixel set 21 of the predetermined image 2 -] _.
  • the relatively smallest deviation becomes from the deviation values ⁇ 2 ⁇ 4 , ⁇ 5, ... ⁇ j, ... and ⁇ n determined for each pixel set 22, 24, 25, ... 2j, ... and 2n of the other image 22 ⁇ chung value, which is given, for example, by ⁇ j with a specific j, and the pixel set 2j of the other image 22 belonging to this relatively smallest deviation value ⁇ j is defined as the position of the selected object 10 in the other image 22, into which this object is expressed its position in the predetermined image 2 -] _ has moved.
  • the procedure is preferably such that for each of a pixel 20- ] _ or 20n the selected pixel set 21 and this pixel 20j_ or 20H associated pixel 2O2 or 2O2 1 , 2O4 or 204 ! , 2O5 or 205 2 , ... 20j or 20j ⁇ , ... or 20 n or 20 n ⁇ of a pixel set 22, 24, 25, ... 2j, ...
  • the specific value dependent on the two pixel values of a pixel pair can, for example,
  • m 1 and thus the difference
  • 20 ⁇ , 202 and 2O21, 20 4 and 20 41 » 20 5 and 20 51- • • • • 20 j and 2 0j ⁇ , ... or 20 n and 20 nl of a pixel set 21, 22, 24, 25, ... 2j,... Or 2n are preferably scanned by a meandering scanning of all the pixels of this set.
  • Such a meandering scanning should expediently begin at a most outer pixel of the relevant pixel set and continue from there in a certain continuation direction through this pixel set, the successive scanning of the pixels of this set alternating in the continuation direction in one Scanning direction is transverse to the continuation direction and then in the scanning direction opposite to this scanning direction.
  • the meandering scanning can be carried out horizontally from the left at a leftmost outline pixel of the relevant pixel set with a continuation direction start to the right and, for example, the first scanning direction from bottom to top or top to bottom.
  • a meandering scan with a continuation direction horizontally from right to left or in a vertical or oblique direction from top to bottom or bottom to top could just as well be used, for example
  • FIG. 9 shows an example of a meandering scan of a pixel set using the example of the defined pixel set 22.
  • the outline pixels 2021 of this defined pixel set 22 are highlighted by small central circles.
  • the meandering scanning of the pixels 2O2 and 2O21 of the defined pixel set 22 begins at the individual leftmost outline pixel 2O2 1 with the coordinates h- ⁇ _ and v-j_ and starts from there in the first scanning direction upwards to the immediately adjacent pixel 203 of the search area pixel set 23, from there in the continuation direction to the right to the immediately adjacent outline pixel 2O2 1 of the defined pixel set 22, from there in the opposite scanning direction downwards to an outline pixel 2O2 1 the defined pixel set 22, from there in the continuation direction to the right to the immediately adjacent outline pixel 2O2 1 of the defined pixel set 22, from there in the first scanning direction upwards to an outline pixel 2O21 of the defined pixel set 22 and via this outline pixel 2O21 to an immediately neighboring picture pu 2O3 of the search area pixel set 23, from there in the continuation direction to the right to an outline pixel 2O2 1 of the defined pixel set 22, from there in the opposite scanning direction down to an outline pixel 2O21
  • the manned shape would be the given one in the first scanning direction vertically from bottom to top at the bottom left-most outline pixel 2O2 1 with the horizontal coordinate h-j_ and the relatively smallest vertical coordinate V ] _.
  • pixels 2O3 of the search area pixel set 23 are also sampled so that all pixels 2O2 and 2O21 of the defined pixel set 22 are detected.
  • the size of the object 10 determines the size of a memory for storing the scanned pixel values and / or certain values formed therefrom.
  • the procedure can advantageously be such that the selected pixel set 21 of the predetermined one Image 2 --_ is subdivided into one or more disjunct image sub-sets such that the image points of all these sub-sets result in all image points of the selected image point set.
  • Each pixel subset of the selected pixel set 21 of the predetermined image 2 ⁇ means, owing to the reversible, unambiguous association between the pixels 20 of the predetermined image 2 --_ and the pixels 20 of each pixel set 22, 24, 25,. ..2j, ... or 2n of the other image 22 in this pixel set 22, 23, 24, 25, ... 2j, ... or 2n of the other image 22 each defines an associated subset of pixels.
  • a subset deviation value is determined in each case by scanning the pixels of this pair of pixel subsets, which is a measure of a deviation between the pixel values of this pair of pixel subsets.
  • An intermediate result memory can be provided for the subset deviation values, in which these subset deviation values can be saved as intermediate results for further processing.
  • FIG. 2 shows an example of a subdivision of the selected pixel set 21 of the predetermined image 2 --_ into two disjoint pixel subsets, which when combined again result in the selected pixel set 21.
  • the division is carried out by a horizontal line 32. All above this line 32 lying pixels 20- j _ _ 13 and 20 of the selected set of pixels 21 form an upper pixel subset and all lying below the horizontal line 32 pixels 20 --_ and the 20 ⁇ selected pixel set 21 form a lower pixel subset.
  • the upper pixel subset of the selected pixel set 21 is each assigned an upper pixel subset in the defined pixel set 22 and each further pixel set 22, 23, 24, 25, ... 2j, ... or 2n.
  • the associated top pixel subset of the defined set of pixels 22 consisting of Figure 4 from all devis ⁇ half of the horizontal line 32 lying pixels 2O22 and 2 ⁇ 2i- the associated top pixel subset of the further set of pixels 24 consists of all above the hori ⁇ zontal line 35 Pixels 2O 4 and 20 41 - D: - e assigned to the upper pixel subset of the further pixel set 25 in FIG. 5 consist of all pixels 20 5 and lying above the horizontal line 36 in FIG.
  • the assigned upper pixel subset of the further pixel set 2j consists of all pixels 20j and 20j ⁇ lying above the horizontal line 37 in FIG. 7.
  • the assigned upper pixel subset of the further pixel set 2n consists of all above the horizontal line 38 in FIG 8 pixels 20 n and 20 n ⁇ .
  • the assigned lower pixel subset of the defined pixel set 22 consists of all pixels 2O2 and 2O2 1 lying below the horizontal line 32 in FIG. 4, the assigned lower pixel subset of the further pixel set 24 consists of all below the line 35 pixels 2O 4 and 2O 41 in FIG. 5, the assigned lower pixel subset of the further pixel set 25 from all the pixels 2O5 and 2O5 1 lying below the horizontal line 36 in FIG. 6, the assigned lower pixel subset of the further pixel set 20H from all the pixels 20j and 20j ⁇ below the horizontal line 37 in FIG. 7 and the assigned lower pixel subset of the further pixel set 2n from all the pixels 20 n and 20 nl lying below the horizontal line 38 in FIG. 8.
  • the upper pixel subsets are scanned separately from the lower pixel subsets.
  • a subset deviation value is given for the upper pixel subset and separately for the lower pixel subset a subset deviation value in the determined in the manner described above.
  • the deviation value for this pixel set 22, 23, 24, 25, ... 2j, ... or 2n results from the sum of the two subset deviation values determined for this set.
  • a device for carrying out the method according to the invention with local accumulation is advantageously designed such that for each pixel set 22, 24, 25, ... 2j, ... or 2n of the other image 22, one of these pixel sets 22, 24, 25, ... 2j, ... or 2n is assigned to the processor element, to which the pixel values of the selected pixel set 21 and the pixel values of the pixel set 22, 24, 25, ... 2j, ... or 2n of the other image 22 zumarin ⁇ bar and from the input pixel values, the average deviation ⁇ 2, ⁇ 4 ⁇ 5, ... Aj, ⁇ n between the pixel values of the selected set of pixels 21 of the voted inconvenience ⁇ image 2] _ and Pixel values of the pixel set 22, 24, 25, ... 2j, ...
  • Local accumulation means that with each cycle with which the pixel values of the pixels of each pixel set of the other image 22 are processed in the assigned processor elements, only one pixel value of each pixel set of the other image 22 together with the assigned one
  • Pixel value of the selected pixel set of the predetermined image 2- ] _ is processed, in contrast to a global accumulation, in which at each cycle all pixel values of only one pixel set of the other image 22 together with the assigned pixel values of the selected one th pixel amount of the predetermined image 2 --_ are processed simultaneously, a processor element being provided for each pixel value of the selected pixel amount of the predetermined image 2j_ or each pixel amount of the other image 22.
  • a processor element being provided for each pixel value of the selected pixel amount of the predetermined image 2j_ or each pixel amount of the other image 22.
  • a processor element array of processor elements arranged in a matrix in P rows and Q columns i.e. Using a processor element array with PxQ processor elements, PxQ pixel sets of the other image 22, including the defined pixel set of this image, can be treated.
  • Each processor element is preferably only in communication with the immediately adjacent processor elements, so that the communication between the processor elements is local and there is a so-called systolic processor element field.
  • the data supply to the processor element field can only take place from the edge of the field.
  • the number of further pixel sets and thus also the number n and the number of processor elements 200 in the processor element field 201 are generally much larger.
  • This exemplary assignment does not take into account the fact that in a systolic processor element field with local accumulation in neighboring processor elements, neighboring pixel sets of the other image 22 are treated, whereby neighboring pixel sets are each two pixel sets that result from one another by a defined minimal shift relative to one another.
  • N for calculating the deviation values ⁇ 2 to ⁇ in ei ⁇ nem systolic processor element array 201 at local Akku ⁇ advantageously the pixels 20 ⁇ and 20 1 mulation:] _ of the selected set of pixels 21 of the predetermined image 2 --_ and j these pixels 20 _ and 20H assigned pixels 2O2 and 2O21, 2O4 and 2O41, 2O5 and 205 !, ... 20j and 20j ⁇ , ... or 20 n and 20 n ⁇ the defined and each further set of pixels 22, 24, 25 , ... 2j, ... or 2n each sequentially sampled in the same order.
  • An example of such a temporally successive scanning of the pixels of each pixel set of the predetermined and other image in a specific sequence is the meandering scanning according to FIG. 9 with respect to the defined pixel set 22 of the other image 22 shown there.
  • each pixel set 22, 24, 25,... 2j,... Or 2n of the other image 22 it is expedient for each pixel set 22, 24, 25,... 2j,... Or 2n of the other image 22 to have the pixel values of each pair of pixels that consist of a pixel 20 ] _ or 20H of the selected one Pixel set 21 of the predetermined image 2- j _ and that pixel 20 -] _ or 20 1: L reversibly uniquely assigned pixel 2O2 or 2O2 1 , 20 4 or 2 ⁇ 4 1; 2O5 or 2O51, ... 20j or 20j l7 ... or 20 n or 20 nl of this pixel set 22, 24, 25, ... 2j, ...
  • each processor element 200 the difference between the two pixel values of this pair is successively formed for each pair of pixel values assigned to one another for each pair of processor elements 200 assigned to this processor element, and successively in this order Summed up deviation value.
  • Processor elements 200 that achieve this are known and will not be described further.
  • each processor element 200 Thereafter, the sum obtained in each processor element 200 after addition of all difference amounts defines the deviation value ⁇ 2 , ⁇ 4 , ⁇ 5 , ... ⁇ j, ... or ⁇ n of the pixel set 22, 24, 25, .. .2j, ... or. 2n of the other image 22, which is assigned to this processor element 200.
  • the calculation of the deviation values must be a real-time calculation for each pixel set 22, 24 to 2n of the other image 22, before the appearance of the image 2 next to the other image 22 in question, in this case before the time t ß , be completed.
  • the pixel values of the pixels of each pixel set 22, 24, 25,... 2j,... Or 2n of the other image 22 are used to calculate the difference amounts or other specific values to form the deviation values for each of these pixel sets in a special purpose provided memory 202, from which these pixel values can be transferred to processor elements 200 in such a way that processing takes place in all processor elements 200 simultaneously.
  • This memory 202 is preferably designed in such a way that the pixel value data are read into it in series via one or a few gates 202-LO and are read out from it in rows or columns. It is expedient, starting from a first pixel set of the other image 22, for example the defined pixel set 22, to form the remaining pixel sets 24, 25, ... 2j, ... 2n of the other image 2 2 in succession in such a way that each the following pixel set corresponds to a shift of a previous pixel set in a certain direction by a certain amount, for example by a single pixel. In the sense defined above, two sets of image points which follow one another in this way are adjacent sets of image points which are treated in adjacent processor elements of the systolic processor element field 201.
  • Such neighboring pixel sets always have a number of pixels in common, and since the pixel values of these common pixels already determined for the previous of these two neighboring pixel sets are then already available for the subsequent pixel set, it is advantageous to increase the computing speed and save storage capacity, the pixel values of the first pixel set of the other image 22 and for each subsequent pixel set of this other image 22, temporarily store the pixel values of the pixels of this following pixel set common to the previous pixel set in a memory 203, referred to below as an overlap memory.
  • the pixel values stored in the overlap memory 203 for each of these pixel sets of the other image 22 are available to the processor element 200 assigned to this bil point set for further processing, and only those pixel values of this pixel set that are not yet available and to be determined by scanning have to be available for a pixel set on this pixel set following pixel amount can be read into this processor element 200 and in the overlap memory 203. Only those pixel values not yet available for a pixel set have to be stored in the memory 202 and read from there into the processor elements 200 and the overlap memory 203.
  • the memory 202 need not have such a large storage capacity that the pixel values of all the pixels 203 of the search area pixel set 23, including the pixels 202 and 202 1 of the defined pixel set 22 of the other image 22, can be stored, but it suffices Part of it.
  • the processor element array 200 is preferably integrated as a VLSI architecture (VLSI stands for Very Large Scale Integration) on a substrate 100, the memory 202 and the overlap memory 203 advantageously also being integrated on the substrate 100.
  • VLSI Very Large Scale Integration
  • the overlap memory 203 preferably consists of a register field consisting of individual registers 203, each represented by an elongated rectangle, which, to facilitate communication with the processor elements 200, preferably consists of two mutually opposite sides 201 ] , 2012 of the processor element field 201 arranged register field sections 203 and 2032.
  • the memory 202 is arranged opposite a third side 20I3 of the processor element field 201 lying between the two opposite sides 201 - * _, 2012 and has, for example, a multiplexer 202 --_ on its output side facing this third side 20I3.
  • the Loading of the memory 202 with data is preferably carried out from a memory which is not integrated on the substrate 100 and which is not shown.
  • FIG. 10 shows communication lines between the multiplexer 202- ] _ of the memory 202 and registers 203-j_o of the register field sections 203-j_ and 2032 as well as processor elements 200 of the processor element field 201, between processor elements 200, for the transmission of the pixel value data to be compared of the processor element field 201 and registers 203 dero of the register field sections 203 --_ and 2032, between processor elements 200 themselves and between registers 203 ⁇ o themselves each represented by a solid connecting line 5 between these components, an arrow on each of these lines 5 indicating the direction of communication and each of these lines 5 has a certain bit width, for example a width of 8 bits.
  • Communication lines for supplying the pixel value data of the individual pixels 20- j _, 20- ] _ ⁇ of the selected pixel set 21 of the predetermined image 22 to the process elements 200 ⁇ are represented by dash-dotted lines 6, with an arrow on each line also here 6 indicates the direction of communication.
  • the pixel value data of the individual pixels 20 ⁇ , 20-j _ - _ of the selected pixel amount 21 of the predetermined image 2 2 do not require a memory on the substrate 100 in the case of local accumulation, since in each clock cycle only the pixel value of a single pixel 20- j _ or 20H ⁇ er selected pixel set 21 of the predetermined image 2- ] _ is required for the entire processor element field 201, which can be loaded from an external memory (not shown).
  • communication lines for transmitting the deviation values determined in the processor elements 200 from the processor elements 200 to an input 300 --_ a comparator preferably integrated on the substrate 100 are device 300, which determines the smallest deviation value by comparison and outputs it at an output 3002, represented by dashed lines 7.
  • an arrow indicates the direction of communication.
  • control unit not shown, which is integrated on the substrate 100 and which has to carry out the controls for memory accesses and for a calculation control and which also has to process the existing information about the outline 101 of the object 10 .
  • the size of the overlap memory 203 determines the maximum object size that can be treated in one run. If the object is larger, the pixel sets determined by this object can be subdivided and processed in the manner described above.
  • an incomplete, i.e. effort-reduced search can be carried out.
  • Such processor elements 200 are masked during an incomplete search during the calculation.
  • the first type is the masking of the entire process element field 201 at specific times, with which pixels which do not belong to the object are masked out.
  • the second type of masking is the constant masking of individual processor elements 200, which are assigned to unnecessary pixel sets of the other image 22.
  • global accumulation can also be used in the invention.
  • a processor element field can be used, for example, that consists of processor elements arranged in a matrix in rows and columns and whose size determines the size of the object to be treated or the size to adjust the object.
  • the pixel values of all the pixels of a single pixel set of the other image 22 are processed simultaneously.
  • a processor element of the processor element field is assigned to each pixel of this pixel set.
  • the object 10 with its outline 101 determines the processor elements that are used for the calculation.
  • processor elements which are assigned to a picture element or outline picture element of each picture element set of the predetermined picture element 2- ] _ and other picture elements 22 determined by this object 10 and its contour 101. All other processor elements which are assigned to a pixel lying outside of this pixel set have to be continuously masked during the calculation. On the other hand, all processor elements which are assigned pixels to a set of pixels and which are not to be taken into account due to an incomplete search must currently be masked.
  • the processor element field with local accumulation constantly offers better utilization than a processor element field with global accumulation.
  • the processor element field for local accumulation also appears to be the cheaper one.
  • the masking of individual processor elements is dispensed with, the results are retained or even improved, whereby the utilization of the processor element field can increase to almost 100%.
  • the masking of individual processor elements for the definition of the object outline cannot be dispensed with.
  • this means that the maximum utilization of the processor element field is strongly dependent on the shape of the object.
  • the basic architecture is identical for one-dimensional and two-dimensional processor element fields. If a one-dimensional processor element field provides sufficient computing power, taking into account the achievable mean utilization, the advantages of one-dimensional compared to two-dimensional processor element fields can also be exploited.
  • a one-dimensional processor element field is considered that, for example, consists only of the processor elements of the leftmost column 302 of the field 201 according to FIG. 10. Since in such a one-dimensional process element field, a direct connection between the memories 202 and 203 integrated on the substrate 100 and each processor element 200 is possible, since all processor elements 200 lie on the edge of the processor element field, no more conditions arise through local communication, but instead data can be read column by column or row by row from these memories in each clock cycle. A meandering flow of data is not necessary. Accordingly, the masking type, in which the entire processor element array is masked in individual cycles, is also omitted, since this type of masking was only caused by the local communication in the two-dimensional ⁇ y ⁇ tolic processor elements.
  • a one-dimensional processor element field with, for example, P processor elements with local accumulation P horizontal or vertically adjacent pixel sets of the other image 22 can be treated in a calculation run P without load losses for any objects.
  • a marking can be used for individual process elements if the associated amount of pixels does not have to be considered. If the masking is not carried out, however, only the fact that more pixel sets than necessary are examined. This can be taken into account when designing the algorithm. If a one-dimensional processor element field with global accumulation is used, a line or a column of pixels of a pixel set is treated in one system cycle. The masking of individual processor elements in accordance with the object shape cannot be dispensed with here, since it is predetermined by the object shape.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Length Measuring Devices By Optical Means (AREA)
  • Image Processing (AREA)

Abstract

L'invention concerne un procédé d'évaluation du mouvement d'un objet dans une séquence d'images par l'examen de cet objet, certains ensembles de points d'image étant explorés et comparés les uns aux autres. Selon l'invention sont utilisés des ensembles de points d'image qui sont limités par le contour (101) de l'objet (10) lui-même de sorte que, par comparaison avec les procédés de la technique antérieure, le nombre de points d'image explorés est réduit et une mise en oeuvre du procédé en temps réel est possible. Sont également décrites des architectures à intégration à très grande échelle, se fondant sur l'invention.
PCT/DE1996/001493 1995-08-28 1996-08-08 Procede pour evaluer le mouvement d'un objet dans une sequence d'images WO1997008654A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
DE19531636.3 1995-08-28
DE19531636 1995-08-28

Publications (1)

Publication Number Publication Date
WO1997008654A1 true WO1997008654A1 (fr) 1997-03-06

Family

ID=7770598

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/DE1996/001493 WO1997008654A1 (fr) 1995-08-28 1996-08-08 Procede pour evaluer le mouvement d'un objet dans une sequence d'images

Country Status (2)

Country Link
AR (1) AR003370A1 (fr)
WO (1) WO1997008654A1 (fr)

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0665507A1 (fr) * 1994-01-14 1995-08-02 Hughes Aircraft Company Méthode et système à réseau neuronal pour estimer la position et l'orientation

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0665507A1 (fr) * 1994-01-14 1995-08-02 Hughes Aircraft Company Méthode et système à réseau neuronal pour estimer la position et l'orientation

Also Published As

Publication number Publication date
AR003370A1 (es) 1998-07-08

Similar Documents

Publication Publication Date Title
DE69728247T2 (de) Pyramidenförmiger pipeline-prozessor für ein bildverarbeitungssystem
DE3407983C2 (de) Mehrprozessorrechnersystem zum Erzeugen von Bildpunktinformationen aus in einer hierarchischen Datenstruktur definierten Objektelementen
DE60109858T2 (de) System zur Ungleichheitsmessung von stereoskopischen Bildern
DE2726746C3 (de) Verfahren und Einrichtung zur Fehlerkontrolle des Oberflächenmusters eines Objektes
DE69433031T2 (de) Bildprozessor
DE60008706T2 (de) Parallele datenverarbeitung und mischung
DE3632639C2 (de) Einrichtung zum Hochgeschwindigkeitsverarbeiten von Bilddaten durch Faltung
DE3141196A1 (de) Videobildverarbeitungsvorrichtung
DE112005000804T5 (de) Multi-Label-Bildsegmentierung mit GPU
DE3804938A1 (de) Bildverarbeitungseinrichtung
DE2746969C2 (de) Einrichtung zum Vergleichen von Mustern
DE102006014632B4 (de) Bildabtastvorrichtung
DE3038195C2 (de) Vorrichtung zur Verarbeitung von visueller Information
DE69818584T2 (de) Methode zur Gewinnung radialer Verzerrungsparameter aus einem einzigen Kamerabild
DE102012221667A1 (de) Vorrichtung und Verfahren zum Verarbeiten von Fernerkundungsdaten
DE3209073A1 (de) Anordnung zum umsetzen der zahl von abtastlinien
DE3490148C2 (de) Vorrichtung zur Erzeugung unterschiedlicher Figuren in einem computergesteuerten Darstellungssystem
DE4437385A1 (de) Verfahren und Vorrichtung zur Bildkorrektur
EP0625762A1 (fr) Méthode de connexion de radiographies et dispositif de réalisation de la méthode
DE19504089A1 (de) Pipelined SIMD-Systolic Array Prozessor und dessen Arbeitsverfahren
DE4230327A1 (de) Multi-port speicher und digitales interpoliergeraet
DE2461651C3 (de) Zählvorrichtung zum Zählen von Mustern
DE2805940C2 (de) Elektronisches Steuersystem für Analogschaltungen
DE69814482T2 (de) Verfahren und Vorrichtung zur Darstellung von Oberflächen aus volumetrischen Daten
DE2934738A1 (de) Verfahren und geraet zur schnellen rasterbilddatenverarbeitung

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A1

Designated state(s): BR CN JP KR RU US

AL Designated countries for regional patents

Kind code of ref document: A1

Designated state(s): AT BE CH DE DK ES FI FR GB GR IE IT LU MC NL PT SE

CFP Corrected version of a pamphlet front page
CR1 Correction of entry in section i

Free format text: PAT.BUL.11/97 UNDER INID(51)"IPC",REPLACE THE EXISTING SYMBOL BY"G06T 7/00, 7/20"

DFPE Request for preliminary examination filed prior to expiration of 19th month from priority date (pct application filed before 20040101)
121 Ep: the epo has been informed by wipo that ep was designated in this application
122 Ep: pct application non-entry in european phase