WO2007118764A1 - Procédé de classification d'un mouvement d'un objet - Google Patents

Procédé de classification d'un mouvement d'un objet Download PDF

Info

Publication number
WO2007118764A1
WO2007118764A1 PCT/EP2007/052834 EP2007052834W WO2007118764A1 WO 2007118764 A1 WO2007118764 A1 WO 2007118764A1 EP 2007052834 W EP2007052834 W EP 2007052834W WO 2007118764 A1 WO2007118764 A1 WO 2007118764A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
optical flow
video
movement
video images
Prior art date
Application number
PCT/EP2007/052834
Other languages
German (de)
English (en)
Inventor
Andreas Simon
Original Assignee
Robert Bosch Gmbh
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Robert Bosch Gmbh filed Critical Robert Bosch Gmbh
Publication of WO2007118764A1 publication Critical patent/WO2007118764A1/fr

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20—Movements or behaviour, e.g. gesture recognition

Definitions

  • the invention relates to a method for classifying a movement of an object, a device for classifying a movement of an object, a computer program and a computer program product.
  • US 2002/0102024 A1 describes a so-called trained procedure for retrieving patterns in individual digital images. Such approaches are also known under the terms “adaptive boosting” and "Adaboost".
  • a pattern of a single image to be found is determined by means of a training set of positive examples for a respective pattern class, e.g. for vehicle rear views, compared for similarity with these positive examples.
  • the cited document discloses a method and a system for the detection of
  • An image integrator receives the one image and calculates an integral image for this image.
  • An image scanner scans the image subdivided into image sections of the same size.
  • An object recognizer uses for the classification of the image sections linked together homogeneous classification functions and checks whether each image section with a certain probability includes a feature of the object.
  • Figures 4 to 6 are diagrams for describing a trained method for retrieving patterns in digital images.
  • FIG. 4 the same portrait photograph of a person is shown in the lower three images 102, 104, 106, the portrait image in the middle image 104 being superimposed by a first image feature 108, as also shown in FIG.
  • the portrait shot in the right image 106 is correspondingly by a second image feature
  • a set of significant rectangular image features 108, 112 is determined and thus learned. These image features 108, 112 are processed in the direction of decreasing significance during a processing phase for each image region to be examined.
  • the classification of a searched object e.g. a face or a view of a vehicle is successful in this image 102, 104, 106 if none of the image features contradicts a previously established object hypothesis for the image 102, 104, 106.
  • FIG. 5 shows five further images 116, 118, 120, 122, 124.
  • these five images 116, 118, 120, 122, 124 in each case a number of adjacent image regions are shown, from which the image features depicted here are typically composed. Within these image regions, a sum of pixel gray values is first calculated in each case, after which the calculated sums are weighted and added.
  • FIG. 6 shows a section 126 of an image with an image region 128 which is delimited by four corner points 130, 132, 134, 136.
  • an integral image is initially calculated.
  • the sum of all the pixel gray values that an originally rectangular sub-image contains is determined.
  • the left one is above a respective considered pixel.
  • Pixel in the image region D denoted by 128 is calculated by four accesses to one integral image I at the vertices 130, 132, 134, 136 of the image region 128 by the following equation (1):
  • the present invention relates to a method having the features of patent claim 1, a device having the features of patent claim 9, a computer program having the features of patent claim 10 and a computer program product having the features of patent claim 11.
  • the device according to the invention for classifying a movement of an object in a video sequence from at least two video images is designed to describe the movement by an optical flow
  • this device can have at least one module designed in particular for data processing for carrying out at least one step of the method.
  • the invention is typically suitable for classifying movements that perform or cause objects, such as people or vehicles, from the first video image to the second video image of the video sequence. Taking into account the optical flow derived from the video images, a description or classification and thus the classification of the movement, for example by means of a comparison with predetermined, in particular object-specific movement patterns, is possible. Furthermore, the invention may be used in general terms for any image sequence comprising a usually temporal sequence of a number of images. Such images and image sequences can be present and / or provided in suitable image-forming formats, for example also as television images, in particular digitized.
  • the present invention it is now possible to extend methods of static gray scale distribution in the video image to movements in video sequences comprising at least two consecutive video images. This makes it possible to classify or describe significant movements in a video sequence for, in particular, immediately consecutive video images.
  • the invention may, for. B. for detecting pedestrians or vehicles, eg. In a vehicle rear view, and their movement can be used.
  • the motion in video sequences is described in the practice of the invention by the optical flow which includes a displacement of the pixels from a video image to a subsequent video image, typically the direct sequential image.
  • the optical flow is regularly defined as a two-dimensional displacement vector field or vector field between two different video images.
  • this optical flow may generally be referred to as a vector function which associates a vector or displacement vector with each pixel or pixels from a first video image to a subsequent video image taking into account a dynamic change of the respective pixel.
  • a vector in particular displacement vector, a shift and thus a change of a gray value with respect to the pixels can be described. It can be made a statement about where a gray value of a respective Pixel and / or where this gray value moves in the course of the video sequence.
  • the optical flow can thus describe the movement of an object between at least two video images of the video sequence.
  • the respective displacement vector for each pixel it is not possible to calculate the respective displacement vector for each pixel, but at least only a portion of all the pixels of at least one video image of the video sequence. This may be the case, for example, in the case of weakly or not at all textured surfaces or areas of the at least one video image.
  • the optical flow is thus not necessarily definable for each pixel.
  • a so-called synthetic integral image is provided by using the optical flow, with which a temporal change or dynamics and thus also the movement can be taken into account in comparison to conventional and thus static integral images.
  • the possibility of a fast classification of static image contents, typically of imaged objects, to the motion in the image caused by these objects can now be extended.
  • the pixels in the video images are arranged in rows and columns.
  • lines in a first spatial direction here in the x-direction
  • the columns in a second spatial direction here in the y-direction, oriented.
  • the optical flow is decomposed into three scalar components:
  • w (x, y) 0 if the optical flux at the point (x, y) is undefined.
  • the values between 0.0 and 1.0 can optionally be used for a confidence weighting, for example a confidence interval.
  • the optical flow weighted with w (x, y) in the two directions x and y as well as the weighting measure are integrated or summed component by component:
  • Equation (5) can be used generally for calculating image features. It has the same structure as the equation (1) already presented in the prior art:
  • Pl, P2, P3 and P4 are corner points of the rectangular image region P, for each of which the synthetic integral image I is to be calculated. (For this, reference is made to the example of FIG. 6: accordingly, P corresponds to the image region 128, Pl to the corner 130, P2 to the corner 132, P3 to the corner 136 and P4 to the corner 134)
  • An average optical flux F for an image region can thus be calculated taking into account equation (5).
  • the integral flux Fx and Fy for the image region is first calculated and then divided by the integral weighting measure W for the image region:
  • Fx (P) Fx (P1) - Fx (P2) - Fx (P3) + Fx (P4) (6)
  • Fy (P) Fy (P1) - Fy (P2) - Fy (P3) + Fy (P4) (7)
  • W (P) W (Pl) -W (P2) -W (P3) + W (4) (8)
  • fx (P) Fx (P) / W (P) (9)
  • fy (P) Fy (P) / W (P) (10)
  • w (x, y) assumes only the binary values 0.0 or 1.0, so that the weighting measure provides a binary weighting mask.
  • W (P) corresponds to the number of vectors of optical flow in the image region P.
  • Figure 1 shows a schematic representation of an embodiment of video images for calculating the optical flow.
  • FIG. 2 shows a schematic representation of images for the vectorial decomposition of the optical flow of the exemplary embodiment from FIG. 1.
  • FIG. 3 shows a schematic representation of a weighting mask of the optical flow of the exemplary embodiment from FIG. 1.
  • FIG. 4 shows a first schematic representation of a procedure according to the prior art.
  • FIG. 5 shows a second schematic representation of a procedure according to the prior art.
  • FIG. 6 shows a third schematic representation of a procedure according to the prior art.
  • FIG. 1 shows a schematic representation of an exemplary embodiment of a first video picture 2 of a video sequence provided as an original picture and of a video sequence
  • Next image provided second video image 4 of the video sequence.
  • the two video images 2, 4 are subdivided in this embodiment into pixels 6, wherein these pixels 6 are arranged in the respective video image 2, 4 to seven lines with eight columns.
  • the pixels are either white, gray or black marked or filled.
  • a shift vector 8 refers to a first pixel 6 of the first video image 2 and a second pixel 6 of the second video image 4.
  • This shift vector 8 is in consideration of the shift of the gray value between the first and second pixels 6 of the first and second Video images 2, 4 aligned between these two pixels 6, so that these two pixels 6 from the two consecutive video images 2, 4 are connected by the displacement vector 8.
  • Pixels 6 in the second column from the right in the second video image 4 center points on. As a comparison with the first video image 2 shows, gray values of these pixels 6 are unchanged, which is marked by the dots.
  • the displacement vectors 8 in their entirety form an optical flow between the two video images 2, 4 of the video sequence.
  • a shift vector 8 is assigned by the optical flow at least the changing pixels 6 within the video sequence, which describes how the pixels 6 from the first video image 2 to the second video image 4 move.
  • the different gray values represent at least one object in each video picture 2, 4. Movements of the at least one object are reflected by the shift of the gray values of the pixels 6 from video image 2 to video image 4. By detecting the displacement vectors 8 and thus providing the optical flow between two video images 2, 4, the motion of the at least one object is described and thus classified.
  • FIG. 2 shows a schematic representation of exemplary embodiments of a first partial image 10 and of a second partial image 12, which are likewise constructed from pixels 6.
  • the first partial image 10 shows a horizontal component of the optical flow from the second video image 4.
  • This horizontal component comprises horizontal displacement vectors 14 which are oriented along the columns.
  • the second subpicture 12 shows a vertical portion of the optical flow of the second video image 4.
  • the vertical portion comprises vertical displacement vectors 16 oriented along the lines.
  • Video image 4 is to be calculated by vector addition of the horizontal displacement vectors 12 and the vertical displacement vectors 14.
  • the pixels 6 which are unchanged with respect to the gray values are identified by dots in the two partial images 10, 12 analogously to the second video image 4, in each case in the middle.
  • FIG. 3 shows a schematic representation of an exemplary embodiment of a weighting mask 18 for image weighting of the optical flow from the first video image 2 to the second video image 4 from FIG. 1.
  • this weighting mask 18 By means of this weighting mask 18, each pixel 6 is given a value that is greater than or equal to 0 and less than 1 is assigned.
  • the weighting mask 18 present here has only binary values, so that 6 zeros are assigned to the pixels 6 shown here in white, and 18 ones are assigned to the gray-shaded pixels.

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Health & Medical Sciences (AREA)
  • Psychiatry (AREA)
  • Social Psychology (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • Image Analysis (AREA)

Abstract

L'invention concerne un procédé de classification d'un mouvement d'un objet dans une séquence vidéo d'au moins deux images vidéo (2, 4), dans lequel le mouvement est décrit par un flux optique. L'invention concerne en outre un dispositif de classification d'un mouvement d'un objet dans une séquence vidéo d'au moins deux images vidéo (2, 4). Ce dispositif est conçu pour décrire le mouvement via un flux optique.
PCT/EP2007/052834 2006-04-13 2007-03-23 Procédé de classification d'un mouvement d'un objet WO2007118764A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
DE102006017452A DE102006017452A1 (de) 2006-04-13 2006-04-13 Verfahren zur Klassifizierung einer Bewegung eines Objekts
DE102006017452.6 2006-04-13

Publications (1)

Publication Number Publication Date
WO2007118764A1 true WO2007118764A1 (fr) 2007-10-25

Family

ID=38197788

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/EP2007/052834 WO2007118764A1 (fr) 2006-04-13 2007-03-23 Procédé de classification d'un mouvement d'un objet

Country Status (2)

Country Link
DE (1) DE102006017452A1 (fr)
WO (1) WO2007118764A1 (fr)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102013217648A1 (de) * 2013-09-04 2015-03-05 Conti Temic Microelectronic Gmbh Kamera-basierte erkennung eines an einem fahrzeug montierbaren trägersystems
CN109697409B (zh) * 2018-11-27 2020-07-17 北京文香信息技术有限公司 一种运动图像的特征提取方法及起立动作图像的识别方法

Non-Patent Citations (6)

* Cited by examiner, † Cited by third party
Title
JÄHNE, BERND: "Digital Image Processing, Chapter 14: "Motion"", 2002, SPRINGER VERLAG, BERLIN, GERMANY, ISBN: 3-540-67764-2, XP002441255 *
KENJI MASE: "RECOGNITION OF FACIAL EXPRESSION FROM OPTICAL FLOW", IEICE TRANSACTIONS, INSTITUTE OF ELECTRONICS INFORMATION AND COMM. ENG. TOKYO, JP, vol. E74, no. 10, 1 October 1991 (1991-10-01), pages 3474 - 3483, XP000279328, ISSN: 0917-1673 *
TEMUJIN GAUTAMA ET AL: "A Phase-Based Approach to the Estimation of the Optical Flow Field Using Spatial Filtering", IEEE TRANSACTIONS ON NEURAL NETWORKS, IEEE SERVICE CENTER, PISCATAWAY, NJ, US, vol. 13, no. 5, September 2002 (2002-09-01), XP011077303, ISSN: 1045-9227 *
VIOLA P ET AL: "Detecting pedestrians using patterns of motion and appearance", PROCEEDINGS OF THE EIGHT IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION. (ICCV). NICE, FRANCE, OCT. 13 - 16, 2003, INTERNATIONAL CONFERENCE ON COMPUTER VISION, LOS ALAMITOS, CA : IEEE COMP. SOC, US, vol. VOL. 2 OF 2. CONF. 9, 13 October 2003 (2003-10-13), pages 734 - 741, XP010662435, ISBN: 0-7695-1950-4 *
YACOOB Y ET AL: "RECOGNIZING HUMAN FACIAL EXPRESSIONS FROM LONG IMAGE SEQUENCES USING OPTICAL FLOW", IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, IEEE SERVICE CENTER, LOS ALAMITOS, CA, US, vol. 18, no. 6, 1996, pages 636 - 642, XP000884587, ISSN: 0162-8828 *
YAN KE ET AL: "Efficient Visual Event Detection Using Volumetric Features", COMPUTER VISION, 2005. ICCV 2005. TENTH IEEE INTERNATIONAL CONFERENCE ON BEIJING, CHINA 17-20 OCT. 2005, PISCATAWAY, NJ, USA,IEEE, 17 October 2005 (2005-10-17), pages 166 - 173, XP010854785, ISBN: 0-7695-2334-X *

Also Published As

Publication number Publication date
DE102006017452A1 (de) 2007-10-18

Similar Documents

Publication Publication Date Title
DE102018116111A1 (de) Ein einheitliches tiefes faltendes neuronales Netzwerk für die Abschätzung von Freiraum, die Abschätzung der Objekterkennung und die der Objektstellung
DE102017220307B4 (de) Vorrichtung und Verfahren zum Erkennen von Verkehrszeichen
DE69024537T2 (de) Infrarot-Überwachungssystem
DE102007035884B4 (de) Linienrauschunterdrückungsvorrichtung, -verfahren und -programm
EP0645037B1 (fr) Procede permettant de detecter des modifications survenant dans des images en mouvement
EP1316057B1 (fr) Evaluation d'informations de direction d'aretes
DE2909153A1 (de) Einrichtung zur elektronischen verarbeitung von bild- und/oder zeichenmustern
EP1770635A1 (fr) Méthode itérative d'interpolation d'image
DE102012209316A1 (de) Verfahren und Vorrichtung zum Verarbeiten von Sensordaten eines Stereosensorsystems
DE102018217091A1 (de) Verfahren, künstliches neuronales Netz, Vorrichtung, Computerprogramm und maschinenlesbares Speichermedium zur semantischen Segmentierung von Bilddaten
DE102005025220B4 (de) Gerät, Verfahren und Programm zum Beseitigen von Poren
WO2008034599A2 (fr) Procédé et dispositif de traitement d'image
DE102017112333A1 (de) Verbesserung eines pyramidalen Optical-Flow-Trackers
WO2007118764A1 (fr) Procédé de classification d'un mouvement d'un objet
DE112016007447T5 (de) Verfahren zum Vergleichen einer ersten Zeichnung mit einer zweiten Zeichnung durch Vergleichen ihrer Zeichnungselemente
DE102011007644A1 (de) Verfahren und Vorrichtung zur Bestimmung von zur Entzerrung eines Bildes geeigneten Werten und zur Entzerrung eines Bildes
EP2893510B1 (fr) Procédé et processeur d'images pour eliminer un objet visuel d'un image
DE102016124123A1 (de) Verfahren zum Erzeugen eines Clusters von Bildbereichen zum Erfassen eines Zielobjekts in einem Bild, Computerprogrammprodukt, Kamerasystem und Kraftfahrzeug
DE60225268T2 (de) Verfahren zum bestimmen von textfeldern in einem videobild
DE102021212483A1 (de) Datenverarbeitungsvorrichtung sowie Verfahren und Programm zum tiefen Lernen eines neuronalen Netzwerkes
DE19802261A1 (de) Verfahren zur Signalverarbeitung von Zeitfolgen digitalisierter Bilder
DE102019127622A1 (de) Abwehrgenerator, Verfahren zur Verhinderung eines Angriffs auf eine KI-Einheit und computerlesbares-Speichermedium
DE60028884T2 (de) Videosignalverarbeitung
DE10041037A1 (de) Bildkodierungsverfahren und Bildkodierer
DE102018213052A1 (de) Verfahren und Vorrichtung zum Ermitteln einer Erklärungskarte

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 07727308

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 07727308

Country of ref document: EP

Kind code of ref document: A1