Classifications

• • F—MECHANICAL ENGINEERING; LIGHTING; HEATING; WEAPONS; BLASTING
  • F16—ENGINEERING ELEMENTS AND UNITS; GENERAL MEASURES FOR PRODUCING AND MAINTAINING EFFECTIVE FUNCTIONING OF MACHINES OR INSTALLATIONS; THERMAL INSULATION IN GENERAL
  • F16P—SAFETY DEVICES IN GENERAL; SAFETY DEVICES FOR PRESSES
  • F16P3/00—Safety devices acting in conjunction with the control or operation of a machine; Control arrangements requiring the simultaneous use of two or more parts of the body
  • F16P3/12—Safety devices acting in conjunction with the control or operation of a machine; Control arrangements requiring the simultaneous use of two or more parts of the body with means, e.g. feelers, which in case of the presence of a body part of a person in or near the danger zone influence the control or operation of the machine
  • F16P3/14—Safety devices acting in conjunction with the control or operation of a machine; Control arrangements requiring the simultaneous use of two or more parts of the body with means, e.g. feelers, which in case of the presence of a body part of a person in or near the danger zone influence the control or operation of the machine the means being photocells or other devices sensitive without mechanical contact
  • F16P3/142—Safety devices acting in conjunction with the control or operation of a machine; Control arrangements requiring the simultaneous use of two or more parts of the body with means, e.g. feelers, which in case of the presence of a body part of a person in or near the danger zone influence the control or operation of the machine the means being photocells or other devices sensitive without mechanical contact using image capturing devices
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
  • G06V20/00—Scenes; Scene-specific elements
  • G06V20/60—Type of objects
  • G06V20/64—Three-dimensional objects
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
  • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
  • G06V40/20—Movements or behaviour, e.g. gesture recognition
  • G06V40/23—Recognition of whole body movements, e.g. for sport training
• • G—PHYSICS
  • G05—CONTROLLING; REGULATING
  • G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
  • G05B2219/00—Program-control systems
  • G05B2219/30—Nc systems
  • G05B2219/40—Robotics, robotics mapping to robotics vision
  • G05B2219/40202—Human robot coexistence

Definitions

• the invention relates to a method for observing a person in an industrial environment according to the preambles of claims 1 and 3.
• the corresponding approaches are divided into two-dimensional methods, with explicit shape models or model-free, and into three-dimensional methods.
• windows of different sizes are moved over the source image; the corresponding image areas are subjected to a Haar wavelet transformation.
• the corresponding wavelet coefficients are obtained by applying differential operators of different scaling and orientation at different positions of the image region. From this possibly quite large set of features, a small subset of the coefficients are selected "on the fly” based on their magnitude and their local distribution in the image
• This reduced set of features is fed to a support vector machine (SVM) for classification.
• SVM support vector machine
• For the purpose of detection windows of different sizes are pushed over the image and the corresponding features are extracted from these image areas, and the SVM then decides whether or not the corresponding window contains a person
• [5] summarizes temporal sequences of two-dimensional Haar wavelet features into high-dimensional feature vectors and classifies them with SVMs, resulting in a gain in recognition performance over the pure frame approach.
• the method of Chamfer Matching applied to the detection of pedestrian outlines in road traffic scenario in non-stationary camera.
• the Chamfer matching technique is combined with a stereo image processing system and a neural network used as a texture classifier with local receptive fields according to [8] in order to achieve a secure and robust classification result.
• Another group of methods for person detection are model-based techniques that use explicit prior knowledge about the appearance of persons in the form of a model. Since this concealment of parts of the body are problematic, many systems also require prior knowledge of the nature of the detected movements and the camera's perspective.
• the persons are z. B. segmented by subtraction of the background, which requires a stationary camera and a no or only slowly changing background.
• the models used are z. B. from straight bars ("stick figures"), wherein the individual body parts are approximated by ellipsoids
• a method for 3D modeling a person from 2D image data is described in [30].
• image data of a person are detected with the aid of a multi-camera system and their body parts, in particular by means of template matching, identified in the 2D image data.
• the identified body parts are then modeled by dynamic template matching using 3D templates. In this way it is achieved that the persons can be identified quickly and continuously even if they are partially hidden or temporarily not by the multi-camera system could be recorded.
• the identified persons are then tracked in the image data.
• the prior art described above demonstrates that a variety of image processing based methods for recognizing persons in different complex environments, for recognizing body parts and their movements, as well as for the detection of complex, compound parts objects and the corresponding assembly activities are known.
• the applicability of these algorithms is often described only in terms of purely academic applications.
• the object of the invention is to make a camera-based person recognition and modeling of use in an industrial environment accessible.
• image data of the person are detected by means of a multi-camera system.
• This image data is then examined for imaging a person, so that when a person has been detected in the image data, this person hypothesis is adapted to an articulated, virtual 3D model of the human body. Subsequently, this virtual body model is continuously adapted to the movement behavior of the person detected in the image data.
• the position and / or the movement behavior of a person located in the environment of the machine or a machine element is determined. Proceeding from this, a hazard potential can then be determined in knowledge of the position and the movement behavior of the virtual body model in the room. The hazard potential determined in this way is subjected to a threshold value comparison in order to influence the motion control of the machine or of the machine part when this threshold value is exceeded. In a particularly advantageous manner, this action on the motion control of the machine or the machine part will cause its shutdown or its movement slowing down. If only a movement slowdown is effected, the machine or its movable machine element is given the opportunity to continue the operation while reducing the risk potential.
• data are derived therefrom continuously, depending on the current shape and position of the virtual body model, which are correlated with the data of a database.
• the database contains a multiplicity of data, which were ascertained in advance based on the shape and position of a body model during a multiplicity of movement phases describing a movement sequence of a person.
• those movement phases are then taken by the observed person as being taken in, if the data derived from their current body model have a certain degree of similarity with the data stored in the database for this movement phase. Is doing a certain sequence detected motion phases stored in the database, then the movement is considered to be completed by the observed person. If, however, the motion sequence is judged to be incomplete, a corresponding signaling takes place.
• Another advantageous application of this alternative embodiment of the invention is the monitoring of newly trained operating personnel. For example, many production errors occur when new workers have to be trained at short notice during the holiday season.
• the work processes can be observed newly trained operators. It can then be an indication when it is detected that within a motion sequence to be performed necessary movement phases were not taken, so that it must be assumed that a work process was not performed correctly.
• an indication of at least one of the movement phases is given in the context of signaling a judged as incomplete movement sequence, which is considered in the context of checking a correct sequence with respect to the movement sequence as not taken. In this way, it is particularly easy for the observed person to recognize the errors in their movement or in their work execution.
• a trainer can recognize which sections of the learned activity are still difficult for the instructed person and possibly require additional explanations or further training.
• ergonomically problematic movement phases can be detected within a whole sequence of movements in a profitable manner and optionally optimized by changing the sequence of movement phases or by modifications to the observed by the person to be monitored plants or objects ,
• the amount of data to be managed in the database as well as the processing effort can also be reduced by subjecting the image data stored therein to a transformation, in particular a main axis transformation.
• a transformation in particular a main axis transformation.
• the creation of the 3D model of the person is based on 3D point data.
• These point data can be created by multi-image analysis, in particular stereo image analysis.
• stereo image analysis For example, by using a spatiotemporal features based stereo method (such as in
• [28] information is obtained for each 3D point in space that goes beyond its location coordinates (x, y, z), such as its speed or acceleration.
• the segmentation of a plurality of 3D point data (3D point cloud) takes place profitably by means of a cluster method, in particular by means of agglomerative clustering.
• the convex hull is then determined.
• simple features, in particular their height or volume are first determined for each cluster. In this way, invalid, meaningless clusters can then be discarded, in particular on the basis of a priori knowledge about the properties of a natural person.
• an articulated 3D model of the human body It is advantageous to model the body parts by interconnected cylinder.
• the posture of the person is given in this model as a vector of the joint angle of the model.
• the evaluation of a posture is preferably done by determining the divergence between the features derived from the 3D point cloud and the images of the scene and the appearance of the model at a given posture, thereby determining a likelihood that the given posture will be the measured shape of the person reproduces.
• a probabilistic approach to explore the search space is a kernel-based particle filter [29].
• the detected movements of the body parts are represented by motion templates.
• motion templates for 3D Measurement of typical human motion sequences contained representative movement patterns that narrow the space of possible joint angles and joint angular velocities of the person model. In this way, a biologically realistic extrapolation of the movements of the person is possible, especially with the aim of detecting an imminent collision between man and machine.
• a movement process can here be regarded as a composite sequence of movement phases.

Landscapes

• Engineering & Computer Science (AREA)
• Physics & Mathematics (AREA)
• General Physics & Mathematics (AREA)
• Multimedia (AREA)
• Theoretical Computer Science (AREA)
• General Engineering & Computer Science (AREA)
• Psychiatry (AREA)
• General Health & Medical Sciences (AREA)
• Computer Vision & Pattern Recognition (AREA)
• Social Psychology (AREA)
• Human Computer Interaction (AREA)
• Health & Medical Sciences (AREA)
• Mechanical Engineering (AREA)
• Image Analysis (AREA)
• Image Processing (AREA)
• Emergency Alarm Devices (AREA)

Applications Claiming Priority (3)

Publications (1)

Family

ID=38885059

Family Applications (1)

Country Status (6)

Families Citing this family (83)

Family Cites Families (18)

• 2006
  • 2006-10-10 DE DE102006048166A patent/DE102006048166A1/de not_active Withdrawn
• 2007
  • 2007-04-04 EP EP07723978A patent/EP2046537A2/de not_active Ceased
  • 2007-04-04 CN CN2007800332128A patent/CN101511550B/zh active Active
  • 2007-04-04 WO PCT/EP2007/003037 patent/WO2008014831A2/de active Application Filing
  • 2007-04-04 JP JP2009522108A patent/JP2009545789A/ja active Pending
• 2009
  • 2009-01-30 US US12/362,745 patent/US8154590B2/en not_active Expired - Fee Related

Non-Patent Citations (3)

Also Published As

Similar Documents

Legal Events