Classifications

• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
  • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
  • G06V40/20—Movements or behaviour, e.g. gesture recognition
  • G06V40/23—Recognition of whole body movements, e.g. for sport training
  • G06V40/25—Recognition of walking or running movements, e.g. gait recognition

Definitions

• the invention relates generally to verification and identification systems, and more particularly to verification and identification systems employing anatomy and gait parameters.
• Anatomy and gait parameters useful for this purpose include arm and torso length, head roll peak, step length, and cadence. These parameters can be used individually or combined to distinguish the individual by comparing the parameters obtained from an individual to those in a reference database of known individuals. Unsuspecting and uncooperative individuals are unlikely to mask both their external anatomy and their gait characteristics.
• Anatomy and gait parameters can be obtained by first securing an image of the individual from a larger image containing both the individual and his surroundings.
• the larger image can be obtained by using an image acquisition device, such as an opto- electric or video system.
• the form of the individual can be segmented, and raw two dimensional segment coordinates can be extracted.
• triangulation can be performed to convert the two-dimensional data into three-dimensional body coordinates, from which a three-dimensional model of the individual can be constructed from polyhedra to aid in the identification of the individual.
• a method for distinguishing an individual includes acquiring image data of an individual, by using a video camera, for example, and computing an anatomy and/or a gait parameter of the individual from the image data.
• the image data can be segmented, tracked, and sequenced, and, additionally, a three-dimensional model of the individual can be constructed from polyhedra. From the data, a match can be determined between the anatomy and/or gait parameter of the individual and a particular anatomy and/or gait parameter in a reference database to distinguish the individual.
• the system includes an image acquisition device for acquiring image data of an individual, an image data manipulation module for computing a gait parameter of the individual from the image data, and a distinguishing module for determining a match between the gait parameter of the individual and a particular gait parameter in a reference database.
• Figure 1 is schematic block diagram of a system for distinguishing an individual, according to the teachings of the present invention.
• Figure 2 is a schematic block diagram of the image data manipulation module of Fig. 1, according to the teachings of the present invention.
• Figure 3 shows details pertaining to the function of the segment tracking/sequencing unit of Fig. 2, according to the teachings of the present invention.
• Figure 4 is a graphical representation of a between-subjects probability density function, and a within-subjects probability density function, according to the teachings of the present invention.
• Figure 5 A and 5B are graphical representations of a gait cycle plot and a gait stance plot, according to the teachings of the present invention.
• Figure 6 is a graphical illustration showing a polyhedron used to construct a three-dimensional body, according to the teachings of the present invention.
• Figure 7A and 7B show a three-dimensional body model of an individual represented by eleven polyhedra, according to the teachings of the present invention.
• Figure 8 shows a flow chart for distinguishing an individual, according to the teachings of the present invention. Detailed Description of the Invention
• FIGs 1 through 8 illustrate an example embodiment of a system and method suitable for distinguishing individuals by utilizing anatomy and gait parameters.
• FIG. 1 illustrates an example embodiment of a system and method suitable for distinguishing individuals by utilizing anatomy and gait parameters.
• a distinguishing system 8 is shown for distinguishing individuals utilizing anatomy or gait parameters.
• An image acquisition device 10 is utilized to obtain image data 12 of an individual in a particular setting.
• the image acquisition device 10 can include any sensor that can capture, obtain, or receive image data 12 of an individual to obtain anatomy or gait information.
• the image acquisition device 10 can include a video camera for taping the individual at a selected location.
• an image acquisition device 10 can include a magnetic resonance device for obtaining image data of an individual.
• suitable devices include CCD cameras and the like.
• the image data can also be inputted to the image acquisition device via any suitable communication links, such as a network connection, and hence need not be a camera.
• the illustrated distinguishing system 8 also includes an image data manipulation module 16 that employs hardware and software to compute an anatomy and/or gait parameter from the image data 12.
• a gait parameter is any property that is derived from the motion of the individual that can be used to identify the individual.
• a gait parameter can be obtained from one or more selected measurements of the individual at more than one time, such as head roll peak, head roll range of motion, trunk pitch, arm-to-leg swing time, and cadence, but can also be obtained from a static measurement of the individual in motion, such as stride length.
• the distinguishing system 8 can also include a reference database 18 that contains selected data, such as names, social security numbers, or other identifiers that allow a person to be identified, and associated anatomy or gait parameters.
• the distinguishing module 20 includes software and hardware for distinguishing the individual by using the anatomy and/or gait parameter of the individual and the reference database 18. Distinguishing an individual includes both positively identifying an individual, as well as excluding an individual by determining that there is no match between parameters obtained from the image data 12 and those in the reference database 18.
• the image acquisition device 10 functions to obtain, receive or capture image data 12 of the individual in a particular setting.
• the image data 12 may then be processed by the image data manipulation module 16 to extract an anatomy and/or gait parameters of the individual.
• several anatomy and/or gait parameters are used to distinguish an individual.
• the distinguishing module 20 determines whether acquired anatomy and/or gait parameters match, within specified tolerances, a respective parameter stored in the reference database 18. If there is a match, then the individual can be positively identified by using the personal identification associated with the matched parameter(s). If there is no match, then the individual is not included among the individuals identified in the reference database.
• anatomy and/or gait parameters By utilizing anatomy and/or gait parameters, individuals can be distinguished for many useful purposes. For example, terrorists at an airport can be identified as potential threats by identifying them based on their anatomy and/or gait. This technique is less intrusive then requiring someone to submit to fingerprinting, or signature analysis.
• anatomy and/or gait parameters can be used to give an individual clearance to an area.
• an individual may be given access to the room after being positively identified using anatomy and/or gait parameters obtained from images according to the principles of the present invention.
• the image data manipulation module 16 which includes hardware and software to extract anatomy and/or gait parameters from the image data 12, includes a data collection and pre-processing unit 30, an image segmentation and identification unit 32, and a segment tracking/sequencing unit 34.
• the data collection and pre-processing unit 30 collects the acquired image data and performs selected image adjustments and filtering of the data. Images recorded from a high speed, high-resolution video cameras are acquired for individuals walking under a variety of circumstances. A frame grabber device is used to segment the analog video stream into digital video clips. Collected data for individual trials consist of a set of 3 to 5 second digital video clips and a set of calibration trials. The calibration trials consist of video footage of a walkway with a set of calibration markers in the field of view of the camera. These data are used to scale humans to their surroundings. To enhance image properties for segmentation and to compensate for lighting conditions, different filtering techniques can be used. Edges and other sharp changes in intensity are associated with high frequencies.
• Frequency filtering using Fourier transforms, is used to attenuate low frequencies, sharpening the image for edge detection. Background subtraction is applied most easily in a controlled environment where the background is known and thus can be subtracted from any images captured after an individual is introduced to the scene. In cases where the background is slowly changing but the target is moving faster, this technique can also be used.
• the image segmentation and identification unit 32 employs edge detection and edge relaxation techniques to contour the region of interest within an image.
• Edge detection techniques such as gradient, Laplacian, and Canny among others, are used to identify image boundaries such as hands, feet, trunk, and arms. As most images have a few locations where the gradient is zero, thresh-holding schemes, known to those of ordinary skill in the art, are employed.
• the following example illustrates the application of a simple gradient edge detector that can be used for features detection. First, a 3 x 3 gradient edge detector is applied to the binary image data with a matrix having values:
• the gradient edge operator is applied to each pixel in the image by moving the center of the mask (filter) from pixel to pixel in the image. In each location, the sum of the product of each cell in the mask and the corresponding pixel is given by
• X represents the color/gray level value of the original pixel/image
• Wj is the value of the z ' th weight in the 3x3 mask/filter.
• the result of this operation is stored in a new file as an edge image.
• the edge detector mask can also be used to detect straight lines by changing the weights to be more sensitive to these lines.
• the segment tracking/sequencing unit 34 helps to detect motion of various anatomical parts of the individual, such as the head, feet, and hands. Once the extremity endpoints are identified, the posture and gait of the individual can be obtained for distinguishing the individual from a collection of persons listed in the reference database.
• the data collection and pre-processing unit 30 employs image data corresponding to an image of an individual.
• the corresponding gray scale image can be used for distinguishing the individual.
• the image can be represented by the intensity of the image at a pixel.
• a computer software tool can be utilized to read pixel data pertaining to the image from a variety of image formats.
• the image can be a 24-bit red, green and blue (RGB) color image. RGB values for each pixel are summed to represent the color value.
• Data can be stored in a new file containing the RGB value of each pixel in the image. For an image size of 480 x640, for example, each pixel is represented as three, eight-bit numbers.
• Histogram equalization with 255 gray level bins may be used to adjust the red, green and blue colors for generating the gray scale image, which may then be processed further for distinguishing the individual. It is highly likely that color information from the video surveillance images will be informative. Color image files are large but easily mapped into a gray scale to produce a gray scale image. In another embodiment, the color of the image can be used for facial recognition or other stages of processing.
• the head 40, hands 42, and feet 44 can be detected from an image obtained at sequential times. As an individual walks, for example, images can be taken at three sequential times producing the sequence of three head, hand, and feet locations 40A-40C, 42A-C, and 44A-C.
• the image acquisition device 10 is responsible for acquiring the image data 12 of the individual.
• the image acquisition system can include video cameras, but can also include infrared sensors for capturing the position of the head 40 and hands 42 day or night. An infrared sensor can also detect thermal footprints.
• the posture of the individual can be obtained. Since the hands and feet normally move anti-phase during gait, when the right foot is ahead of the head, the left hand is ahead of the head. Contouring and edge detection can be used to surround the body and generate a full-body polyhedral model.
• the practical ability to develop a polyhedral model to represent the individual depends on the ability to isolate body segments (arms, trunk and legs).
• One approach for isolating body segments involves a template/block matching algorithm, and techniques such as the Generalized Hough transformation.
• Objects in an image such as human body segments (head, trunk, arms, and legs) are template matched based on Euclidian distance and cross correlation. Scaling of the template may be performed based on the size of the image (determined with calibration).
• the expected shape of human body segments is known and their orientation can be logically estimated. In cases where separation of the body segments, such as head 40, hands 42, or feet 44 is difficult, for instance where the legs overlap or arms cross the trunk, template matching and the Hough transform may not be appropriate.
• Region growing techniques seek image areas with pixels of the same or similar features. Techniques available for region growing include local techniques, such as blob coloring, global techniques, such as histogram thresh-holding, and splitting and merging techniques. Once the polyhedral model of the individual exists, the model can be used to distinguish the individual using anatomy and gait parameters, and a reference database 18.
• Anatomy and gait parameters can be used to distinguish individuals.
• An opto- electric system can be used to establish the anatomical and gait parameters that best discriminate among individuals.
• a lOKHz active-marker tracking system consisting of four opto-electric cameras (Selspot II, Selective Electronics Inc. Partille, Sweden) can be used for tracking arrays of infrared light emitting diodes (irLEDs).
• the irLEDs are strapped to eleven body segments (e.g., both feet, shanks, thighs and upper arms, and the pelvis, upper trunk and head).
• Each array is a rigid plastic disk with 3-to-5 embedded irLEDs that allows the determination of all six degrees of freedom (DOF), three rotations and three translations, of each of the eleven body segments (total of 64 irLEDs) at 150 Hz.
• the precision of the system is ⁇ lmm in translation and ⁇ 1 deg in rotation.
• the raw two-dimensional irLED data from each of four cameras can be utilized to generate 3-D "body segment" kinematics.
• a computer program can automatically, without user input for body part tracking, fit a "standard" body configuration of 11 polyhedra to the anatomy of the individual, determining the anatomic (length, width, volume) and inertial (mass and mass moment) properties of the body segments.
• Six degrees of freedom (6 DOF) kinematics of body segments e.g., trunk and head rotations
• relative movements among segments e.g., neck or knee flexion
• spatio-temporal parameters e.g., cadence, velocity and step length
• Biometric data consists of m anatomical and gait parameters for n individuals. Furthermore, each individual undergoes q repeated measures of each parameter. Thus any single measurement for an individual can be denoted j i k- The within-subjects mean and standard deviation can be computed from each individual's repeated measures assuming all parameters are measured q times. An average over the repeated measurements, and the associated standard deviation is given by
• the between-subjects standard deviation, sfc, and the average within-subjects standard deviation, s ws may be used for analysis by seeking anatomy and gait parameters that yield a small s ws (a variable which is relatively invariant) relative to the variance across subjects sfa.
• the parameters chosen to identify individuals have large precision, but wide between-subjects distributions.
• FIG. 4 graphs are shown of a between-subjects probability density function 50, and a within-subjects probability density function 52.
• the within-subjects probability density 52 has a standard deviation of s ws
• the between-subjects probability density 50 has a standard deviation of sf ⁇ .
• the probability of inclusion is evaluated within the boundary set by the within-subjects standard deviation.
• a single measurement extracted from a sensor-software system of an unknown individual can be denoted by X, and the population mean of x ⁇ for many humans (j - 1, 2, ... n) can be denoted by x.
• the standard deviation of the population mean is sfc.
• the measurement X is bounded by Zws s ws > where z-ws * s a population standardized score based on the level of confidence desired, creating a search region having a prescribed probability of enclosing the true matching value (x) of X.
• the search region encloses a certain percentage of the population who do not have matching values of X.
• the percentages from the cumulative z-distribution give 11.8% for height, 30.8% for cadence and 45.5% for trunk yaw.
• the probability of there being a matching value of x in both the height and cadence regions is 3.6%, and inclusion of trunk yaw further reduces the probability to 1.7%. In a data base of 100 identified humans, approximately 2 would be flagged for these characteristics. From this type of analysis useful anatomy and gait parameters can be identified for distinguishing an individual.
• the gait parameters consist of body segment movement summaries (such as peak rotation angles and range of motion), postural summaries (relative alignment of segments) and spatio-temporal parameters (such as step length and cadence).
• body segment movement summaries such as peak rotation angles and range of motion
• postural summaries relative alignment of segments
• spatio-temporal parameters such as step length and cadence.
• cycle time refers to the time that encompasses a full cycle of movement (such as heel strike-to-heel strike off the same foot)
• stance time refers to the time when the foot is in contact with the ground.
• force platforms embedded into the floor, or foot switches (on-off pressure switch) are used to document these time events.
• the gait cycle plot 60 is determined from the time of peak knee flexion-to-peak knee flexion of the same leg in the sagittal (side) plane view. Virtually any periodic event can be used to document cycle time (knee flexion is quite reliable). Therefore, should the knees not be visible (e.g., masked by a dress or coat), other events such as peak head vertical displacement can also be used.
• the gait stance plot 62 is shown, which involve the times when the foot contacts and leaves the ground.
• Foot center of mass (CoM) vertical acceleration can be used to determine "heel strike” and "toe off' events, as shown in Fig. 5B, with an average error of 7 to 13 ms (equivalent to 1 to 2 frames with a 150 Hz acquisition system).
• heel strike and toe off times are known, the toe off and heel strike times of the contralateral foot can then be determined (they occur between the heel strike and toe off times of the ipsilateral foot).
• a variety of parameters are selected to serve as biometric anatomy and gait parameters for distinguishing an individual.
• the anatomy and gait parameters are measurable from a three-dimensional body model that consists of 11 polyhedra.
• one polyhedron 70 is shown, whose position is characterized by six degrees of freedom.
• Such a three dimensional structure can be obtained from two dimensional video images, for example, by triangulation, provided more than one video camera is utilized.
• the polyhedron 70 corresponds to a torso of an individual.
• the six degrees of freedom are shown in a degrees of freedom coordinate system 72, and include three translational coordinates of the center of mass, and three rotation coordinates.
• the position of the 11 polyhedra representing various body parts can be used to ascertain useful anatomy and gait parameters that can be utilized to distinguish an individual.
• HSH Head-to-shoulder width ratio
• Useful gait parameters include:
• Head roll peak (HRP) peak roll angle (front view angle) of the head during the gait cycle
• Head roll RO (HRR) range of motion (ROM) of head roll during the gait cycle
• Trunk roll, pitch and yaw RO (TRR, TPR, TYR) ROM of trunk roll, pitch and yaw during the gait cycle;
• Arm abduction angle (AAA) average abduction angle (front view angle) of the arms relative to the trunk during the gait cycle;
• Foot internal / external rotation (FTR) Internal/ external rotation (yaw, above view angle) of the foot relative to the room at heel strike;
• Step length (STL) anterior (parallel to direction of progression) distance between ankle joint centers of left and right feet when flat on the floor;
• Step width (STW) lateral (normal to direction of progression) distance between ankle joint centers of left and right feet when flat on the floor;
• Gait velocity (GVL) average forward velocity of the body's combined center of mass during stance phase of gait;
• Cadence (CAD) number of steps per minute;
• Heel strike-foot flat time the time from heel strike to when the foot is flat on the floor (time between heel strike and opposite toe off).
• a three-dimensional body model 80 of an individual represented by 11 polyhedra is shown.
• a profile 82 and front view 84 of the body model 80 representing the individual is shown.
• Combined and individual segment lengths, and gait parameters can be determined from the body model 80. For example, arm length is
• Mass estimation which together with other parameters can be utilized to distinguish individuals, is performed by first calculating the volume of the polyhedra used to model each body segment, then multiplying by their respective body segment densities tabulated in standard reference manuals. The resulting mass estimations across all eleven segments are then summed to obtain the total mass. Algorithms can perform these regression fits from tape-measure diameters, lengths and anatomical landmark information, automatically, without further user input. Standing height and body weight can be estimated to ⁇ 2 cm and ⁇ 5 kg, respectively, of actual values.
• the distinguishing module 20 of Fig. 1 processes the information obtained by the image data manipulation module 16 to distinguish an individual. Distinguishing the individual can mean either positively identifying the individual, if there is a match with parameters in the reference database 18, or negatively identifying the individual, if there is no match with parameters in the reference database 18. Whether or not there is a match is determined within some tolerance.
• the tolerances can be chosen based on the sensitivity and specificity sought.
• the sensitivity is the ratio of true positives to the sum of true positives and false negatives
• specificity is the ratio of true negatives to the sum of true negatives and false positives.
• the positive predictive value (PV + ) and the negative predictive value (PV_), as defined below, may also be computed.
• Sensitivity measures the ability of a test to give a positive identity match if the target human really is in the database. Specificity is the ability of the test to give a negative match when the enrolled human really does not exist in the database. The predictive values are indicative of the accuracy.
• the PV + is the likelihood that the positively matched individual really exists in the database
• the PV. is the likelihood that the negatively matched individual really does not exist in the database.
• the methods and systems of the present invention would yield high sensitivity, specificity and predictive values. Increasing sensitivity decreases specificity. Specificity is important to avoid unnecessary costs and suffering, such as the detainment of an innocent individual and the costs associated with that action. Receiver operator curve analysis, known to those of ordinary skill in the art, can be used to balance sensitivity and specificity.
• a flow chart is shown for distinguishing individuals utilizing anatomy or gait parameters.
• an image of an individual is acquired using the image acquisition device 10.
• an anatomy or gait parameter of the individual is computed with the help of the image data manipulation module 16.
• a match between the gait parameter of the individual and a particular gait parameter in the reference database 18 is determined with the distinguishing module 20 to distinguish the individual.

Landscapes

• Engineering & Computer Science (AREA)
• Health & Medical Sciences (AREA)
• Computer Vision & Pattern Recognition (AREA)
• General Health & Medical Sciences (AREA)
• Psychiatry (AREA)
• Social Psychology (AREA)
• Human Computer Interaction (AREA)
• Physics & Mathematics (AREA)
• General Physics & Mathematics (AREA)
• Multimedia (AREA)
• Theoretical Computer Science (AREA)
• Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)

Priority Applications (1)

Applications Claiming Priority (2)

Publications (1)

Family

ID=22710817

Family Applications (1)

Country Status (3)

Cited By (5)

Families Citing this family (41)

• 2001
  • 2001-03-27 WO PCT/US2001/009822 patent/WO2001073680A1/en active Application Filing
  • 2001-03-27 AU AU2001245993A patent/AU2001245993A1/en not_active Abandoned
  • 2001-03-27 US US09/819,149 patent/US20020028003A1/en not_active Abandoned

Non-Patent Citations (3)

Also Published As

Similar Documents

Legal Events