JP2013064671A - Measurement system and measurement method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a measurement device capable of accurately identifying and classifying an object while tracking a moving object within an observation region using a distance measurement sensor.SOLUTION: A measurement device 100 includes: a plurality of laser range finders 10.1 to 10.4 arranged so as to measure a horizontal distance to an object; a tracking module 5610 which estimates a position and a moving velocity of the object from a measurement result; a feature extraction and calculation section 5622 which calculates a feature vector representing a shape of the object on the basis of the measurement result; a prior probability calculation section 5624 which calculates a probability that the object belongs to each predetermined classification class as a prior probability on the basis of the feature vector; and a label allocation processing section 5628 which determines whether or not the object is in a synchronized state where the object moves in synchronization with another object and also determines the classification class to which the object belongs based on the calculated prior probability as well as likelihood that the object in the synchronized state corresponds to each classification class.

Description

  The present invention relates to a measurement system and a measurement method, and more particularly to, for example, a measurement system and a measurement method for specifying a measurement target while tracking the position and shape of the measurement target.

  With the recent development of robot technology, future robots are required to play a role in coexistence with humans. For this purpose, the robot needs various functions, for example, a function for the robot itself to judge and understand the environment.

  In the field of environmental intelligence, environmental sensors and the like need to be designed to promote services provided to users and maximize convenience. For that purpose, not only healthy users but also users with specific needs need to be considered extensively. When elderly people or users with handicap use a variety of assistive devices such as walker, wheelbarrow, wheelchair, etc., for example, in shopping centers, use users instead of stairs or escalators There is a possibility that the need to guide the user to do so will occur. In such a case, a technique for identifying a moving object in a certain environment is required.

  Therefore, pedestrian detection and tracking has become a popular research topic in a wide range of applications.

  Here, such methods for detecting and tracking pedestrians can be broadly divided into image recognition-based technology and distance sensor-based technology.

  Here, the distance sensor-based technology generally has an advantage that measurement can be performed relatively accurately in a congested environment and can be stably measured even in an environment exposed to direct sunlight. In such a distance sensor-based technique, as a technique for tracking the position of a person moving in a predetermined space, an example of a measuring device that detects and tracks a person based on distance sensor data is available. For example, it is disclosed in Patent Document 1.

  In Patent Document 1, the measurement device includes a computer, and a plurality of laser range finders are connected to the computer. A plurality of laser range finders are arranged in an environment, and the computer tracks a person based on the detection results of the laser range finders. For example, a computer estimates the position and moving speed of a person using a particle filter, and uses a human shape model that combines a torso and both arms with three circles to determine the direction of the person's body and the movement of the arm. presume. Similar technical contents are also disclosed in Non-Patent Document 1.

  Also, when detecting and tracking humans with a laser range finder, each pole placed at one position has three levels of height, head height, upper body height, and leg height. A technique for installing a laser range finder corresponding to is disclosed in Non-Patent Document 2.

  Further, Non-Patent Document 3 discloses a technique that uses a Kalman filter to track a human leg and discriminate between a human pattern and a non-human pattern.

JP 2009-168578 A

D. F. Glas, T.W. Miyashita, H .; Ishiguro, and N.K. Hagita, “Laser-based tracking of human position and orientation using parametric shape modeling”, Advanced Robotics, vol. 23, no. 4, pp. 405-428, 2009 O. M.M. Mozos, R.A. Kurazume, and T.K. Hasegawa, “Multi-Part people detection using 2D range data”, Int. Journal of Social Robots, vol. 2, no. 1, pp. 31-40, 2010 K. O. Arras, S.M. Grzonka, M.M. Luber, and W.L. Burgard, “Efficient people tracking in laser range data using a multi-hypothesis leg-tracker with adaptive occlusion probes,” in Proc. of IEEE Int. Conf. on Robotics and Automation, 2008, pp. 1710-1715

  However, the technique disclosed in Patent Document 1 is a technique for estimating the position, moving speed, body orientation, and arm position of a subject by a distance measurement sensor, and is detected based on the data of the distance sensor. It is not intended to identify the subject that was made.

  Although there is a technique such as Non-Patent Document 3, there is a plurality of moving objects at the same time, and in an environment where humans and non-human objects exist in the observation area, the target is accurately tracked. However, it is not always easy to identify the target.

  The present invention has been made to solve the above-described problems, and its purpose is to accurately identify an object while tracking a moving object in an observation region using a distance measurement sensor. It is to provide a measurement system and a measurement method that can be classified.

  According to one aspect of the present invention, a measurement system for tracking the movement of an object in an observation region and classifying the object, and measuring a plurality of distances arranged so as to be able to measure a horizontal distance to the object Based on the measurement results, the acquisition means for acquiring the measurement results obtained by measuring the object by the plurality of distance measurement means, the tracking means for estimating the position and moving speed of the object from the measurement results acquired by the acquisition means, A feature extraction unit that calculates a feature vector that expresses a feature of a horizontal two-dimensional cross-sectional shape of the target, and which class of each class of a predetermined class the target belongs to based on the feature vector Discriminating means for discriminating, and the discriminating means calculates the probability that the target belongs to each class of a predetermined class as a prior probability based on the feature vector. Based on the movement speed of the target and the distance between the targets, it is determined whether the target is in a synchronized state moving in synchronization with other targets, and the prior probability and the target in the synchronized state correspond to the class Synchronization state detecting means for discriminating a class to which the object belongs based on the likelihood of performing.

  The measurement system according to claim 1, wherein the feature vector preferably represents a size of the horizontal two-dimensional cross-sectional shape of the target and an aspect ratio of the horizontal two-dimensional cross-sectional shape of the target.

  Preferably, the synchronization state detecting means is based on an angle formed by a speed vector of the target pair and an angle formed by a speed vector of one target of the target pairs and an interval vector between the targets of the target pair. , It is determined whether the target pair is in a synchronized state.

  Preferably, the feature extraction means includes means for interpolating data at a predetermined interval into a plurality of measurement results at the same time acquired by the acquisition means.

  According to another aspect of the present invention, there is provided a measurement method for tracking the movement of an object in an observation region and classifying the object, and measuring a plurality of distances arranged so as to be able to measure a horizontal distance to the object A step of acquiring a measurement result obtained by measuring a target by means, a step of estimating a position and a moving speed of the target from the acquired measurement result, and a feature of a horizontal two-dimensional cross-sectional shape of the target based on the measurement result A step of calculating a feature vector to be expressed; and a step of determining which class of each class of a predetermined class the object belongs to based on the feature vector, wherein the step of determining includes: Based on the step of calculating the probability that the target belongs to each class of a predetermined classification as a prior probability, based on the moving speed of the target and the distance between the targets Determine whether the target is in a synchronized state moving in synchronization with other targets, and determine the class to which the target belongs based on the prior probability and the likelihood that the synchronized target corresponds to the class Steps.

  According to the present invention, when a target moving within the observation region is tracked by a plurality of distance measuring means and it is determined which class of the class the target belongs to the feature vector reflecting the shape of the target. Since the determination is based on this, it is robust with respect to occlusion and can be determined independently with respect to the direction of the object.

  In addition, according to the present invention, it is possible to improve the accuracy of the determination by performing the determination based on the posterior probability in consideration of the likelihood based on the synchronization state of the movement of the two objects, with the determination based on the feature vector as the prior probability. Become.

It is a figure for demonstrating the structure of the measurement system 1000 of Embodiment 1. FIG. It is a figure which shows the level of the detection height of a laser range finder, and a detection target. FIG. 3 is a block diagram for explaining a hardware configuration of a measurement arithmetic device 100. It is a functional block diagram which shows the function implement | achieved when the CPU56 mentioned above performs software in the measurement calculating apparatus 100 of this Embodiment. It is a conceptual diagram which shows the case where a several person exists in an observation area | region and an observation is made with one laser range finder. It is a figure which shows the relationship between the number of the objects tracked simultaneously, and the time interval when the observation was made. It is a conceptual diagram which shows a mode that the person H and the baby cart B are measured by the four laser range finders 10.1 to 10.4. It is a conceptual diagram which shows the feature vector about the object X (person H) and the object Y (baby cart B). It is a figure which shows the normal distribution of each class H, W, S, and B about a discriminant function value. It is a figure which models and shows the object which performs specification of a synchronous state. For two moving objects within a range of distances d _0, it is a conceptual diagram illustrating the distance measurement data (edge information). 22 is a flowchart for explaining processing executed by a tracking module unit 5610 and an identification processing module 5620. 22 is a flowchart for explaining label assignment processing to each target executed by the identification processing module 5620. It is a figure which shows one scene when tracking is implemented. It is a figure which shows the result of the allocation of a label. It is a figure which shows the accuracy of discrimination | determination of each pair about the discriminated target pair.

  Hereinafter, a measurement system according to an embodiment of the present invention will be described with reference to the drawings. In the following embodiments, components and processing steps given the same reference numerals are the same or equivalent, and the description thereof will not be repeated unless necessary.

(Embodiment 1)
FIG. 1 is a diagram for explaining a configuration of a measurement system 1000 according to the first embodiment.

  For example, FIG. 1 shows a state in which four distance measurement sensors 10.1 to 10.4 are arranged in a partial area of a shopping center. In the measurement system 1000, a measurement arithmetic device 100 is also arranged for executing target tracking and specific processing. Here, the distance measuring sensor may be any measuring device that measures the distance between two points in a non-contact manner, but in the following, each of the distance measuring sensors 10.1 to 10.4 is a laser range finder. Will be described.

  In FIG. 1, by setting a predetermined observation area as a shopping center, a baby cart (hereinafter referred to as symbol B), a shopping cart (hereinafter referred to as symbol B), a shopping cart (hereinafter referred to as symbol B) are used as auxiliary tools for humans (hereinafter referred to as symbol H) during movement. Assume a wheelchair (denoted by symbol S) and a wheelchair (hereinafter denoted by symbol W).

  In FIG. 1, a state where the human H is pushing the baby cart B, a state where the human H is pushing the shopping cart S, and a state where the human H is walking side by side with other humans H are illustrated.

  Moreover, the measurement arithmetic device 100 is not particularly limited, but can be realized by a general-purpose computer, and is connected to the laser range finders 10.1 to 10.4 via an interface of a predetermined standard. In the example of FIG. 1, all the laser range finders 10.1 to 10.4 are connected to one measurement arithmetic device 100, but two or more computers connected via a network such as a LAN. You may make it share processing. In that case, for example, any one computer may function as a central computer.

  Further, in the example of FIG. 1, four laser range finders 10.1 to 10.4 are provided to track an object in a predetermined observation area, but this is merely an example. Therefore, the number of laser range finders is appropriately changed according to the size, shape, and position and number of obstacles. The laser range finders 10.1 to 10.4 are arranged at positions where a moving object in a predetermined observation area in the building can be detected, and each of the laser range finders 10.1 to 10.4 is at least two or more. The laser range finders 10.1 to 10.4 are arranged so as to overlap.

  The laser range finder 10.1-10.4 measures the distance for the said object from the time it irradiates with a laser, is reflected on an object, and returns. For example, a laser beam emitted from a transmitter (not shown) is reflected by a rotating mirror (not shown), and the front is scanned in a fan shape by a certain angle (for example, 0.5 degrees).

  Here, the measurement range of the laser range finder 10.1 to 10.4 is indicated by a semicircular shape (fan shape) with a radius R. That is, the laser range finder 10.1 to 10.4 can measure a predetermined angle with respect to the left and right, for example, a direction of 90 ° within a predetermined distance (R) when the front direction is the center.

  The laser used is a class 1 laser in Japanese Industrial Standard JIS C 6802 “Safety Standards for Laser Products”, and is at a safe level that does not affect human eyes.

[Hardware block]
FIG. 2 is a diagram showing the detection height level of the laser range finder and the detection target.

  As illustrated in FIG. 2, the laser range finder 10.1 to 10.4 is 52 cm from the floor in order to identify the baby cart B, shopping cart S, and wheelchair W, which are aids when a person moves. The distance is measured at the level L2 of the height. Further, the laser range finders 10.1 to 10.4 perform distance measurement at a level L1 having a height of 87 cm from the floor in order to track a moving object (pedestrian). This height L1 is a level close to the upper end of any assisting device, and is a height set so as to be able to detect the torso of the target pedestrian. Calculated from average height. Therefore, the distance measurement height L1 of the laser range finder 10.1 to 10.4 may be appropriately changed according to the place (region or country) where the measurement system 1000 is provided.

  Here, although illustration is omitted, the measurement result of the measurement arithmetic device 100 is given to, for example, a robot (communication robot) that communicates with the subject, and the communication robot facilitates communication with the subject using this measurement result. It is possible to adopt a configuration such as For example, the communication robot confirms the position of the subject, approaches the subject, and executes a communication behavior by at least one of an action (behavior) such as gesture or hand gesture and voice. In addition, when the subject moves, the communication robot can run parallel to the subject at that speed and communicate while moving.

  FIG. 3 is a block diagram for explaining a hardware configuration of the measurement arithmetic device 100.

  As shown in FIG. 3, the measurement arithmetic device 100 includes a drive device 52 that can read data recorded on the external recording medium 64, and a central processing unit (CPU: Central Processing Unit) 56 connected to a bus 66. A ROM (Read Only Memory) 58, a RAM (Random Access Memory) 60, a non-volatile storage device 54, and a data input interface for fetching distance measurement data from the laser range finders 10.1 to 10.4 (Hereinafter, data input I / F) 68.

  As the external recording medium 64, for example, an optical disk such as a CD-ROM or a DVD-ROM or a memory card can be used. However, the target recording medium is not limited to this as long as the device that realizes the function of the recording medium drive 52 is a device that can read data stored in a nonvolatile recording medium such as an optical disk or a flash memory. In addition, a device that realizes the function of the nonvolatile storage device 54 may be a magnetic storage device such as a hard disk or a flash memory as long as it can store data in a nonvolatile manner and can be accessed randomly. A solid state drive (SSD) that uses a nonvolatile semiconductor memory such as a storage device can also be used.

  The main part of such a measurement arithmetic device 100 is realized by computer hardware and software executed by the CPU 56. In general, such software is recorded by mask ROM, programmable ROM, or the like at the time of measurement and calculation device 100 manufacture. The software may be read into RAM 60 at the time of execution, or read from recording medium 64 by drive device 52. It may be configured to be temporarily stored in the nonvolatile storage device 54 and read to the RAM 60 at the time of execution. Alternatively, when the device is connected to a network, the server is temporarily copied from the server on the network to the nonvolatile storage device 54, read from the nonvolatile storage device 54 to the RAM 60, and executed by the CPU 56. There may be.

  The computer hardware itself and its operating principle shown in FIG. 3 are general. Accordingly, one of the most essential parts of the present invention is software stored in a recording medium such as the nonvolatile storage device 54.

[System functional blocks]
As will be described below, in the measurement arithmetic device 100 according to the present embodiment, the computer uses the particle filter based on the outputs (distance data) from the laser range finders 10.1 to 10.4, and the target position and Estimate speed.

  FIG. 4 is a functional block diagram showing functions realized by the above-described CPU 56 executing software in the measurement arithmetic device 100 according to the present embodiment.

  Note that the functional blocks realized by the CPU 56 among the functional blocks shown in FIG. 4 are not limited to software processing, and a part or all of them may be realized by hardware.

  Referring to FIG. 4, ranging signals from laser range finders 10.1 to 10.4 are controlled by data capture processing unit 5602 and input as digital data, and captured data recording processing unit 5604, for example, is nonvolatile. Stored in a storage device such as the sexual storage device 54.

  As will be described in detail later, the measurement arithmetic device 100 further executes a tracking module unit 5610 for performing tracking processing of the position of the target using the particle filter method, and processing for identifying and specifying the target. Identification processing module 5620.

  The tracking module unit 5610 predicts the state of the target time t based on the behavior model of the target, the state of the previous time (t−1), and the weight of the particle, and stores the distance measurement data. Read from the device 54, calculate the likelihood of the distance measurement data and the state at time t for each particle, assign the weight of each particle based on the likelihood, and objects represented by all particles The state evaluation calculation unit 5616 for estimating the current target state from the state and the weight of the current state, and the resampling calculation unit 5618 for executing the resampling of the particle based on the estimated state and the weight. The operation of the tracking module unit 5610 will be described in detail later.

  The identification processing module 5620 extracts a feature vector, which will be described later, for each target from the distance measurement data, and calculates a prior probability for classifying the target based on the extracted feature vector. Based on the probability calculation unit 5624, the distance between the targets and the velocity vector of the target, the CQ detection unit 5626 that detects the synchronization state (CQ: Coherence Quality), and the target according to the detected synchronization state A label assignment processing unit 5628 for assigning a label belonging to which classification.

(Detection and tracking of target position)
Below, the detection of the position of the object containing the person mentioned above and its tracking process are demonstrated.

  In this algorithm, tracking is performed by associating one set of particles called a particle filter for each target.

  In such tracking, it is necessary to detect events of entry into the observation area of the object, continuous tracking of the object, and exit from the observation area of the object.

  The intrusion into the target observation area is detected by comparing the background model based on the distance measurement data up to the previous time and the current sensor data. The background model is also updated over time. In this way, foreground data (current target data) is specified, continuous segments, that is, patterns serving as target candidates are extracted from such foreground data, and such candidates are registered.

  At the tracking stage, each such candidate is assigned a particle filter.

The state s _X [t] expressed as follows is estimated for the target candidate X by the particle filter.

Then, after the target candidate X as described above continues to exist for a predetermined period, for example, 0.4 seconds, this candidate is registered as an effective target.

  Object tracking is performed by periodically updating the particle filter.

  In the tracking process using such a particle filter, the following steps are repeated.

i) Update step The state of each particle is updated according to the behavior model. The behavior model reflects the dynamics to be tracked.

  Here, the human walking motion cannot be approximated by a simple linear function or a Brownian motion representing a random motion because the human stops or suddenly changes the walking direction. Therefore, as the motion model, when the particle state is the position and velocity as described above, a model in which Gaussian noise is added to the velocity component and the position moves according to such velocity is adopted. . That is, for example, the following model is employed.

Here, w _x and w _y are Gaussian noises having an average of 0 and standard deviations σ _x and σ _y .

ii) Weight assignment step Assign a weight representing a relative likelihood to each particle according to a likelihood model. Here, the likelihood is expressed as follows.

The likelihood model will be described in detail later.

iii) State evaluation step The state of each target is calculated as an average of the state of the particles.

iv) Resampling step In order to more accurately reflect the state of the system, particles are restored and extracted with a probability proportional to their weights, and a new set of particles at time t is obtained. As a resampling method, a SIR (Sampling Importance Resampling) method can be used, but it is not limited to this method.
(Likelihood model)
Below, the likelihood model mentioned above is demonstrated easily. More detailed explanation is disclosed in Non-Patent Document 1, for example.

  The purpose of the likelihood model is to provide a model that approximates the likelihood described above. Here, in calculating the likelihood, in this embodiment, measurement data obtained by a plurality of laser range finders is obtained as a result of measurement. Therefore, the robust likelihood model must be capable of giving a robust likelihood evaluation to noise, occlusion, and the like included in the sensor measurement values.

  From the laser range finder, the following information useful for evaluating the position of the object can be obtained.

    i) Occupancy information: Information indicating whether a certain point in the observation area is occupied or not.

    ii) Edge information: Information on (a part of) the contour line corresponding to the detected target edge.

  FIG. 5 is a conceptual diagram showing a case where a plurality of people exist in the observation area and observation is performed by one laser range finder.

  FIG. 5A shows a state of scanning by one laser range finder. FIG. 5B shows occupancy information. Here, the area shown in black is an area that is a shadow area (occupied area) from the laser range finder due to the presence of a person. On the other hand, the gray region is a region that is a visible region (non-occupied region) from the laser range finder.

  FIG. 5C shows edge information. That is, it is the position information of the point that is the edge of the occupied area from the measurement of the laser range finder when scanning at a predetermined scan interval.

  Occurrence likelihood is obtained by dividing the region into “visible region”, “shadow region”, and “unobservable region” by measurement for each laser range finder. Here, the “unobservable region” is a region outside the background model for the laser range finder.

  Here, the shadow area always exists behind the edge.

  In order to calculate the state described above, the following equation is used.

Here, N _S is the number of laser range finder, the constant p _col, the two particle filters, is a constant to be subtracted in order not to track the same target.

That is, basically, the likelihood when in the shadow region can be obtained by integrating the sum of the constant p _s and the likelihood p _e with respect to the laser range finder. Here, the likelihood p _e is a normal distribution centered on the position of a predetermined distance behind the observed edge. Here, 50 cm is adopted as the predetermined distance.

The constant p _o is a constant that represents the likelihood when it is in the visible region. Here, although the likelihood should be 0 in the visible region, the boundary between the visible region and the shadow region is actually completely binary due to various errors. Since it cannot be separated, it is assumed that it has a certain value in order to give error tolerance to the operation. As a result, even if the particles are present outside the shadow area, they do not disappear immediately.

  As for the exit of the target from the observation area, i) when the particle associated with the target has spread beyond a predetermined value, or ii) the average likelihood for the particle filter is a predetermined value. If so, the particle filter is no longer tracking the target and the associated target is removed from the registration.

  Further, as will be described later, a tracking target is a target to be identified and classified if it exists in the observation region for a predetermined time, for example, 2 seconds or more.

(Modeling and feature extraction)
Hereinafter, the identification and classification of the object as described above will be described.

  In order to identify such an object, a feature vector as described below is extracted from the distance measurement data of the laser range finder.

  Here, in the environment such as the shopping center as described above, the object to be identified is similar in shape to the object, but the trajectory of the object can take various shapes. Moreover, in general, the number of laser range finders is often the same as the number of objects to be detected simultaneously.

  FIG. 6 is a diagram showing the relationship between the number of objects tracked simultaneously by the tracking method as described above and the time interval at which the observation was made.

  In an actual experiment described later, at least a plurality of objects are simultaneously tracked within a predetermined observation area, and in the case of many, about 6 to 7 objects are tracked simultaneously.

  More specifically, 25%, 29% and 22% of the observation time, respectively, are one, two and three periods respectively tracked simultaneously. In addition, 24% of the observation time is a period of four or more objects tracked simultaneously.

  Therefore, it is desirable that the feature vector for such identification has the following characteristics.

  First, feature vectors need to be robust to occlusion and must be independent of the distance to the object.

  Second, the trajectory of the object can change significantly, but the feature vector must be independent of the orientation of the object.

  When the distance measurement data of the laser range finder is converted into the xy plane (horizontal plane), if the distance resolution is set to 100 mm or less, the measurement sample (edge information) obtained from the distance measurement data is obtained from the segment ( Divided area). Further, as will be described below, since a feature vector expressing the feature of the target shape is adopted, preprocessing is performed so that the raw measurement sample has a more uniform distribution. That is, as the laser range finder scans in a fan shape, raw measurement samples at the same time are present at intervals of, for example, 10 mm to 100 mm mainly depending on the distance from the laser range finder. Even in such a case, the measurement sample after the pretreatment is generated by linearly interpolating the raw measurement sample so that one exists at a predetermined interval, for example, every 10 mm.

  It is assumed that the feature vector is expressed as follows by using a parameter expressing the shape of the target X using the measurement sample after such preprocessing.

Therefore, the first component f ¹ _X has a larger value in the wheelchair W and the baby cart B, a smaller value for the person H, and a smallest value for the shopping cart S. This is because, in the shopping cart S, the cross-sectional area at the detection height level L2 used for identification is the smallest as shown in FIG. That is, the first component is a component that represents the size of the horizontal two-dimensional cross-sectional shape of the target.

The second component f ² _X has a relatively broad distribution for the person H and the baby cart B. This is because the value of person H varies depending on clothes, age, gender, etc., and in baby cart B, the value varies because a bag is put on it or a shopping basket is attached. become. Compared to this, in the wheelchair W and the shopping card S, the variation of the second component is small. The second component corresponds to the aspect ratio of the target horizontal two-dimensional cross-sectional shape.

  Such feature vectors satisfy all of the general requirements for feature vectors as described above.

  That is, as described above, since the measurement sample is first interpolated between the raw measurement samples, even if a part of the object is shaded by occlusion, Even if there is a region where the measurement sample does not exist, the influence on the feature vector is reduced.

  Furthermore, since the measurement sample is uniformly distributed by performing the pre-processing as described above, the feature vector is independent of the distance from the laser range finder, and further, the first component and the second component Is independent of the orientation of the object.

  Note that the feature vector does not depend on the orientation of the object, and if the feature of the object shape is expressed by a parameter, it corresponds to the size or aspect ratio in the cross section of the specific horizontal plane as described above. It is not limited to parameters. For example, when the number of horizontal planes for measuring the distance of the laser range finder is increased, it is also possible to take into account such other horizontal plane shapes, or changes in the shape in the vertical direction. . Therefore, the feature vector is defined as a sequence of a plurality of parameters expressing the shape of the object that does not depend on the direction of the object.

  FIG. 7 is a conceptual diagram showing how the person H and the baby cart B are measured by the four laser range finders 10.1 to 10.4 in the example shown in FIG.

  FIG. 8 is a conceptual diagram showing feature vectors for the target X (person H) and the target Y (baby cart B) in the case shown in FIG.

  It is assumed that distance measurement data as shown by black dots in FIG. 8 is obtained by distance measurement as shown in FIG.

  In FIG. 8, since the raw distance measurement data is shown, for example, for the baby cart B, there is an area where the measurement sample is missing. However, also in such a region, interpolated data is generated in the measurement sample after the preprocessing as described above.

The target X (person H) has N ^r _X samples, and the target Y (baby cart B) has N ^r _Y measurement samples, and accordingly, the first component of the feature vector Are calculated respectively.

Furthermore, eigenvalues v ¹ _X and v ⁰ _X are obtained for the object _X , and eigenvalues v ¹ _Y and v ⁰ _Y are obtained for the object Y. Based on these, the second component of the feature vector is Calculated.
(A priori probability for classification based on shape)
First, Fisher's linear discriminant (FLD) is used as a first stage for performing classification of objects based on features in shape.

  Fischer's linear discrimination itself is well known and disclosed in, for example, the following documents.

Reference 1: RO Duda, PE Hart, and DG Stork, Pattern Classification, 2nd ed. Newyork Wiley, 2001
Fisher's linear discriminant function g for the feature vector f _X is expressed in the following format.

Here, w is a weight vector, and b is a weight threshold value.

  For example, classes included in the classification are expressed by class ei and class ej. If the following relationship holds, the class ei is assigned to the feature vector.

In order to obtain prior probabilities for classification, a normal distribution is fitted to the linear discriminant function values of each class obtained in Fisher's linear discriminant function training.

  FIG. 9 is a diagram illustrating normal distributions of the classes H, W, S, and B with respect to the discriminant function values.

  In FIG. 9, for example, it is assumed that an object having a discriminant function value as indicated by a one-dot chain line is observed.

  In this case, from such a graph, the prior probability calculation unit 5624, as the prior probability of the target classification, the probability P (B) that the target is a baby cart B, the probability P (H) that the target is a person H, the target The probability P (W) that is a wheelchair W can be obtained.

  In the above description, Fischer's linear discriminant function has been described as an example. However, the discriminant function is not limited to this, and can be obtained from the feature vector as a probability to which class the corresponding target belongs. Any other discriminant function can be used.

(Preprocessing and synchronization status detection)
Hereinafter, a method will be described in which the CQ detection unit 5626 checks the relationship between the velocity vector and the interval vector for two objects in order to identify the objects that move together and detect the synchronization state.

  Here, the synchronized state includes horizontal parallel movement (side by side) and vertical movement (tandem).

  FIG. 10 is a diagram showing a model of a target on which the synchronization state is specified.

  As shown in FIG. 10, it is assumed that three objects X, Y, and Z exist. It is assumed that the target X moves following the target Y, and the target Z moves adjacent to the target X.

  At this time, the velocity vector and the interval vector are defined as follows.

First, a procedure for acquiring the velocity vector and the interval vector of each target from the actually measured distance measurement data will be described.

When the center position of the target X obtained by the tracking process sequence data _{^{C 0 X [t] = {}} c 0 X [t]} from the time series of the smoothing processing position data _{C X [t] = {c} X [T]} is calculated by the following procedure.

  That is, in order to reduce the influence of noise, the following convolution operation using a low-pass filter is executed.

Further, the following averaging is performed at a predetermined time interval, for example, every Δt = 100 msec, and a velocity vector is calculated every two time intervals (2Δt) in order to further improve the smoothing effect.

On the other hand, the interval vector is calculated as the position at each time of the position after smoothing, for example, the difference between the position c _X [t] and the position c _Y [t].

  The magnitude of the speed calculated in this way was 950 mm / sec on average and 340 mm / sec dispersion in the test in the real environment described later.

  As described above, when the velocity vector and the interval vector are calculated for each predetermined time interval, the synchronization state is determined based on this as follows.

The following determination is made for a pair of objects in which two objects exist within a predetermined distance d ₀ and the magnitudes of their velocity vectors are greater than or equal to a predetermined magnitude V ₀ . Executed.

  The model shown in FIG. 10 means that the following conditions are satisfied.

FIG. 11 is a conceptual diagram showing distance measurement data (edge information) for two objects moving within such a range within the distance d ₀ in the example shown in FIG.

Here, as an example, it is assumed that the distance d ₀ is 1 m and the size V ₀ is 75 mm / sec.

  In FIG. 10, since the target X and the target Y are moving in the same direction, the relationship of the following formula (3) is established.

With respect to two objects existing within the range of the predetermined distance described above, if a relationship such as the equation (3) is established between the two objects, it can be said that the objects are moving together.

  Since the combination of two objects that make such a move is a horizontal parallel move (eg, H-H pair) or a column move (eg, H-S pair or H-B pair), it further adds 2 By examining the relative position between the two objects, it is possible to identify which movement mode it is.

For example, since the target X and the target Y in FIG. 10 are in the column movement mode in which the target X is moving behind the target Y in synchronization, the interval vector d _XY is the velocity vector V _X for the target _X. Since they are oriented in the same direction as V _Y , the following equation (4) is established.

In this way, the column movement mode in which the object X is moving together behind the other object Y is particularly referred to as a rear column movement mode (represented by the symbol R).

On the other hand, when the same value as that of the equation (4) is calculated with respect to the target Y, the interval vector d _YX is directed in the opposite direction to the velocity vectors V _X and V _Y, and the following equation (4) ′ is established. .

In this way, the column movement mode in which the object Y is moving together on the front side of the other object X is particularly referred to as a front column movement mode (represented by the symbol F). However, the absolute value is also used for classification.

In FIG. 10, since the object X and the object Z are moving side by side in parallel, the interval vector d _XZ is oriented in the vertical direction with respect to the velocity vectors V _X and V _Z. Holds.

Therefore, based on the distance between the objects, the movement speed of the objects, and the relations of the above formulas (3) to (5), the two objects that move together are in the horizontal parallel movement mode, and In the embodiment, it is possible to specify whether the pair corresponds to the H-H pair, the front column movement mode or the rear column movement mode, and the H-S pair or the H-B pair.

(Conditions for determining synchronization status)
In reality, since the object can be accelerated, decelerated, or the interval can be changed independently, the equations (3) to (5) can be strictly 0 or 1. Absent.

  Therefore, if both of the expressions (3) and (4) (or the absolute value of the expression (4) ′) are, for example, 0.8 or more, the CQ detection unit 5626 has two such objects as It is determined that the movement is in the column movement mode and corresponds to the H-S pair or the H-B pair. Whether formula (4) is met or formula (4) ′ is met determines whether it is the rear column movement mode or the front column movement mode.

  On the other hand, since the value of Equation (5) is determined by the relative relationship of movement between humans, it does not take a clear range of values as in the column movement mode. Therefore, in the following description, the CQ detection unit 5626 has the value of the expression (3) of 0.8 or more, but the expression (4) (or the absolute value of the expression (4) ′) is less than 0.8. The target pair is in the horizontal parallel movement mode, and in the present embodiment, it is determined that it is an HH pair.

  If none of the above cases apply, the CQ detection unit 5626 determines that the target is moving independently.

(Distinction based on a combination of prior probability and synchronization status (movement mode))
The class of the target classification is represented by the symbol e, and the class e means any of E = {H, W, S, B}.

  Also, the synchronization state is represented by the symbol cq, and the synchronization state cq means any one of {D, S, F, R}. Here, symbol D is a state where the object is moving independently, symbol S is a state of horizontal parallel movement mode, symbol F is a state of forward tandem movement mode, and symbol R is Indicates the state of backward column movement mode.

  According to the Bayes rule, the posterior probability p (e | cq) that the object belongs to the class e when in the synchronization state cq is expressed as the following equation (6). Therefore, the label allocation processing unit 5628 can determine to which class the target belongs by using Expression (7).

Here, the prior probability P (e) can be derived from the above-described feature vector, and the likelihood P (cq | e) is experimentally determined in advance based on actual measurement results of a plurality of targets. Is possible.

  FIG. 12 is a flowchart for explaining processing executed by the tracking module unit 5610 and the identification processing module 5620 described above.

  Referring to FIG. 12, when the process is started, tracking module unit 5610 initializes particles (S100). Here, the tracking module unit 5610 distributes all (for example, but not limited to 400) particles equally in the state space. However, the state space corresponds to a region that can be detected by the laser range finder 10.1 to 10.4.

  Subsequently, the tracking module unit 5610 initializes the variable m (m = 1) (S102). However, the variable m is a variable for identifying particles arranged in the state space.

Further, the tracking module unit 5610 includes the state s ^m _X [t−1] of the target X at the time t−1 represented by the ^mth particle, the particle weight π ^m _X [t−1], and the behavior model. Then, the next state s ^m _X [t] is obtained.

  However, as described above, m is a subscript for individually identifying particles.

  As the motion model, a model in which Gaussian noise is combined with linear movement is used. Specific examples thereof are as described above.

Subsequently, the tracking module unit 5610 increments the variable m (S106). Then, it is determined whether or not the variable m exceeds the maximum value (current total number of particles) (S108). That is, it is determined whether or not the next state s ^m _X [t] has been obtained for all particles.

If “NO” in the step S108, that is, if the variable m is equal to or less than the maximum value, the process returns to the step S104 to obtain the next state s ^m _X [t] for the next particle m (← m + 1).

On the other hand, if “YES” in the step S081, that is, if the variable m exceeds the maximum value, it is determined that the next state s ^m _X [t] for all the particles m is obtained, and the tracking module unit 5610 Reads the occupancy information (referred to as sensor data) based on the distance measurement data from the storage device 54 in step S110.

Further, the tracking module unit 5610 calculates the likelihood between the sensor data r [t] and the state at time t for each particle m. That is, the similarity between the state s ^m _X [t] estimated in step S104 and the actual state based on the sensor data is calculated (S112), and the weight π ^m _X [t] is obtained (S114).

  The likelihood is as described above.

In step S114, the tracking module unit 5610 sets the weight π ^m _X [t] of each particle to the ^{height of} the likelihood p (r [t] | s ^m _X [t]) obtained in step S112. Calculate proportionally. For example, the weight π ^m _X [t] may be determined so as to change linearly in order from the particle having the highest likelihood p (r [t] | s ^m _X [t]), or a constant threshold value. Each time, the weight π ^m _X [t] may be determined to change stepwise.

Subsequently, the tracking module unit 5610 estimates the current target state from the human state s ^m _X [t] and the weight π ^m _X [t] represented by all the particles. Although not particularly limited, for example, the current state of the object is estimated as an average using the state s ^m _X [t] of the object represented by the particle m having a weight π ^m _X [t] of 2σπ or more. However, σπ is a standard deviation for the weight π ^m _X [t] of all particles m.

  As described above, when the estimation of the state of the target up to the present is completed, the identification processing module 5620 classifies the target and assigns a label to each target (S118). This classification process will be described later.

  Then, the tracking module unit 5610 determines a position (region) where particles are to be focused on and a particle to be erased from the estimation result of the current target state in step S1116, and disperses and erases the particles. Then, the process is restored and extracted, and the process returns to step S102. That is, the resampling process is executed.

  FIG. 13 is a flowchart for explaining the process of assigning labels to each target executed by the identification processing module 5620 in step S118.

  Referring to FIG. 13, first, the feature extraction calculation unit 5622 calculates the size of the target horizontal two-dimensional cross-sectional shape and the aspect ratio of the target horizontal two-dimensional cross-sectional shape based on the measured edge information. A feature vector to be represented is extracted (S200).

  Next, based on the preset linear discriminant function, the prior probability calculation unit 5624 calculates a prior probability representing the probability that the target belongs to each class of the classification based on the discriminant function value obtained from the feature vector (S202). ).

  Next, the CQ detection unit 5626 determines a pair of targets that are within a predetermined distance range and move at a speed equal to or greater than a predetermined size among a plurality of targets based on the target velocity vector and the interval vector. The synchronization state is classified (S204).

  Next, the label allocation processing unit 5628 obtains the posterior probability based on the prior probability obtained in step 202 and the likelihood of the synchronization state for the target class obtained experimentally in advance. A label of a class having a high posterior probability is assigned to the target (S206).

In this way, the object can be tracked by applying the particle filter, and the object can be identified from the feature vector. However, since one particle filter is required for one target, the same number of particle filters are required when tracking a plurality of targets simultaneously.
(Experimental result)
Hereinafter, the results of checking the accuracy and consistency of the tracking and identification process described above will be described.

  The experiment was conducted between 15:00 and 16:00 on weekdays in the entrance hall of the shopping center. The entrance hall is a place where many customers who carry bags or push carts come and go.

  The area (observation area) where the tracking is performed is a rectangular area of 7.5 m × 8 m. Four poles were set up at the four corners of this area, and as shown in FIG. 2, a laser range finder was fixed at positions 52 cm and 87 cm high from the floor in each pole. In particular, the height of 52 cm from the floor is set as a height at which a cross-sectional shape that can easily distinguish the object can be measured.

  The sampling rate of the laser range finder is 40 Hz, and the angular resolution is 0.25 degrees. Further, the range from 30 mm to 10 m from the laser range finder is the range in which distance measurement is possible. For 35 minutes, video images were taken at the same time as the distance measurement data from the laser range finder. Video shooting is performed in order to verify the validity of label assignment by the method of the present embodiment.

  A total of 504 objects were detected, and according to the video image, 391 of them were human (H), 34 were wheelchairs (W), 48 were shopping carts (S), and 31 were Is a baby cart (B). Of the 391 pedestrians, there were 44 groups of 2, 3 or 4 people.

  FIG. 14 is a diagram illustrating one scene when tracking is performed.

  In this figure, 11 objects are tracked as moving. Of these, three groups surrounded by ellipses are moving together with two people. After that, there are two pedestrians, one shopping cart, and one pedestrian moving while pushing a baby cart.

(Distinction result before considering synchronization status)
In the following, a description will be given first of the result of discriminating an object based only on the feature vector without considering the synchronization state.

  Here, it is assumed that a target sample for training for discrimination is different from a target sample in a test operation for actually evaluating discrimination performance.

  FIG. 15 is a diagram illustrating a result of label allocation, and FIG. 15A is a diagram illustrating a result when label allocation is performed based only on feature vectors.

  That is, in this table, the vertical direction indicates the label that should be actually assigned to the object confirmed from the video image, and the horizontal direction indicates the label assigned in the test operation.

  Therefore, when the determination in the test operation has 100% accuracy, an element other than 0 appears only in the diagonal direction.

  However, as is clear from FIG. 15A, although it is possible to determine to some extent even by determination using only the feature vector, in reality there are a considerable proportion of cases where labels are erroneously assigned.

  FIG. 16 is a diagram showing the accuracy of discrimination of each pair for the discriminated target pair.

According to FIG. 16, 35 pairs out of 44 pairs are correctly identified as the H-H pairs.
Of the 44 pairs, 43 pairs are correctly identified among the HS pairs. Of the 24 pairs of H-B pairs, 23 pairs are correctly identified.

  Therefore, discrimination errors are mainly caused by HH pairs. This is because in the H-H pair, the moving speed is extremely low, or a certain group stops, changes its direction, and merges with another group.

  On the other hand, since the operations of the H-S pair and the H-B pair are more regular, they are correctly discriminated at a higher rate.

(Distinction result after considering the synchronization status)
FIG. 15B is a diagram illustrating a result of label assignment in consideration of both the feature vector and the synchronization state.

  Considering the synchronization state, the discrimination for the person H is improved from 60% to 95%. The success rate for wheelchairs W has also improved from 74% to 81%.

  When only the feature vector is considered, the person H and the wheelchair W are often erroneously recognized as the baby cart B, but it is understood that such erroneous recognition is suppressed by considering the synchronization state. .

  As described above, when the object that moves in the observation region is tracked by a plurality of distance measurement sensors and the target belongs to which class of the classification, as in the measurement arithmetic device 100, the object If a feature vector reflecting the size and shape of the image is introduced, it is robust against occlusion and can be determined independently with respect to the target direction.

  Moreover, it is possible to improve the accuracy of the determination by performing the determination based on the posterior probability in consideration of the likelihood based on the synchronization state of the movements of the two objects using the determination based on the feature vector as the prior probability.

  Embodiment disclosed this time is an illustration of the structure for implementing this invention concretely, Comprising: The technical scope of this invention is not restrict | limited. The technical scope of the present invention is shown not by the description of the embodiment but by the scope of the claims, and includes modifications within the wording and equivalent meanings of the scope of the claims. Is intended.

  10.1 to 10.4 Laser range finder, 52 drive device, 54 nonvolatile storage device, 56 CPU, 58 ROM, 60 RAM, 64 external recording medium, 66 bus, 100 measurement arithmetic device, 5602 data capture processing unit, 5604 Capture data recording processing unit, 5610 tracking module unit, 5612 update calculation unit, 5614 weight allocation calculation unit, 5616 state evaluation calculation unit, 5618 resampling calculation unit, 5620 identification processing module, 5622 feature extraction calculation unit, 5624 prior probability calculation unit , 5626 CQ detection unit, 5628 Label allocation processing unit.

Claims (5)

A measurement system for tracking movement of an object in an observation area and classifying the object,
A plurality of distance measuring means arranged to be able to measure the horizontal distance to the object;
Obtaining means for obtaining a measurement result obtained by measuring the object by the plurality of distance measuring means;
Tracking means for estimating the position and moving speed of the object from the measurement result obtained by the obtaining means;
Feature extraction means for calculating a feature vector that represents a feature of a horizontal two-dimensional cross-sectional shape of the object based on the measurement result;
Determination means for determining, based on the feature vector, which class of each class of the predetermined classification the object belongs to;
The discrimination means includes
A prior probability calculating means for calculating, as a prior probability, a probability that the target belongs to each class of a predetermined classification based on the feature vector;
Based on the movement speed of the object and the distance between the objects, it is determined whether the object is in a synchronized state moving in synchronization with another object, and the prior probability and the object in the synchronized state And a synchronization state detecting means for determining the class to which the object belongs based on the likelihood corresponding to the class.   The measurement system according to claim 1, wherein the feature vector represents a size of a horizontal two-dimensional cross-sectional shape of the target and an aspect ratio of the horizontal two-dimensional cross-sectional shape of the target. The synchronization state detecting means includes
Based on the angle formed by the velocity vector of the target pair and the angle formed by the velocity vector of one target of the target pair and the interval vector between the targets of the target pair, the target pair is The measurement system according to claim 2, wherein it is determined whether the synchronization state is established.   The measurement system according to claim 2, wherein the feature extraction unit includes a unit that interpolates data at a predetermined interval into the plurality of measurement results obtained at the same time acquired by the acquisition unit. A measurement method for tracking movement of an object in an observation area and classifying the object,
Obtaining a measurement result obtained by measuring the object by a plurality of distance measuring means arranged so as to be able to measure a horizontal distance to the object;
Estimating the position and moving speed of the object from the acquired measurement result;
Calculating a feature vector representing a feature of a horizontal two-dimensional cross-sectional shape of the object based on the measurement result;
Determining, based on the feature vector, which class of each class of the predetermined classification the object belongs to,
The step of determining includes
Calculating the probability that the target belongs to each class of a predetermined classification based on the feature vector as a prior probability;
Based on the movement speed of the object and the distance between the objects, it is determined whether the object is in a synchronized state moving in synchronization with another object, and the prior probability and the object in the synchronized state Determining the class to which the object belongs based on the likelihood corresponding to the class.