JP2018147157A - Movable matter quantity-of-state prediction unit and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To make it possible to precisely obtain a movable matter quantity-of-state by integrating results of a plurality of sensors even when the number of a plurality of asynchronous sensors is undefined.SOLUTION: A sensing results acquisition unit 14 receives detection results of detecting a plurality of movable matters from any of a plurality of sensors which are indefinite in number. A movable matter quantity-of-state prediction unit 18 predicts a movable matter quantity-of-state of next time using the quantity-of-state stored in a movable matter quantity-of-state storage unit 16 relative to the respective plurality of movable matters. A movable matter quantity-of-state matching unit 20 matches the movable matter quantity-of-state predicted relative to the respective plurality of movable matters by the movable matter quantity-of-state prediction unit 18 according to time of detection results using a movable matter movable range map 30 each time the detection result is received by the sensing results acquisition unit 14 with the detection results of the respective plurality of movable matters to update the quantity-of-state and the latest observation time of the respective plurality of movable matters stored in the movable matter quantity-of-state storage unit 16.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a moving object state quantity estimation device and a program.

  Conventionally, using multiple camera images, to capture the movement of a pedestrian that is hidden (occluded) from a scene where multiple pedestrians are walking, extracted from multiple camera placement / orientation information and multiple camera images A technique for separating and tracking pedestrians while performing spatiotemporal tracking using the pedestrian's foot area information is known (Non-Patent Document 1).

  In addition, a template position representing a vehicle region is extracted using a plurality of camera images, multi-view processing and time-series processing are performed on the template position information, and association of vehicles detected by the plurality of cameras is performed. A traveling vehicle detection device that can calculate the motion trajectory of a traveling vehicle with high accuracy by performing the method is known (Patent Document 1).

S.M.Khan, et.al., “A multiview approach to tracking people in crowded scenes using a planar homography constraint,” Proceedings of the 9th European conference on Computer Vision, ECCV06, Vol. 4, pp133-146.

JP 2001-357489

  The techniques of Non-Patent Document 1 and Patent Document 1 are based on the premise that video frames can be synchronized between a plurality of cameras (sensors) at regular intervals.

  However, if frame synchronization (synchronization of detection result transmission timing) cannot be achieved between multiple sensors due to communication delays or sensor specifications, frames of a certain time are accumulated on the device that integrates the results, and data interpolation, etc. It is necessary to perform the synchronization process. Further, when the number of sensors is indefinite, it is difficult to perform the synchronization process because the synchronized sensor is indefinite.

  In addition, when the observation amount obtained from the sensing result is clearly wrong as inside the obstacle, a new state amount is generated unless it corresponds to the existing state amount.

  The present invention has been made in view of the above circumstances, and even if the number of asynchronous multiple sensors is indefinite, the results of the multiple sensors can be integrated to accurately determine the state quantity of the moving object. It is an object of the present invention to provide a moving object state quantity estimation device and a program.

  In order to achieve the above object, a moving object state quantity estimation device according to a first invention comprises a moving object state quantity storage means for storing a state quantity and the last observed time of each of a plurality of moving objects; For each of the moving objects, any one of the moving object state quantity predicting means for predicting the state quantity of the moving object at the next time using the state quantity stored in the moving object state quantity storage means, and a plurality of indefinite number of sensors. Sensing result acquisition means for receiving detection results obtained by detecting a plurality of moving objects, and each time the detection result is received by the sensing result acquisition means, the movable object movable range information in which the movable range of the moving object is recorded. Using the state quantity of the moving object predicted for each of the plurality of moving objects by the moving object state quantity predicting unit corresponding to the time of the detection result, and before each of the plurality of moving objects A moving object state quantity matching unit that associates with a detection result and updates a state quantity and the last observed time of each of the plurality of moving objects stored in the moving object state quantity storage unit. It consists of

  According to a second aspect of the present invention, there is provided a program including a moving object state quantity storage unit that stores a state quantity and a last observed time for each of a plurality of moving objects. Detection by detecting a plurality of moving objects from either a moving object state quantity predicting means for predicting a state quantity of the moving object at the next time using a state quantity stored in the quantity storage means, or an indefinite number of sensors. Sensing result acquisition means for receiving a result, and each time the detection result is received by the sensing result acquisition means, the moving object movable range information in which the movable range of the moving object is recorded is used to correspond to the time of the detection result. Correspondence between the state quantity of the moving object predicted for each of the plurality of moving objects by the moving object state quantity predicting means and the detection result of each of the plurality of moving objects Only performed, stored in said moving object state quantity storing means, of each of the plurality of moving object is a program for functioning as a moving object state quantity matching means for updating the state variable and finally the observed time.

  According to the first and second aspects of the invention, the sensing result acquisition means receives detection results obtained by detecting a plurality of moving objects from any of a plurality of indefinite number of sensors. For each of the plurality of moving objects, the moving object state quantity predicting means predicts the state quantity of the moving object at the next time using the state quantity stored in the moving object state quantity storage means.

  Then, every time the detection result is received by the sensing result acquisition means by the moving object state quantity matching means, the moving object movable range information in which the movable range of the moving object is recorded is used to correspond to the time of the detection result. The moving object state quantity predicting unit associates the state quantity of the moving object predicted for each of the plurality of moving objects with the detection result of each of the plurality of moving objects, and stores the moving object state quantity The state quantity and the last observed time of each of the plurality of moving objects stored in the means are updated.

  In this way, every time receiving a detection result of detecting a plurality of moving objects from any of a plurality of indefinite number of sensors, using the moving object movable range information in which the movable range of the moving object is recorded, The state quantity of the moving object predicted for each of the plurality of moving objects corresponding to the time is associated with the detection result of each of the plurality of moving objects, and the state quantity of each of the plurality of moving objects and finally By updating the observed time, even if the number of sensors of the asynchronous multiple sensors is indefinite, the results of the multiple sensors can be integrated and the state quantity of the moving object can be obtained with high accuracy.

  The moving object state quantity estimation device according to a third aspect of the present invention is the moving object state quantity storage means for storing the state quantity and the last observed time of each of the plurality of moving objects, and the moving object A moving object state quantity predicting means for predicting a state quantity of the moving object at the next time using the state quantity stored in the object state quantity storing means, and a plurality of moving objects from any of a plurality of indefinite number of sensors. Sensing result acquisition means for receiving the detected detection result, and when the detection result is received by the sensing result acquisition means, each of the plurality of moving objects by the moving object state quantity prediction means corresponding to the time of the detection result The plurality of movements stored in the moving object state quantity storage unit, in which the state quantity of the moving object predicted for the moving object is associated with the detection results of the plurality of moving objects. Renew each state quantity and the last observed time, and when the detection result is not received by the sensing result acquisition means, using the moving object movable range information in which the movable range of the moving object is recorded, The state quantity of each of the plurality of moving objects stored in the moving object state quantity storage means after correcting the state quantity of the moving object predicted for each of the plurality of moving objects by the moving object state quantity prediction means. And moving object state quantity matching means for updating.

  According to a fourth aspect of the present invention, there is provided a program including a moving object state quantity storage unit that stores a state quantity and a last observed time for each of a plurality of moving objects, the moving object state for each of the plurality of moving objects. Detection by detecting a plurality of moving objects from either a moving object state quantity predicting means for predicting a state quantity of the moving object at the next time using a state quantity stored in the quantity storage means, or an indefinite number of sensors. When the detection result is received by the sensing result acquisition means for receiving the result and the sensing result acquisition means, each of the plurality of moving objects is predicted by the moving object state quantity prediction means corresponding to the time of the detection result. The state quantity of the moving object is associated with the detection results of each of the plurality of moving objects, and the compound stored in the moving object state quantity storage unit is stored. When the state quantity and the last observed time of each moving object are updated, and the detection result is not received by the sensing result acquisition means, the moving object movable range information in which the movable range of the moving object is recorded is recorded. And correcting the state quantity of the moving object predicted for each of the plurality of moving objects by the moving object state quantity predicting means, and storing each of the plurality of moving objects stored in the moving object state quantity storage means. This is a program for functioning as a moving object state quantity matching means for updating the state quantity.

  According to the third and fourth aspects of the invention, the sensing result acquisition means receives detection results obtained by detecting a plurality of moving objects from any of a plurality of indefinite number of sensors. For each of the plurality of moving objects, the moving object state quantity predicting means predicts the state quantity of the moving object at the next time using the state quantity stored in the moving object state quantity storage means.

  Then, when the detection result is received by the sensing result acquisition means by the moving object state quantity matching means, each of the plurality of moving objects is predicted by the moving object state quantity prediction means corresponding to the time of the detection result. The state quantity and the last of each of the plurality of moving objects stored in the moving object state quantity storage means are associated with the detected state quantity of the moving object and the detection results of each of the plurality of moving objects. Update the observed time.

  Further, when the detection result is not received by the sensing result acquisition means by the moving object state quantity matching means, the moving object state quantity is recorded using the moving object movable range information in which the movable range of the moving object is recorded. The state quantity of the moving object predicted for each of the plurality of moving objects by the prediction unit is corrected, and the state quantity of each of the plurality of moving objects stored in the moving object state quantity storage unit is updated.

  As described above, when the detection result of detecting a plurality of moving objects is not received from any of a plurality of indefinite number of sensors, the moving object moving range information in which the moving range of the moving object is recorded is used. The state quantity of the moving object predicted for each of the plurality of moving objects is corrected, and when the detection result of detecting the plurality of moving objects is received from any of a plurality of indefinite number of sensors, the time of the detection result is Correspondingly, the state quantity of the moving object predicted for each of the plurality of moving objects is associated with the detection result of each of the plurality of moving objects, and the state quantity and the last observation of each of the plurality of moving objects are performed. By updating the time, even if the number of sensors of the asynchronous multiple sensors is indefinite, the results of the multiple sensors can be integrated to obtain the state quantity of the moving object with high accuracy.

  Moreover, the program of the said invention can also be stored in a recording medium and provided.

  As described above, according to the moving object state quantity estimation device and the program of the present invention, even if the number of sensors of the asynchronous plural sensors is indefinite, the results of the plural sensors are integrated to obtain the moving object state quantity. The effect that it can obtain | require with sufficient precision is acquired.

It is a block diagram which shows the pedestrian state quantity estimation system which concerns on the 1st Embodiment of this invention. It is a block diagram which shows the local server which concerns on the 1st Embodiment of this invention. It is a flowchart which shows the content of the pedestrian state quantity estimation process routine in the local server which concerns on the 1st Embodiment of this invention. It is a figure for demonstrating the method to integrate the result of multiple sensors in consideration of the movable range of a pedestrian. It is a figure which shows transmission / reception of the data in a pedestrian state quantity estimation system. It is a figure for demonstrating the effect of the method of integrating the result of multiple sensors in consideration of the movable range of a pedestrian. It is a figure for demonstrating the method of correcting the prediction result of a state quantity in consideration of the movable range of a pedestrian. It is a block diagram which shows the local server which concerns on the 2nd Embodiment of this invention. It is an image figure of sensing result integration based on existence probability using the detection result from a plurality of asynchronous sensors. It is a flowchart which shows the content of the pedestrian state quantity estimation process routine in the local server which concerns on the 2nd Embodiment of this invention. It is a block diagram which shows the local server which concerns on the 3rd Embodiment of this invention.

  Embodiments of the present invention will be described below with reference to the drawings.

<Outline of Embodiment of the Present Invention>
The embodiment of the present invention takes an approach in which frame synchronization is not performed between a plurality of sensors. At the timing when the sensing result from a sensor is received, the state quantity (hypothesis) of the pedestrian position is associated with the received sensing result, the error is calculated, and the pedestrian is detected while correcting the hypothetical pedestrian position.・ Follow. This process is performed every time the sensing result reaches the integrated processing device from each sensor.

  However, since this approach sequentially performs the above processing using only the result of one sensor, if the detection rate of the sensor is bad (the frequency of the result is high if there is a pedestrian accidentally where there is no pedestrian) Case), there is a problem that hypotheses of pedestrians that do not exist increase and the number of pedestrians is estimated incorrectly.

  In order to solve this problem, a detection result of a pedestrian with a high possibility of erroneous detection is rejected using the detection rate of each sensor, the elapsed time since the last observation, map information, and the like.

  In addition, matching processing is performed according to the observation amount generated from information that arrived asynchronously from each sensor, its spatial movement possibility (movable range), and the proximity of the state prediction that has already been held. (Association and correction of state quantity) are performed, and when there is no appropriate association, a state quantity is generated around the observation quantity in consideration of the movable range. Even when the observation amount does not arrive, the correction of the state amount by the movable range continues. As a result, common sense predictions such as pedestrians traveling on the sidewalk can be realized even when the observation amount is not obtained due to communication interruption, so the observation amount after communication return and the existing state amount are appropriately Can be associated. Since the state quantity already held is deleted in the elapsed time since the last correspondence, the robustness against the extremely long-term interruption is maintained.

[First Embodiment]
In the first embodiment of the present invention, a local server that estimates the amount of state of a pedestrian by integrating detection results of a pedestrian detector mounted on a vehicle or a pedestrian detector as an infrastructure sensor. A case where the present invention is applied will be described as an example.

<System configuration of pedestrian state quantity estimation system>
As shown in FIG. 1, the pedestrian state quantity estimation system 100 according to the first embodiment includes a local server 10, a base station 50, a plurality of detectors 60 mounted on a plurality of vehicles, an infrastructure The base station 50 and the local server 10 are connected by a network 70 such as the Internet, and the base station 50 and the detectors 60 and 62 are connected by wireless communication. Yes. The local server 10 is an example of a moving object state quantity estimation device.

  The detectors 60 and 62 detect a pedestrian at any time using a camera or a radar, and transmit a detection result to the local server 10 via the base station 50 each time it is detected.

  For example, the detector 60 extracts image feature quantities (SIFT, FIND, HOG, etc.) for each sliding window from the road image in front captured by a camera that images the front of the host vehicle, and images for each sliding window. A pedestrian is detected using the feature amount and a pedestrian detection model (SVM, AdaBoost), and image coordinates representing the detected pedestrian position are obtained. In addition, the detector 60 converts the image coordinates representing the pedestrian position into a three-dimensional position. At this time, an error variance matrix is obtained according to the detected height of the pedestrian. Moreover, the detector 60 calculates | requires the absolute three-dimensional position of a pedestrian based on the absolute coordinate of the own vehicle measured by GPS mounted in the own vehicle, and the calculated | required three-dimensional position.

The three-dimensional position and the error variance matrix of the pedestrian, determined for each detected pedestrian, a set of three-dimensional position of the pedestrian detected at the detector 60 of the vehicle _{C i Y i = {y i} , 1, ... y _{i, m} } and observation O _i = {Y _i , R _i } consisting of observation error variance matrix set R _i = {ri _{, 1} , ..., ri _{, m} } for each observation, Transmit to the local server 10.

  Similarly to the detector 60, the detector 62 transmits an observation made up of a set of the detected three-dimensional position of the pedestrian and an observation error variance matrix to the local server 10 for each observation.

  The plurality of detectors 60 and 62 detect pedestrians asynchronously. Since the plurality of detectors 60 are mounted on different vehicles, the number of detectors 60 and 62 that transmit detection results to the local server 10 is indefinite.

  The local server 10 includes a CPU, a RAM, and a ROM that stores a program for executing a pedestrian state quantity estimation processing routine described later, and is functionally configured as follows. As shown in FIG. 2, the local server 10 includes a communication unit 12, a sensing result acquisition unit 14, a moving object state quantity storage unit 16, a moving object state quantity prediction unit 18, and a moving object state quantity deletion unit 19. A moving object state quantity matching unit 20, a moving object state quantity updating unit 24, and a moving object movable range map 30.

  The moving object movable range map 30 stores a likelihood field m representing a movable range map of a pedestrian by an obstacle or a sign display. The likelihood field m is acquired from, for example, the remote server 80, and the likelihood field m of the moving object movable range map 30 is updated at a predetermined cycle. For example, assuming a change caused by a traffic light, the likelihood field m of the moving object movable range map 30 is updated at a period of 1 to 10 seconds. When the restriction by the maintenance work (repair, planting pruning, etc.) is assumed, the likelihood field m of the moving object movable range map 30 is updated at a cycle of 1 hour. In addition, when it is assumed that the road structure is changed (road removal / new construction, building construction / extension / removal), the likelihood field m of the moving object movable range map 30 is updated in a cycle of one day.

  The sensing result acquisition unit 14 receives the pedestrian coordinates transmitted from any of the detectors 60 and 62 and the observation error variance matrix every time the communication unit 12 receives an observation made up of the coordinates of the pedestrians. Obtain an observation consisting of a set of and a set of observation error variance matrices.

The moving object state quantity storage unit 16 stores the state quantity (position and speed of the pedestrian) updated by the moving object state quantity update unit 24 and the last observed time for each of the observed plurality of pedestrians. . Specifically, the pedestrian state quantity set X = {x ₁ ,... X _n }, and the state quantity variance-covariance matrix V = {v ₁ ,..., V _n }, the last observed time The set T = {t ₁ ,... T _n } is stored. In this embodiment,

Is called a hypothesis in belief space and its components

Is called a tracker.

The moving object state quantity prediction unit 18 walks at the next time using a state quantity stored in the moving object state quantity storage unit 16 for each of a plurality of pedestrians according to the prediction step of the particle filter in accordance with the current time. Predicting the state quantity of the pedestrian, the set of state quantities of the pedestrian at the current time X = {x ₁ ,... X _n }, and the variance-covariance matrix V = {v ₁ ,. _n }. For example, the state quantity of the pedestrian at the next time is predicted by constant speed prediction or the like.

Specifically, hypothesis

For the current time state

Is generated.

When the moving object state quantity matching unit 20 receives the latest observation O _i from the sensing result acquisition unit 14, the moving object state quantity matching unit 20 uses the likelihood field m of the moving object movable range map 30 to correspond to the current time. The state quantity predicted for each of the plurality of pedestrians by the quantity prediction unit 18 is associated with the detection results of each of the plurality of pedestrians represented by the latest observation O _i .

Specifically, using the likelihood field m of the moving object movable range map 30, the pedestrian state quantity set X = {x ₁ ,... X _n } at the current time and the latest observation O _i are detected. Is associated with the set Y _i = {y _{i, 1} ,... Y _{i, m} } of the three-dimensional position of the pedestrian. For example, a tracker for observation O _i

Observation likelihood means goodness of fit

Including the likelihood field m (x _n ) as follows.

And the associated tracker

The correlation that maximizes the product of the observation likelihood θ _{i, j} of the combination with the detected three-dimensional position y _{i, j of} the pedestrian is calculated at high speed by a technique such as the Hungarian method.

  In addition, an observation likelihood that is not associated is set as a set value, and combinations of observation likelihoods that are smaller than the set value are not associated.

When the moving object state quantity matching unit 20 does not receive the latest observation O _i from the sensing result acquisition unit 14, the moving object state quantity matching unit 20 is predicted for each of the plurality of trackers by the moving object state quantity prediction unit 18 corresponding to the current time. A plurality of particles representing the pedestrian's state quantity obtained by sampling based on the state quantity are generated for each of the trackers, and the likelihood of the moving object movable range map 30 for each of the particles. A map likelihood ω is calculated using the field m.

Specifically, state prediction

With respect to sigma points

Sampling by.

Here, α and к are setting parameters for adjusting the spread from the average value of the sigma points. D is the dimension of x _n . For example, three particles ψ _{n, 1} , ψ _{n, 2} , ψ _{n, 3} are generated for one tracker.

And the map likelihood for sigma point ψ

Is calculated from the likelihood field m (ψ _n ) as follows.

When the moving object state quantity update unit 24 receives the latest observation O _i from the sensing result acquisition unit 14, the moving object state quantity update unit 24 performs tracking for each of the trackers associated with the latest observation O _i by the moving object state quantity matching unit 20. By using the corresponding three-dimensional position y _{i, j} as an observation value, the error variance matrix r _{i, j} , the pedestrian state quantity and the variance covariance matrix obtained in the prediction step, and the filtering step of the particle filter The state quantity is updated, and the last observed time is updated.

Also, the moving object state quantity updating unit 24, based on the association of the results of the detection results of the most recent observation O _i by the moving object state quantity matching unit 20, among the latest observation O _i, either with and corresponding tracker A tracker including a new state quantity x is generated from the detection result of the pedestrian that did not exist, and the current time is stored in the moving object state quantity storage unit 16 as the last observed time. The tracker including the new state quantity x has the same coordinates and error variance matrix as the observed three-dimensional position y of the pedestrian.

The moving object state quantity updating unit 24 weights the map likelihood ω obtained by the moving object state quantity matching unit 20 for each of the trackers when the sensing result acquisition unit 14 does not receive the latest observation O _i. Tracker by weighted average with

Update the state quantity and the variance-covariance matrix.

  The moving object state quantity deleting unit 19 deletes the tracker including the state quantity of the pedestrian who has passed a predetermined time or more from the last observed time from the moving object state quantity storage unit 16.

  The local server 10 outputs the state quantity set X updated by the above series of processes as an integrated result of the detection results of the pedestrians.

<Operation of Pedestrian State Quantity Estimation System 100>
Next, the operation of the pedestrian state quantity estimation system 100 according to the first embodiment will be described. First, a pedestrian is sequentially detected by each of a plurality of detectors 60 mounted on a plurality of vehicles and a detector 62 as an infrastructure sensor, and each time a pedestrian is detected, a detection result is obtained via the base station 50. When the data is transmitted to the local server 10, the local server 10 executes a pedestrian state quantity estimation processing routine shown in FIG.

  In step S100, the hypothesis is initialized to an empty set as follows.

  In step S102, a difference between the processing time in the previous step S104 and the current time is calculated.

In step S104, the moving object state quantity storage unit 16 stores each of the plurality of trackers corresponding to the plurality of pedestrians by the particle filter prediction step by the difference time calculated in step S102. It repeats predicting the pedestrian's state quantity at the next time using the state quantity, and sets the pedestrian's state quantity at the current time X = {x ₁ ,... X _n } and its variance-covariance matrix V = {V ₁ ,..., V _n } is obtained.

  In step S106, the tracker including the state quantity of the pedestrian who has passed a certain time or more from the last observed time is deleted from the moving object state quantity storage unit 16 (see FIG. 4).

In step S108, a pedestrian three-dimensional position set Y _i = {y _{i, 1} ,... Y _{i, m} } as a pedestrian detection result from any of the plurality of detectors 60 and 62. Then, it is determined whether or not the latest observation O _i composed of the set of observation error variance matrices R _i = {ri _{, 1} ,... Ri _{, m} } has been received. If the latest observation O _i has been received, the process proceeds to step S110, and if the latest observation has not been received, the process proceeds to step S120.

In step S110, the three-dimensional position of the pedestrian included in the latest observation O _i received in step S100 is converted from the sensor coordinate system to the world coordinate system.

In step S112, based on the latest observation O _i , the prediction result predicted in step S104, and the likelihood field m of the moving object movable range map 30, the tracker

For each of the above, an observation likelihood θ _i indicating the goodness of fit of the tracker is calculated.

In step S114, based on the observation likelihood θ _i calculated in step S112, the tracker

Is associated with the three-dimensional position of the pedestrian detected by the latest observation O _i .

In step S116, for each of the trackers associated with the three-dimensional position of the pedestrian detected by the latest observation O _i in step S114, the corresponding three-dimensional position y _{i, j} is used as an observation value, and the error is determined. Using the variance matrix r _{i, j} and the pedestrian's state quantity and variance covariance matrix obtained in step S104, the state quantity is updated by the filtering step of the particle filter, and finally observed. Update the time. In addition, a tracker including a new state quantity x is generated based on the three-dimensional position of the pedestrian not associated with the tracker among the three-dimensional positions of the pedestrian detected by the latest observation O _i (see FIG. 4), the current time is stored in the moving object state quantity storage unit 16 as the last observed time.

  In step S118, the updated set X of state quantities is output as an integrated result of the pedestrian detection results, and the process returns to step S102. For example, the integration result of the detection result of the pedestrian is transmitted to the remote server 80 (see FIG. 5).

  In step S120, a plurality of particles representing a pedestrian's state quantity obtained by sampling based on the state quantity predicted for the tracker by the moving object state quantity prediction unit 18 corresponding to the current time are tracked. Generate for each instrument.

  In step S122, the map likelihood ω is calculated for each of the plurality of trackers using the likelihood field m of the moving object movable range map 30 for each of the particles generated in step S120.

  In step S124, for each of the trackers, based on each of the particles generated in step S120 and the map likelihood ω obtained in step S122, a weighted average with the map likelihood ω as a weight is used. The state quantity of the tracker is updated, and the process returns to step S102.

  As described above, according to the pedestrian state quantity estimation system according to the first embodiment, every time a detection result of detecting a plurality of pedestrians is received from any of a plurality of indefinite number of detectors, Using the moving object movable range map, the state quantity of the pedestrian predicted for each of the plurality of pedestrians corresponding to the time of the detection result and the detection result of each of the plurality of pedestrians are associated, By updating the state quantity and the last observed time of each pedestrian, even if the number of sensors of asynchronous multiple sensors is indefinite, the results of multiple sensors are integrated and the pedestrian state quantity is accurately determined. Can be sought.

  Moreover, when the detection result which detected the several pedestrian was not received from any of several indefinite number of detectors, it was estimated about each of several pedestrians using a moving object movable range map. When the detection result of detecting a plurality of pedestrians is received from any of a plurality of indefinite number of detectors after correcting the state quantity of the pedestrian, each of the plurality of pedestrians corresponding to the time of the detection result By associating the predicted state quantity of the pedestrian with the detection results of each of the plurality of pedestrians, and updating the state quantity and the last observed time of each of the plurality of pedestrians, the plurality of asynchronous sensors Even if the number of sensors is indefinite, the results of multiple sensors can be integrated to determine the pedestrian's state quantity with high accuracy.

  Also, when sensing results come from multiple unspecified sensors asynchronously, the state quantity added from one sensor information is deleted based on the elapsed time since detection, so that the influence of unreliable detection results As a result, the detection error of each sensor is robust. As described above, when the sensing results from a plurality of sensors coming asynchronously are integrated in real time, the detection error of each sensor is robust.

  Further, by directly incorporating the movable range of the pedestrian into the matching process, it is possible to appropriately associate the tracker with the detection result even if the observation is incorrect (see FIG. 6). Further, every time the sensing result arrives, it is possible to calculate the proximity (error) in association with the state quantity (hypothesis) such as the position / velocity of the pedestrian and to track it while correcting the hypothetical state quantity. While the observation amount does not reach, the existing state amount is continuously predicted for movement with reference to the movable range of the pedestrian. Furthermore, since the movable range of the pedestrian is directly incorporated into the matching process, it is possible to prevent a state quantity from being generated at a position where a pedestrian cannot clearly exist, such as the inside of an obstacle.

  Moreover, if rough information such as whether a pedestrian or a vehicle is known from the sensing result, the movement prediction result is corrected within the matching process, so a reasonable prediction can be made without a detailed movement control model for moving objects. This is possible (see FIG. 7).

  Further, the attribute of the moving object may be recognized from the sensing result, and the likelihood field that varies depending on the attribute of the moving object may be used. For example, as shown in Table 1 below, a pedestrian likelihood field, an automobile likelihood field, and a bicycle likelihood field are prepared, and filtering is performed by switching to a different likelihood field according to the attribute of the particle. You can also.

  Also, when sensing results from a plurality of sensors that arrive asynchronously via a network are integrated in real time, the system is robust against temporary communication interruptions and erroneous detection / non-detection.

[Second Embodiment]
<System configuration of pedestrian state quantity estimation system>
Next, a pedestrian state quantity estimation system according to the second embodiment will be described. In addition, about the part which becomes the structure similar to 1st Embodiment, the same code | symbol is attached | subjected and description is abbreviate | omitted.

  In the second embodiment, the existence probability of each of the plurality of pedestrians is held, and the existence probability of each of the plurality of pedestrians is updated based on the association between the detection result and the state quantity, and the state The point which considers the movable range of a pedestrian when estimating quantity differs from 1st Embodiment.

  As shown in FIG. 8, the local server 210 according to the second embodiment includes a communication unit 12, a sensing result acquisition unit 14, a moving object state quantity storage unit 16, and a moving object state quantity prediction unit 218. , The moving object state quantity erasing unit 19, the moving object state quantity matching unit 20, the moving object state quantity updating unit 24, the moving object existence probability updating unit 220, the moving object existence probability storage unit 222, and the moving object movable range. And a map 30.

  The moving object existence probability storage unit 222 stores the existence probabilities of the plurality of pedestrians corresponding to the plurality of trackers stored in the moving object state quantity storage unit 16.

  The moving object existence probability update unit 220 determines the detection result of the pedestrian that has not been associated with the tracker among the latest observations based on the result of the association with the latest observation by the moving object state quantity matching unit 20. The set value is stored in the moving object existence probability storage unit 222 as the existence probability of the pedestrian. The set value may be determined according to the detection accuracy predetermined for the detectors 60 and 62.

  The moving object existence probability update unit 220 corresponds to the latest observation detection result by the moving object state quantity matching unit 20 among the existence probabilities of the plurality of pedestrians stored in the moving object existence probability storage unit 222. Each of the presence probability of the pedestrian of the attached tracker is updated so as to increase according to the detection accuracy predetermined for the detectors 60 and 62 (see FIG. 9).

  The moving object existence probability update unit 220 corresponds to the latest observation detection result by the moving object state quantity matching unit 20 among the existence probabilities of the plurality of pedestrians stored in the moving object existence probability storage unit 222. Each of the pedestrian presence probabilities of the tracker that is not attached is updated by being attenuated so as to have an existence probability corresponding to the elapsed time from the last observed time (see FIG. 9).

  The moving object state quantity elimination unit 19 erases the tracker including the state quantity of the pedestrian whose existence probability stored in the moving object existence probability storage unit 222 is below the threshold (see FIG. 9). Note that the threshold value is a value corresponding to the existence probability of a pedestrian who has passed a certain time since the last observed time.

The moving object state quantity prediction unit 218 uses the state quantity stored in the moving object state quantity storage unit 16 for each of a plurality of pedestrians according to the prediction step of the particle filter in accordance with the current time. repeatedly predicting a state quantity of persons, set _{X = {x 1, ... x} n} of the state of the pedestrian at the current time, and its variance-covariance matrix _{V = {v 1, ...,} v n} Ask for. At this time, when the position included in the pedestrian's state quantity is outside the range where the pedestrian determined by the moving object movable range map 30 can be located, the position included in the pedestrian's state quantity is The state quantity of the pedestrian is corrected so that the pedestrian defined by the moving object movable range map 30 is within a range where the pedestrian can be located.

  In addition, since the other structure of the pedestrian state quantity estimation system which concerns on 2nd Embodiment is the same as that of 1st Embodiment, description is abbreviate | omitted.

<Operation of pedestrian state quantity estimation system>
Next, the operation of the pedestrian state quantity estimation system according to the second embodiment will be described. First, a pedestrian is sequentially detected by each of a plurality of detectors 60 mounted on a plurality of vehicles and a detector 62 as an infrastructure sensor, and the detection result is sent to the local server 210 via the base station 50. During transmission, the local server 210 executes a pedestrian state quantity estimation processing routine shown in FIG. In addition, about the process similar to 1st Embodiment, the same code | symbol is attached | subjected and detailed description is abbreviate | omitted.

  In step S100, the hypothesis is initialized to an empty set.

  In step S102, a difference between the processing time in the previous step S200 and the current time is calculated.

In step S200, the moving object state quantity storage unit 16 stores each of the plurality of trackers corresponding to the plurality of pedestrians by the particle filter prediction step for the difference time calculated in step S102. It repeats predicting the pedestrian's state quantity at the next time using the state quantity, a set of state quantities of a plurality of pedestrians at the current time X = {x ₁ ,... X _n }, and its variance covariance A matrix V = {v ₁ ,..., V _n } is obtained. And when the position included in the pedestrian's state quantity is outside the range where the pedestrian determined by the moving object movable range map 30 can be located, the position included in the pedestrian's state quantity moves The state quantity of the pedestrian is corrected so that the pedestrian determined by the object movable range map 30 is within a range where the pedestrian can be located.

  In step S <b> 202, the tracker including the state quantity of the pedestrian whose existence probability stored in the moving object existence probability storage unit 222 falls below the threshold is deleted from the moving object state quantity storage unit 16.

In step S108, it is determined whether or not the latest observation O _i is received from any of the plurality of detectors 60 and 62. If the latest observation O _i has been received, the process proceeds to step S110, and if the latest observation has not been received, the process proceeds to step S120.

In step S110, the three-dimensional position of the pedestrian included in the latest observation O _i received in step S100 is converted from the sensor coordinate system to the world coordinate system.

In step S112, based on the latest observation O _i , the prediction result predicted in step S104, and the likelihood field m of the moving object movable range map 30, the tracker

For each of the above, an observation likelihood θ _i indicating the goodness of fit of the tracker is calculated.

In step S114, based on the observation likelihood θ _i calculated in step S112, the tracker

Is associated with the three-dimensional position of the pedestrian detected by the latest observation O _i .

  In step S204, among the existence probabilities of the plurality of pedestrians stored in the moving object existence probability storage unit 222, the pedestrian corresponding to the tracker associated with the latest observation detection result in step S114 is used. For each of the existence probabilities, the detectors 60 and 62 are updated so as to increase in accordance with a predetermined detection accuracy. In addition, among the existence probabilities of the plurality of pedestrians stored in the moving object existence probability storage unit 222, the presence of pedestrians corresponding to the tracker that is not associated with the latest observation detection result in step S114. Each of the probabilities is updated by being attenuated so as to have an existence probability corresponding to the elapsed time from the last observed time.

In step S116, for each of the trackers associated with the three-dimensional position of the pedestrian detected by the latest observation O _i in step S114, the corresponding three-dimensional position y _{i, j} is used as an observation value, and the error is determined. Using the variance matrix r _{i, j} and the walker's state quantity and variance covariance matrix obtained in step S200, the state quantity is updated by the filtering step of the particle filter and finally observed. Update the time. Further, a tracker including a new state quantity x is generated based on the three-dimensional position of the pedestrian that has not been associated with the tracker among the three-dimensional positions of the pedestrian detected by the latest observation O _i. The current time is stored in the moving object state quantity storage unit 16 as the observed time. Further, the predetermined existence probability is stored in the moving object existence probability storage unit 222.

  In step S118, the updated set X of state quantities is output as an integrated result of the pedestrian detection results, and the process returns to step S102.

  In step S120, a plurality of particles representing a pedestrian's state quantity obtained by sampling based on the state quantity predicted for each of the plurality of trackers by the moving object state quantity prediction unit 18 corresponding to the current time, Generate for each tracker.

  In step S122, the map likelihood ω is calculated for each of the plurality of trackers using the likelihood field m of the moving object movable range map 30 for each of the particles generated in step S120.

  In step S124, for each of the trackers, based on each of the particles generated in step S120 and the map likelihood ω obtained in step S122, a weighted average with the map likelihood ω as a weight is used. The state quantity of the tracker is updated, and the process returns to step S102.

  As described above, according to the pedestrian state quantity estimation system according to the second embodiment, when a sensing result comes from a plurality of unspecified sensors asynchronously, a state quantity added from one sensor information, By deleting based on the existence probability determined from the sensor performance or the like, the influence of the unreliable detection result does not remain as a state quantity, so that it is robust against erroneous detection of each sensor.

  In addition, when the moving object movable range map has precise information on the pedestrian, the predicted range of the pedestrian is determined using the range in which the pedestrian defined by the moving object movable range map can be located as a restriction. Correct the state quantity. As described above, the moving object movable range information is directly reflected in the prediction of the moving object state quantity, thereby enabling a more accurate matching process.

[Third Embodiment]
<System configuration of pedestrian state quantity estimation system>
Next, a pedestrian state quantity estimation system according to the third embodiment will be described. In addition, about the part which becomes the same structure as 1st Embodiment and 2nd Embodiment, the same code | symbol is attached | subjected and description is abbreviate | omitted.

  The third embodiment is different from the second embodiment in that the pedestrian existence probability is updated when the movable range of the pedestrian is taken into consideration.

  As shown in FIG. 11, the local server 310 according to the third embodiment includes a communication unit 12, a sensing result acquisition unit 14, a moving object state quantity storage unit 16, and a moving object state quantity prediction unit 18. The moving object state quantity erasing unit 19, the moving object state quantity matching unit 20, the moving object state quantity updating unit 24, the moving object existence probability updating unit 320, the moving object existence probability storage unit 222, and the moving object movable range. And a map 30.

  The moving object existence probability update unit 320 associates the tracker with the tracker out of the latest observation detection results based on the result of the association between the tracker and the latest observation detection result by the moving object state quantity matching unit 20. For the detection result of the pedestrian that did not exist, the setting value is stored in the moving object existence probability storage unit 222 as the existence probability of the pedestrian, as in the second embodiment.

  In addition, the moving object existence probability update unit 320 corresponds to the detection result of the latest observation by the moving object state quantity matching unit 20 among the existence probabilities of the plurality of pedestrians stored in the moving object existence probability storage unit 222. For each of the pedestrian existence probabilities corresponding to the attached trackers, the detectors 60 and 62 are updated so as to increase in accordance with the predetermined detection accuracy, as in the second embodiment.

  In addition, the moving object existence probability update unit 320 corresponds to the detection result of the latest observation by the moving object state quantity matching unit 20 among the existence probabilities of the plurality of pedestrians stored in the moving object existence probability storage unit 222. For each of the pedestrian existence probabilities corresponding to the trackers not attached, as in the second embodiment, the pedestrian existence probabilities are attenuated so as to become the existence probabilities corresponding to the elapsed time from the last observed time. Update.

  In addition, the moving object existence probability update unit 320 can be located at a position included in the pedestrian state quantity stored in the moving object state quantity storage unit 16 by a pedestrian determined by the moving object movable range map 30. If it is out of the range, it is updated so as to reduce the existence probability of the pedestrian.

  In addition, since the other structure and effect | action of the pedestrian state quantity estimation system which concern on 3rd Embodiment are the same as that of 2nd Embodiment, description is abbreviate | omitted.

  In this way, by using the range in which the pedestrian defined by the moving object movable range map can be located as a restriction, by updating the existence probability of the pedestrian, appropriately eliminating the state quantity of the pedestrian that is not Can do.

  In the above-described embodiment, the case where a pedestrian is targeted as a moving object to be detected has been described as an example. However, the present invention is not limited to this. For example, another moving object such as a vehicle is detected. It is good.

  Moreover, although the case where a plurality of detectors mounted on a plurality of vehicles and a detector as an infrastructure sensor is an asynchronous plurality of sensors has been described as an example, the present invention is not limited to this, and detection as an infrastructure sensor You may comprise so that a container may not be used.

  A plurality of detectors that are mounted on a single vehicle and that use a plurality of measuring devices including a camera and a radar may be asynchronous multiple sensors. For example, when a detector using a different measuring instrument is added to the vehicle as a retrofit, the number of multiple sensors is assumed to be indefinite.

  Further, the case where the state quantity is predicted and updated using the particle filter has been described as an example. However, the present invention is not limited to this, and the state quantity may be predicted and updated using the Kalman filter. Good.

  The program of the present invention can be provided by being stored in a recording medium.

10, 210, 310 Local server 12 Communication unit 14 Sensing result acquisition unit 16 Moving object state quantity storage unit 18, 218 Moving object state quantity prediction unit 19 Moving object state quantity elimination unit 20 Moving object state quantity matching unit 24 Moving object state Quantity updating unit 30 Moving object movable range map 50 Base station 60, 62 Detector 70 Network 100 Pedestrian state quantity estimation system 220, 320 Moving object existence probability updating unit 222 Moving object existence probability storage unit

Claims (12)

A moving object state quantity storage means for storing a state quantity and the last observed time of each of the plurality of moving objects;
For each of the plurality of moving objects, a moving object state quantity prediction unit that predicts a state quantity of the moving object at the next time using the state quantity stored in the moving object state quantity storage unit;
Sensing result acquisition means for receiving detection results obtained by detecting a plurality of moving objects from any of a plurality of indeterminate sensors;
Each time the detection result is received by the sensing result acquisition means, the moving object state amount prediction means corresponding to the time of the detection result using the moving object movable range information in which the movable range of the moving object is recorded. The plurality of movements stored in the moving object state quantity storage means by associating the state quantities of the moving objects predicted for the plurality of moving objects with the detection results of the plurality of moving objects, respectively. Moving object state quantity matching means for updating the state quantity and the last observed time of each object;
A moving object state quantity estimation device. A moving object state quantity storage means for storing a state quantity and the last observed time of each of the plurality of moving objects;
For each of the plurality of moving objects, a moving object state quantity prediction unit that predicts a state quantity of the moving object at the next time using the state quantity stored in the moving object state quantity storage unit;
Sensing result acquisition means for receiving detection results obtained by detecting a plurality of moving objects from any of a plurality of indeterminate sensors;
When the detection result is received by the sensing result acquisition means, the state quantity of the moving object predicted for each of the plurality of moving objects by the moving object state quantity prediction means corresponding to the time of the detection result; Associating with the detection results of each of a plurality of moving objects, updating the state quantities and the last observed time of each of the plurality of moving objects stored in the moving object state quantity storage means,
When the detection result is not received by the sensing result acquisition unit, the moving object state amount prediction unit uses the moving object moving range information in which the moving range of the moving object is recorded for each of the plurality of moving objects. A moving object state quantity matching unit that corrects the predicted state quantity of the moving object and updates the state quantity of each of the plurality of moving objects stored in the moving object state quantity storage unit;
A moving object state quantity estimation device.   The moving object state quantity matching means, for each of the plurality of moving objects, when the moving object state quantity prediction means predicts the moving object for each of the plurality of moving objects when the sensing result acquisition means does not receive the detection result. Generating a plurality of particles representing the state quantity of the moving object obtained by sampling based on the state quantity of the object, and predicting the moving object based on the plurality of particles and the moving object movable range information The moving object state quantity estimation apparatus according to claim 2, wherein the state quantity of each of the plurality of moving objects stored in the moving object state quantity storage unit is updated by correcting the state quantity of the moving object.   The moving object state quantity estimation device according to claim 1, wherein the plurality of sensors include sensors mounted on different moving bodies.   The state quantity of the moving object that has not been associated with the detection result by the moving object state quantity matching unit, and the state quantity of the moving object that has passed a predetermined time or more from the last observed time, The moving object state quantity estimation device according to any one of claims 1 to 4, further comprising a moving object state quantity erasing means for erasing from the object state quantity storage means. Moving object existence probability storage means for storing the existence probability of each of the plurality of moving objects;
The moving object existence probability is set in the moving object existence probability storage means according to the result of association with the detection result by the moving object state quantity matching means or the detection accuracy predetermined for the sensor. Or the existence probability of each of the plurality of moving objects stored in the moving object existence probability storage means, as a result of the association with the detection result by the moving object state quantity matching means, from the last observed time Moving object existence probability update means for increasing or decreasing according to the elapsed time of the above, or detection accuracy predetermined for the sensor,
Further including
The moving object state quantity erasure unit is configured to extract the state quantity of the moving object from the moving object state quantity storage unit based on the existence probability of each of the plurality of moving objects stored in the moving object existence probability storage unit. The moving object state quantity estimation device according to claim 5 to be erased.   The moving object according to any one of claims 1 to 6, wherein the moving object state quantity predicting unit predicts a state quantity of the moving object at a next time using the moving object movable range information as a constraint. State quantity estimation device.   The moving object existence probability update means increases or decreases each existence probability of the plurality of moving objects stored in the moving object existence probability storage means, using the moving object movable range information as a constraint. Moving object state quantity estimation device. The plurality of sensors are a plurality of detectors for detecting a pedestrian or a vehicle mounted on a plurality of vehicles or used for an infrastructure sensor,
The state quantity of the moving object is the position and speed of a pedestrian or vehicle,
The moving object state estimation device according to claim 1, wherein the moving object movable range information is a map representing a road or a sidewalk. The plurality of sensors are a plurality of detectors for detecting a pedestrian or a vehicle using a plurality of measuring devices including a camera and a radar mounted on a single vehicle,
The state quantity of the moving object is the position and speed of a pedestrian or vehicle,
The moving object state estimation device according to claim 1, wherein the moving object movable range information is a map representing a road or a sidewalk. A computer including moving object state quantity storage means for storing the state quantity and the last observed time of each of the plurality of moving objects,
For each of the plurality of moving objects, a moving object state quantity predicting unit that predicts a state quantity of the moving object at the next time using a state quantity stored in the moving object state quantity storage unit;
Sensing result acquisition means for receiving detection results obtained by detecting a plurality of moving objects from any of a plurality of indefinite sensors, and each time the detection result is received by the sensing result acquisition means, the movable range of the moving object is recorded. Using the moving object movable range information, the state quantity of the moving object predicted for each of the plurality of moving objects by the moving object state quantity prediction unit corresponding to the time of the detection result, and each of the plurality of moving objects Functioning as a moving object state quantity matching means for updating the state quantity and the last observed time of each of the plurality of moving objects stored in the moving object state quantity storage means. Program to let you. A computer including moving object state quantity storage means for storing the state quantity and the last observed time of each of the plurality of moving objects,
For each of the plurality of moving objects, a moving object state quantity predicting unit that predicts a state quantity of the moving object at the next time using a state quantity stored in the moving object state quantity storage unit;
Sensing result acquisition means for receiving detection results obtained by detecting a plurality of moving objects from any of a plurality of indeterminate sensors, and when the detection result is received by the sensing result acquisition means, corresponds to the time of the detection results. The moving object state quantity predicting unit associates the state quantity of the moving object predicted for each of the plurality of moving objects with the detection result of each of the plurality of moving objects, and the moving object state quantity storing unit Update the state quantity and the last observed time of each of the plurality of moving objects stored in
When the detection result is not received by the sensing result acquisition unit, the moving object state amount prediction unit uses the moving object moving range information in which the moving range of the moving object is recorded for each of the plurality of moving objects. A program for correcting the predicted state quantity of the moving object and functioning as a moving object state quantity matching means for updating the state quantity of each of the plurality of moving objects stored in the moving object state quantity storage means .