JP2020154526A - Mobile object state quantity estimation device and program - Google Patents

Abstract

To accurately calculate a state quantity of a mobile object while taking a detection frequency into consideration even in a case where the number of multiple asynchronous sensors is undefined.SOLUTION: A sensing result acquisition section (14) acquires a detection result of detecting multiple mobile objects by an undefined number of multiple sensors. A mobile object state quantity prediction section (18) predicts, for each of the multiple mobile objects, a state quantity of the mobile object using state quantities stored in a mobile object state quantity storage section (16). A mobile object state quantity alignment section (20) makes the predicted state quantity of the mobile object correspond to the detection result of each of the multiple mobile objects each time the detection result is received by the sensing result acquisition section (14). A mobile object detection frequency prediction section (21) predicts detection frequencies of the mobile objects which are detected by the multiple sensors. A mobile object state quantity erasure section (22) erases a state quantity smaller than the detection frequencies which are predicted by the mobile object detection frequency prediction section (21).SELECTED DRAWING: Figure 2

Description

The present invention relates to a moving object state quantity estimation device and a program.

Conventionally, in order to capture the movement of a pedestrian by a plurality of sensors that are asynchronous and have an indefinite number of sensors, each time a detection result is obtained from a specific sensor, the detection result and the movement indicating the position and speed of the pedestrian are shown. A technique is known in which a state quantity of an object is associated with the state quantity, the last observed time is updated, and the state quantity of a moving object that has not been associated with the detection result for a certain period of time is erased (patented). Document 1).

In addition, when detecting an object by integrating the detection results of each camera using a plurality of camera images, the detection range in the image is limited in advance based on the positional relationship between the object and each camera. There is known an object tracking device capable of improving tracking accuracy by suppressing false detection of (Patent Document 2).

Japanese Unexamined Patent Publication No. 2018-92368 Japanese Unexamined Patent Publication No. 2016-71830

In the technique of Patent Document 1, erroneous detection is suppressed by erasing the state quantity of a moving object when a certain time has passed since the last observation.
However, for example, when the observation time interval changes according to the sensor performance or the detection environment such as the brightness around a moving object changes, the detection frequency of erroneous detection changes. In particular, when the frequency of false detections increases in a time shorter than a certain period of time, it is difficult to distinguish between positive detections and false positives, and there is room for improvement in suppressing false positives.

Further, in Patent Document 2, the detection range is limited in advance based on the positional relationship between the object and the camera to suppress erroneous detection. However, for example, the detection frequency of erroneous detection is different in each of a plurality of sensors having different sensor performances. Even if the detection range is limited based on the positional relationship between the object and the camera, it may be difficult to suppress erroneous detection.

The present invention has been made in view of the above circumstances, and even if the number of sensors of a plurality of asynchronous sensors is indefinite, the results of the plurality of sensors are integrated in consideration of the detection frequency to obtain the state quantity of a moving object. It is an object of the present invention to provide a moving object state quantity estimation device and a program that can be obtained with high accuracy.

In order to achieve the above object, the moving object state quantity estimating device according to the first aspect includes a moving object state quantity storage unit that stores the state quantity and the last observed time of each of the plurality of moving objects, and the plurality of moving object state quantity storage units. For each moving object, either a moving object state quantity predicting unit that predicts the state quantity of the moving object at the next time using the state quantity stored in the moving object state quantity storage unit, or a plurality of sensors having an indefinite number. From the detection result of detecting a plurality of moving objects, and the sensing result acquisition unit that receives the detection performance information, and each time the detection result is received by the sensing result acquisition unit, the moving object corresponds to the time of the detection result. The state amount of the moving object predicted for each of the plurality of moving objects by the state quantity predicting unit is associated with the detection result of each of the plurality of moving objects, and stored in the moving object state quantity storage unit. , The plurality of moving objects for each of the plurality of moving objects based on the state quantity and the moving object state quantity matching unit for updating the last observed time of each of the plurality of moving objects and the detection performance information of each of the plurality of sensors. Among the sensors, the moving object detection frequency predicting unit that predicts the detection frequency detected by the detectable sensor, and the moving object state quantity matching unit that does not correspond to the detection result by the moving object state quantity matching unit. The moving object state in which the detection frequency determined by the detection results of detecting a plurality of moving objects is smaller than the detection frequency predicted by the moving object detection frequency predicting unit is deleted from the moving object state quantity storage unit. It is a moving object state quantity estimation device including a quantity erasing unit.

The second aspect is the moving object state quantity estimation device according to the first aspect, wherein the plurality of sensors include sensors mounted on different moving bodies.

In the third aspect, in the moving object state quantity estimating device according to the first aspect or the second aspect, a moving object existence probability storage unit that stores the existence probability of each of the plurality of moving objects and the moving object state quantity matching unit. The existence probability of the moving object is set in the moving object existence probability storage unit, or the moving object existence probability is set according to the result of the association with the detection result by the above or the detection accuracy predetermined for the sensor. As a result of associating the existence probability of each of the plurality of moving objects stored in the storage unit with the detection result by the moving object state quantity matching unit, the elapsed time from the last observed time, or the sensor. Further includes a moving object existence probability updating unit that increases or decreases according to a predetermined detection accuracy or the detection frequency by the moving object detection frequency prediction unit, and the moving object state quantity erasing unit includes the moving object existence probability. Based on the existence probability of each of the plurality of moving objects stored in the storage unit, the state quantity of the moving object is deleted from the moving object state quantity storage unit.

A fourth aspect is the moving object state quantity estimation device according to claim 3, wherein the moving object detection frequency prediction unit includes a detection performance information storage unit that stores the detection performance information of each of the plurality of sensors, and the detection performance. The detection performance information of each of the plurality of sensors stored in the information storage unit, the moving object detectable range information in which the detectable range of the moving object is recorded, and the moving object updated by the moving object state quantity matching unit. Includes a detection frequency calculation unit that calculates the detection frequency based on the state quantity of.

A fifth aspect is the moving object state quantity estimation device according to claim 3, wherein the moving object detection frequency prediction unit indicates a detection frequency for each moving object detection position predicted by the moving object detection frequency prediction unit. The detection frequency storage unit is based on the detection frequency storage unit that stores information, the detection performance information received by the sensing result acquisition unit, and the moving object detectable range information in which the detectable range of the moving object is recorded. A detection frequency update unit that updates the detection frequency for each detection position of the moving object stored in the storage unit, a detection frequency extraction unit that extracts the detection frequency corresponding to the state amount of the moving object from the detection frequency storage unit, and a detection frequency extraction unit. including.

A sixth aspect is the moving object state quantity estimation device according to the fourth or fifth aspect, wherein the plurality of sensors detect a pedestrian or a vehicle mounted on a plurality of vehicles or used as an infrastructure sensor. The state quantity of the moving object is the position and speed of a pedestrian or a vehicle, and the moving object detectable range information is the position of a structure that reduces the sensor detectable range. It is a map to show.

In the seventh aspect, a computer including a moving object state quantity storage unit that stores the state quantity and the last observed time of each of the plurality of moving objects is installed in the moving object state quantity storage unit for each of the plurality of moving objects. Sensing that receives the detection result of detecting a plurality of moving objects from either the moving object state quantity predicting unit that predicts the state quantity of the moving object at the next time using the stored state quantity or a plurality of sensors of an indefinite number. Each time the result acquisition unit and the sensing result acquisition unit receive the detection result, the state of the moving object predicted by the moving object state quantity prediction unit for each of the plurality of moving objects corresponding to the time of the detection result. The amount is associated with the detection result of each of the plurality of moving objects, and the state quantity and the last observed time of each of the plurality of moving objects stored in the moving object state quantity storage unit are updated. Moving object detection frequency prediction that predicts the detection frequency detected by the detectable sensor among the plurality of sensors for each of the plurality of moving objects based on the detection performance information of the moving object state quantity matching unit and the plurality of sensors. The moving object detection frequency prediction unit determines the detection frequency determined by the detection results of detecting a plurality of moving objects, which is the state quantity of the moving object that does not correspond to the detection result by the moving object state quantity matching unit. This is a program for functioning as a moving object state quantity erasing unit that erases the state quantity of the moving object that is smaller than the detection frequency predicted by the above from the moving object state quantity storage unit.

Further, the program of the present disclosure can also be provided by storing it in a recording medium.

As described above, according to the present disclosure, even if the number of sensors of a plurality of asynchronous sensors is indefinite, the results of the plurality of sensors are integrated in consideration of the detection frequency to accurately obtain the state quantity of a moving object. The effect of being able to do it is obtained.

It is a block diagram which shows the pedestrian state quantity estimation system which concerns on 1st Embodiment. It is a block diagram which shows the pedestrian state quantity estimation apparatus which concerns on 1st Embodiment. It is an image diagram of sensing result integration based on the detection frequency using the detection result from a plurality of asynchronous sensors. It is a flowchart which shows the content of the pedestrian state quantity estimation processing routine in the pedestrian state quantity estimation apparatus which concerns on 1st Embodiment. It is a block diagram which shows the pedestrian state quantity estimation apparatus which concerns on 2nd Embodiment. It is an image diagram of sensing result integration based on the detection frequency using the detection result from a plurality of asynchronous sensors. It is a flowchart which shows the content of the pedestrian state quantity estimation processing routine in the pedestrian state quantity estimation apparatus which concerns on 2nd Embodiment. It is a block diagram which shows the moving object detection frequency prediction part in the pedestrian state quantity estimation apparatus which concerns on 3rd Embodiment. It is a block diagram which shows the moving object detection frequency prediction part in the pedestrian state quantity estimation apparatus which concerns on 4th Embodiment. It is an image diagram which shows the frequency map.

Hereinafter, embodiments that realize the technique of the present disclosure will be described in detail with reference to the drawings.

<Outline of Embodiment>
The embodiment that realizes the technique of the present disclosure takes an approach that does not synchronize frames between a plurality of sensors. At the timing when the sensing result from a certain sensor is received, the state quantity (hypothesis) of the pedestrian position is associated with the received sensing result, the error is calculated, and the pedestrian is corrected while correcting the hypothetical pedestrian position. Detects and follows. This process is performed every time the sensing result arrives at the integrated processing device from each sensor.

However, since this approach sequentially performs the above processing using only the results of one sensor, for example, when the detection performance such as the detection accuracy of the sensor is low (there is a pedestrian erroneously in a place where there is no pedestrian). Then, when the frequency of producing results is high), the hypothesis of non-existent pedestrians due to false detection increases, and the number of pedestrians may be estimated incorrectly.

Therefore, in consideration of the detection frequency of each sensor, the detection result of a pedestrian with a high possibility of erroneous detection is rejected.

[First Embodiment]

In the first embodiment, the present disclosure applies to a pedestrian state amount estimation device that estimates the state amount of a pedestrian by integrating the detection results of a pedestrian detector mounted on a vehicle and a pedestrian detector as an infrastructure sensor. The case where the above technique is applied will be described as an example.

<System configuration of pedestrian state quantity estimation system>
As shown in FIG. 1, the pedestrian state quantity estimation system 100 according to the first embodiment includes a pedestrian state quantity estimation device 10, a base station 50, a plurality of detectors 60 mounted on a plurality of vehicles, and a plurality of detectors 60. A detector 62 as an infrastructure sensor is provided, and the base station 50 and the pedestrian state amount estimation device 10 are connected by a network 70 such as the Internet, and the base station 50 and the detectors 60 and 62 communicate wirelessly. Is connected by.

The detectors 60 and 62 detect pedestrians at any time using a camera or radar, and each time they detect a pedestrian, the detectors 60 and 62 transmit the detection result to the pedestrian state amount estimation device 10 via the base station 50.

For example, the detector 60 extracts an image feature amount (SIFT, FIND, HOG, etc.) for each sliding window from the road image in front captured by a camera that images the front of the own vehicle, and an image for each sliding window. A pedestrian is detected using a feature amount and a pedestrian detection model (SVM, AdaBoost), and image coordinates representing the detected pedestrian position are obtained. Further, the detector 60 converts the image coordinates representing the pedestrian position into a three-dimensional position. At this time, the error variance matrix is obtained according to the detected height of the pedestrian. Further, the detector 60 obtains the absolute three-dimensional position of the pedestrian based on the absolute coordinates of the own vehicle measured by the GPS mounted on the own vehicle and the obtained three-dimensional position.

The three-dimensional position and the error variance matrix of the pedestrian, determined for each detected pedestrian, a set of three-dimensional position of the pedestrian detected at the detector 60 of the vehicle _{C i Y i = {y i} , 1, ... y _{i, m} } and a set of observation error variance matrix _Ri = {ri _{, 1} , ..., ri _{, m} } are transmitted to the pedestrian state amount estimation device 10 for each observation. Further, the detector 60 can transmit sensor information including detection performance information. Examples of the detection performance information of the sensor information include information indicating the detection time, the detection range of the detector 60 (including the position of the detector at the time of detection), the operation cycle, and the undetected rate of moving objects.

Similar to the detector 60, the detector 62 transmits the detected three-dimensional position set of the pedestrian and the set of the observation error variance matrix to the pedestrian state quantity estimation device 10 for each observation.

The plurality of detectors 60 and 62 asynchronously detect pedestrians. Further, since the plurality of detectors 60 are mounted on different vehicles, the number of detectors 60 and 62 for transmitting the detection result to the pedestrian state quantity estimation device 10 is indefinite.

The pedestrian state quantity estimation device 10 is composed of, for example, a server, and the pedestrian state quantity estimation device 10 stores a CPU, a RAM, and a program for executing a pedestrian state quantity estimation processing routine described later. It is functionally configured as shown below. As shown in FIG. 2, the pedestrian state amount estimation device 10 includes a communication unit 12, a sensing result acquisition unit 14, a moving object state amount storage unit 16, a moving object state amount prediction unit 18, and a moving object state amount. It includes a matching unit 20, a moving object detection frequency prediction unit 21, a moving object state quantity erasing unit 22, and a moving object state quantity updating unit 24.

The sensing result acquisition unit 14 receives the pedestrian coordinate set and the observation error dispersion matrix set transmitted from any of the detectors 60 and 62 by the communication unit 12, each time the pedestrian coordinate set and the pedestrian coordinate set are received. Get a set of observation error variance matrices. Further, the sensing result acquisition unit 14 can acquire the sensor information of each of the detectors 60 and 62.

The moving object state quantity storage unit 16 stores the state quantity (position and speed of the pedestrian) updated by the moving object state quantity updating unit 24 of each of the observed pedestrians, and the last observed time. ..

The moving object state quantity prediction unit 18 is stored in the moving object state quantity storage unit 16 by the prediction step of the Kalman filter for each of the plurality of pedestrians in accordance with the time of the latest data acquired by the sensing result acquisition unit 14. Using the state quantity to repeatedly predict the state quantity of the pedestrian at the next time, the set X = {x ₁ , ... x _n } of the state quantity of the pedestrian at the time of the latest data, and the state quantity. The dispersion co-dispersion matrix V = {v ₁ , ..., v _n } of. For example, the state quantity of a pedestrian at the next time is predicted by constant velocity prediction or the like.

Each time the moving object state quantity matching unit 20 receives the latest data from the sensing result acquisition unit 14, the moving object state quantity predicting unit 18 predicts each of the plurality of pedestrians in accordance with the time of the latest data. The state quantity is associated with the detection results of each of the plurality of pedestrians represented by the latest data.

Specifically, a set X = {x ₁ , ... x _n } of the pedestrian state quantity at the time of the latest data and a set Y _i = {of the three-dimensional position of the pedestrian detected in the latest data. Correspondence with y _{i, 1} , ... y _{i, m} }. For example, a Hungarian association that maximizes the product of the probability p (y _{i, j} | x _k ) of the combination of the associated state quantity x _k and the detected three-dimensional position y _{i, j of} the pedestrian. Calculate at high speed by a method such as a method.

The probability p (y _{i, j} | x _k ) of the combination of the state quantity x _k and the detected three-dimensional position y _{i, j of} the pedestrian is the variance of the corresponding Kalman filter centered on the state quantity x _k. Probability based on the probability of 3D position y _{i, j} in a multidimensional normal distribution with the sum of v _k and the observation error variance matrix r _{i, j} corresponding to the 3D position y _{i, j} as the variance p (y _{i, j} | x _k ) may be obtained.

In addition, the probability of not being associated is given as a set value, and a combination of probabilities smaller than this set value is not associated.

The moving object state quantity updating unit 24 uses the corresponding three-dimensional positions y _{i and j} as observation values for each of the state quantities associated with the detection result of the latest data by the moving object state quantity matching unit 20, and the error thereof. Using the variance matrices r _{i and j} and the pedestrian state quantity and the covariance matrix obtained in the prediction step, the state quantity is updated and the last observed time is updated by the filtering step of the Kalman filter.

Further, the moving object state quantity updating unit 24 is based on the result of associating with the detection result by the moving object state quantity matching unit 20, among the detection results of the latest data, the pedestrians who did not correspond to the state quantity. A new state quantity x is generated from the detection result, and the current time is stored in the moving object state quantity storage unit 16 as the last observed time. It is assumed that the new state quantity x has the same coordinates and error variance matrix as the observed three-dimensional position y of the pedestrian.

The moving object detection frequency prediction unit 21 predicts the detection frequency detected by the detectable detector among the plurality of detectors 60 and 62 within a predetermined time between a common time for each of the plurality of pedestrians. That is, a plurality of pedestrians are used by using the detection time, the detection range (including the position of the detector at the time of detection), the operation cycle, and the undetected rate of moving objects, which are the sensor information of each of the plurality of detectors 60 and 62. The detection frequency is predicted by obtaining the detection frequency within a predetermined time between the common times for each.

Specifically, the detectors 60 and 62 that can commonly detect a certain pedestrian within a predetermined time between common times by using the sensor information of each of the detectors 60 and 62 acquired by the sensing result acquisition unit 14 Is specified, and the sum of the detection frequencies of each of the identified detectors is defined as the predicted detection frequency. In this case, a detector whose detection range is the same as the observed three-dimensional position y of the pedestrian may be specified. For example, as shown in FIG. 3, consider a situation in which a camera is used as a plurality of detectors 60 and 62 to detect a pedestrian. In the example shown in FIG. 3, as the detection performance, the undetected rate is 5% (0.05), the operating cycle of the camera 1 is 10 Hz, and the operating cycle of the camera 2 is 1 Hz for all the cameras. Then, when the camera 1 detects a pedestrian by one observation, it is predicted that the pedestrian is detected at a frequency of 9.5 times / second, and the camera 2 detects the pedestrian by one observation. It is predicted that pedestrians will be detected at a frequency of 0.95 times / second. In the observations of the two cameras 1 and 2, the detection timings are different, but the total detection frequency detected within the predetermined time between the common times, in this case, the pedestrian has a frequency of 10.45 times / second. Predict to be detected. In other words, when a pedestrian existing at a certain position is observed by a plurality of cameras, the pedestrian is detected by the total detection frequency of the detection frequencies of the multiple cameras within a predetermined time between common times. Will be done.

The moving object state quantity erasing unit 22 detects that the observed detection frequency of the pedestrian state quantities associated with the detection result by the moving object state quantity matching unit 20 is predicted by the moving object detection frequency predicting unit 21. The state quantity of the moving object having a detection frequency lower than the frequency is deleted from the moving object state quantity storage unit 16. The moving object state quantity erasing unit 22 sets a threshold value determined by the detection frequency predicted by the moving object detection frequency prediction unit 21 as a detection frequency that is unlikely to exist as a moving object. Therefore, in the moving object state quantity erasing unit 22, the detection frequency predicted by the moving object detection frequency prediction unit 21 is less than the threshold value among the pedestrian state quantities associated with the detection result by the moving object state quantity matching unit 20. The state quantity of a small moving object is erased from the moving object state quantity storage unit 16. For example, when the threshold value is set to, for example, 50% of the predicted detection frequency, in the example shown in FIG. 3, a pedestrian whose detection frequency observed by the camera 1 is 1 time / sec is the predicted detection. Eliminate the amount of pedestrian status that is less than the threshold value determined by the frequency of 9.5 times / sec. The threshold value may be appropriately set according to the sensor information including the detection performance information.

As a simple process, the moving object state quantity erasing unit 22 elapses for a certain period of time or more from the last observed time among the pedestrian state quantities that correspond to the detection result by the moving object state quantity matching unit 20. It is also possible to have a function of erasing the state amount of the pedestrian who is moving from the moving object state amount storage unit 16.

The pedestrian state quantity estimation device 10 outputs the set X of the state quantity updated by the above series of processes as the integrated result of the pedestrian detection result.

<Operation of pedestrian state quantity estimation system 100>
Next, the operation of the pedestrian state quantity estimation system 100 according to the first embodiment will be described. First, pedestrians are sequentially detected by each of the plurality of detectors 60 mounted on the plurality of vehicles and the detector 62 as an infrastructure sensor, and each time the pedestrian is detected, the detection result is transmitted via the base station 50. , The pedestrian state amount estimation processing routine shown in FIG. 4 is executed in the pedestrian state amount estimation device 10 while being transmitted to the pedestrian state amount estimation device 10.

In step S100, as a result of detecting the pedestrian from any of the plurality of detectors 60 and 62, a set of three-dimensional positions of the pedestrian Y _i = {y _{i, 1} , ... y _{i, m} }. Upon receiving the set _Ri = {ri _{, 1} , ..., Ri _{, m} } of the observation error variance matrix, the process proceeds to step S102.

In step S102, according to the time of the detection result received in step S100, for each of the plurality of pedestrians, the next time using the state amount stored in the moving object state amount storage unit 16 by the prediction step of the Kalman filter. The pedestrian state quantity is repeatedly predicted, and the set X = {x ₁ , ... x _n } of the pedestrian state quantity at the time of the detection result received in step S100, and the variance-covariance matrix V thereof. = {V ₁ , ..., v _n } is obtained.

In step S104, the state quantity predicted for each of the plurality of pedestrians in step S102 corresponding to the time of the detection result received in step S100, and the plurality of pedestrians represented by the detection result received in step S100, respectively. Corresponds to the three-dimensional position of.

In step S105, for each of the plurality of pedestrians in step S102 corresponding to the time of the detection result received in step S100, among the plurality of detectors, the detectors can be detected within a predetermined time between common times. Predict the detection frequency detected by.

In step S106, among the pedestrian state quantities that correspond to the detection results in step S104, the pedestrian state quantity whose observed detection frequency is smaller than the threshold value predicted in step S105 is the moving object state. Erase from the quantity storage unit 16.

In step S108, for each of the state quantities associated with the detection result in step S104, the error variance matrix ri _{, j} and the error variance matrix ri _{, j} are obtained in step S102, with the corresponding three-dimensional positions y _{i, j} as observed values. Using the state quantity of the pedestrian and the variance-covariance matrix, the state quantity is updated and the last observed time is updated by the filtering step of the Kalman filter. Further, among the detection results received in step S100, a new state quantity x is generated based on the detection result of the pedestrian that does not correspond to the state quantity, and the current time is set as the last observed time of the moving object. It is stored in the state quantity storage unit 16.

Then, in step S110, the updated state quantity set X is output as the integrated result of the pedestrian detection result, and the process returns to step S100.

As described above, according to the pedestrian state quantity estimation system according to the first embodiment, even if the number of sensors of the asynchronous plurality of sensors is indefinite, the results of the plurality of sensors are integrated in consideration of the detection frequency. , The state quantity of pedestrians can be obtained accurately.

In addition, when sensing results come from multiple unspecified sensors asynchronously, by deleting them in consideration of the detection frequency, the influence of unreliable detection results does not remain as a state quantity, so that each sensor is erroneous. Become robust to detection. In this way, when the sensing results from a plurality of sensors that come asynchronously are integrated in real time, it becomes robust to false detection of each sensor.

[Second Embodiment]
<System configuration of pedestrian state quantity estimation system>
Next, the pedestrian state quantity estimation system according to the second embodiment will be described. The parts having the same configuration as that of the first embodiment are designated by the same reference numerals and the description thereof will be omitted.

In the second embodiment, the existence probabilities of each of the plurality of pedestrians are held, and the existence probabilities of each of the plurality of pedestrians are updated based on the correspondence between the detection result and the state quantity, and the state quantity is determined. It differs from the first embodiment in that the existence probability that changes according to the predicted detection frequency is taken into consideration when erasing.

As shown in FIG. 5, the pedestrian state amount estimation device 210 according to the second embodiment includes a communication unit 12, a sensing result acquisition unit 14, a moving object state amount storage unit 16, and a moving object state amount prediction unit 18. , The moving object state quantity matching unit 20, the moving object detection frequency prediction unit 21, the moving object state quantity erasing unit 22, the moving object state quantity updating unit 24, the moving object existence probability updating unit 220, and the moving object existence. It includes a probability storage unit 222.

The moving object existence probability storage unit 222 stores the existence probabilities of each of the plurality of pedestrians stored in the moving object state quantity storage unit 16.

The moving object existence probability updating unit 220 is based on the result of associating with the detection result by the moving object state quantity matching unit 20, and among the detection results of the latest data, the detection result of the pedestrian who did not correspond to the state quantity. On the other hand, as the existence probability of the pedestrian, a set value is stored in the moving object existence probability storage unit 222. The set value may be set according to the detection accuracy predetermined for the detectors 60 and 62.

Further, the moving object existence probability updating unit 220 corresponds to the detection result of the latest data by the moving object state quantity matching unit 20 among the existence probabilities of each of the plurality of pedestrians stored in the moving object existence probability storage unit 222. For each of the attached pedestrian existence probabilities, the detectors 60 and 62 are updated so as to increase according to the detection frequency (see FIG. 6).

Further, the moving object existence probability updating unit 220 corresponds to the detection result of the latest data by the moving object state quantity matching unit 20 among the existence probabilities of each of the plurality of pedestrians stored in the moving object existence probability storage unit 222. Each of the pedestrians' existence probabilities that were not attached is attenuated and updated so that the existence probabilities correspond to the detection frequency (see FIG. 6).

In the moving object state quantity erasing unit 22, the detection frequency predicted by the moving object detection frequency predicting unit 21 is the first threshold value among the pedestrian state quantities associated with the detection result by the moving object state quantity matching unit 20. The state amount of the pedestrian whose existence probability stored in the moving object existence probability storage unit 222 corresponding to the state amount of the pedestrian smaller than the threshold value in the first embodiment is smaller than the second threshold value is erased. The second threshold value is a value corresponding to the existence probability of a pedestrian, which is predetermined as the probability of being unlikely to exist as a moving object.

Here, it will be further described that the moving object existence probability updating unit 220 updates the existence probability according to the detection frequency. Here, consider a case where a camera is used as a plurality of detectors 60 and 62 to detect a pedestrian.

When the elapsed time from the previous time is expressed by Δt and the decrease rate of the existence probability is expressed by ν for the state quantity of a certain moving object, the change amount Δp of the existence probability can be expressed by the following equation.
Δp = −νΔt

If the N cameras exist as a pedestrian state quantity estimating unit 210, i-th camera (i = 1,2, ···, N ) the operating frequency at time t of the _{f i,} the imaging range of the camera Let _Ri (t) be the field of view of the camera i in consideration of a predetermined area (for example, a blind spot due to a wall or the like) that is invisible due to the structure to be reduced. Further, the function σ _i (x, t) indicating whether or not a certain place x is included in the field of view of the camera i is expressed by the following equation.

Further, the undetected rate of the object by the camera i at the place x is expressed as _{pFN, i} (x, t). The undetected rate of moving objects may be derived from the distance from the detector, sunshine conditions, and the like. Then, when a moving object exists at the position x, the total detection frequency f (x, t) by the N cameras can be calculated by the following formula.

Therefore, it is possible to express the decrease amount ν _j (t) of the existence probability of the place x _j and the j-th moving object at time t by the following equation.
ν _j (t) = ν ₀ · f (x, t)
By doing so, it is possible to update the existence probability according to the detection frequency.

Further, instead of calculating the detection frequency of each sensor each time the existence probability of each moving object is updated, the detection frequency calculated for each detector is stored in advance, and a value corresponding to the position of the moving object is extracted. You may. By updating the information stored only in the detectors whose distribution of the detection frequency has changed, it becomes possible to efficiently calculate when the number of detectors and moving objects increases.

Since other configurations of the pedestrian state quantity estimation system according to the second embodiment are the same as those of the first embodiment, the description thereof will be omitted.

<Operation of pedestrian state quantity estimation system>
Next, the operation of the pedestrian state quantity estimation system according to the second embodiment will be described. First, pedestrians are sequentially detected by each of the plurality of detectors 60 mounted on the plurality of vehicles and the detector 62 as an infrastructure sensor, and the detection result is estimated by the pedestrian state quantity via the base station 50. The pedestrian state quantity estimation processing routine shown in FIG. 7 is executed in the pedestrian state quantity estimation device 210 while being transmitted to the device 10. The same processing as that of the first embodiment is designated by the same reference numerals and detailed description thereof will be omitted.

In step S100, as a result of detecting the pedestrian from any of the plurality of detectors 60 and 62, a set of three-dimensional positions of the pedestrian Y _i = {y _{i, 1} , ... y _{i, m} }. Upon receiving the set _Ri = {ri _{, 1} , ..., ri _{, m} } of the observation error variance matrix, the process proceeds to step S200.

In step S102, according to the time of the detection result received in step S100, for each of the plurality of pedestrians, the next time using the state amount stored in the moving object state amount storage unit 16 by the prediction step of the Kalman filter. The pedestrian state quantity is repeatedly predicted, and the set X = {x ₁ , ... x _n } of the pedestrian state quantity at the time of the detection result received in step S100, and the variance-covariance matrix V thereof. = {V ₁ , ..., v _n } is obtained.

In step S104, the state quantity predicted for each of the plurality of pedestrians in step S102 corresponding to the time of the detection result received in step S100, and the plurality of pedestrians represented by the detection result received in step S100, respectively. Corresponds to the three-dimensional position of.

In step S105, for each of the plurality of pedestrians in step S102 corresponding to the time of the detection result received in step S100, among the plurality of detectors, the detectors can be detected within a predetermined time between common times. Predict the detection frequency detected by.

In step S106, among the pedestrian state quantities that correspond to the detection results in step S104, the pedestrian state quantity whose observed detection frequency is smaller than the threshold value predicted in step S105 is the moving object state. Erase from the quantity storage unit 16.

In step S202, among the existence probabilities of each of the plurality of pedestrians stored in the moving object existence probability storage unit 222, each of the pedestrian existence probabilities associated with the detection result of the latest data in step S104 is , The detectors 60 and 62 are updated so as to increase according to the detection frequency predicted in step S105. Further, among the existence probabilities of each of the plurality of pedestrians stored in the moving object existence probability storage unit 222, each of the existence probabilities of pedestrians not associated with the detection result of the latest data in step S104 is described. It is attenuated and updated so that the existence probability corresponds to the detection frequency predicted in step S105.

In step S204, among the pedestrian state quantities associated with the detection result in step S104, the pedestrian state quantity whose existence probability stored in the moving object existence probability storage unit 222 is smaller than the second threshold value is deleted. To do.

In step S108, for each of the state quantities associated with the detection result in step S104, the error variance matrix ri _{, j} and the error variance matrix ri _{, j} are obtained in step S200, with the corresponding three-dimensional positions y _{i, j} as observed values. Using the state quantity of the pedestrian and the variance-covariance matrix, the state quantity is updated and the last observed time is updated by the filtering step of the Kalman filter. Further, among the detection results received in step S100, a new state quantity x is generated based on the detection result of the pedestrian that does not correspond to the state quantity, and the current time is set as the last observed time of the moving object. It is stored in the state quantity storage unit 16. Further, a predetermined existence probability is stored in the moving object existence probability storage unit 222.

Then, in step S110, the updated state quantity set X is output as the integrated result of the pedestrian detection result, and the process returns to step S100.

As described above, according to the pedestrian state quantity estimation system according to the second embodiment, when sensing results come from a plurality of unspecified sensors asynchronously, the detected state quantity is determined by the detection frequency. By deleting based on the above, the influence of the unreliable detection result does not remain as a state quantity, so that the false detection of each sensor becomes robust.

[Third Embodiment]
<System configuration of pedestrian state quantity estimation system>
Next, the pedestrian state quantity estimation system according to the third embodiment will be described. The parts having the same configurations as those of the first embodiment and the second embodiment are designated by the same reference numerals and the description thereof will be omitted.

In the third embodiment, when updating the existence probability of a pedestrian from the result of predicting the detection frequency of a moving object, the latest sensor information of each of the detectors 60 and 62 stored in advance, and a structure such as a wall It is different from the second embodiment in that the detection frequency of moving objects is predicted by using a map showing the position of.

As shown in FIG. 8, the moving object detection frequency prediction unit 321 in the pedestrian state quantity estimation device according to the third embodiment includes a sensor log storage unit 322, a map 323, and a detection frequency calculation unit 324. ..

The sensor log storage unit 322 stores the latest sensor information of each of the detectors 60 and 62. Further, the map 323 stores map information indicating the position of a structure such as a wall that reduces the detectable range of the detectors 60 and 62.

In the detection frequency calculation unit 324, the sensor log storage unit 322 displays the latest sensor information of the detectors 60 and 62, respectively, and the map 323 indicates the position of a structure such as a wall that reduces the detectable range of the detectors 60 and 62. The detection frequency is calculated using the map information shown.

Specifically, for example, when a camera is used as a plurality of detectors 60 and 62 to detect a pedestrian, the detection frequency calculation unit 324 sets the detection position (including the detection direction) and the detection range of each camera. , The blind spot or undetectable area (area undetectable by the detector due to the structure) caused by the structure such as the wall is obtained from the position of the structure such as the wall. The area excluding the undetectable area is defined as the field of view _Ri (t) of the camera i, and a function σ _i (x, t) indicating whether or not a certain place x is included in the field of view of the camera i is defined.

Further, the total detection frequency f (x, t) by N cameras is calculated by using the latest undetected rate p _{FN, i} (x, t) based on the sensor information. Therefore, it is possible to update the existence probability according to the detection frequency by obtaining the decrease amount ν _j (t) of the existence probability of the place x _j and the jth moving object at the time t by the above equation. ..

Since other configurations and operations of the pedestrian state quantity estimation system according to the third embodiment are the same as those of the second embodiment, the description thereof will be omitted.

In this way, by updating the existence probability of the pedestrian by using the range that cannot be detected by the detector determined by the map as a constraint, the state quantity of the non-existent pedestrian can be appropriately erased.

[Fourth Embodiment]
<System configuration of pedestrian state quantity estimation system>
Next, the pedestrian state quantity estimation system according to the fourth embodiment will be described. The parts having the same configurations as those of the first embodiment and the second embodiment are designated by the same reference numerals and the description thereof will be omitted.

The fourth embodiment is different from the second embodiment in that the detected frequency of moving objects predicted for each position is stored and the detected frequency of moving objects is predicted using the stored detection frequency. There is.

As shown in FIG. 9, the moving object detection frequency prediction unit 421 in the pedestrian state quantity estimation device according to the fourth embodiment includes a frequency map update unit 422, a map 323, a frequency map 423, and a detection frequency calculation unit 424. And have.

The frequency map 423 stores the detection frequency for each position in the detectable range of the pedestrian by using the map information stored in the map 323 and the latest sensor information of each of the detectors 60 and 62. Specifically, for example, when a plurality of detectors 60 and 62 are two cameras 1 and 2, as shown in FIG. 10, detection by sensor information is performed at each camera position of the camera 1 and the camera 2. It is within the possible range. Further, it is assumed that the wall exists in the detectable range based on the map information. In this case, the entire area where pedestrians may exist is an area where pedestrians cannot be detected by cameras 1 and 2 (number of cameras Nc = 0), and pedestrians can be detected only by camera 1 or camera 2. Areas (number of cameras Nc = 1) and areas where pedestrians can be detected by both cameras 1 and 2 (number of cameras Nc = 2). Therefore, the number of detectable cameras differs depending on the position of the pedestrian, and the detection frequency changes according to the number of detectable cameras. Therefore, as shown in FIG. 10, the area showing the number of cameras for each position is used as the frequency map. In this frequency map, the detection frequency of each camera can be calculated from the operation cycle of the cameras corresponding to each area, and the detection frequency of the corresponding area can be derived by adding the number of cameras. Therefore, the frequency map update unit 422 updates the frequency map to the latest state by using the sensor information and the map information stored in the map 323.

The detection frequency calculation unit 424 calculates and outputs the detection frequency by extracting the detection frequency of the position indicated by the state quantity of the pedestrian from the detection frequencies stored in the frequency map 423.

Since other configurations and operations of the pedestrian state quantity estimation system according to the fourth embodiment are the same as those of the second embodiment, the description thereof will be omitted.

In this way, by updating the frequency map from the sensor information using the range that cannot be detected by the detector determined by the map as a constraint, the detection frequency corresponding to the position according to the state quantity of the pedestrian can be quickly obtained. Is possible, and the state quantity of a non-existent pedestrian can be appropriately erased.

In the above embodiment, the case where a pedestrian is targeted as a moving object to be detected has been described as an example, but the present invention is not limited to this, and other moving objects such as vehicles are to be detected. May be.

Further, the case where a plurality of detectors mounted on a plurality of vehicles and a detector as an infrastructure sensor are used as a plurality of asynchronous sensors has been described as an example, but the present invention is not limited to this, and detection as an infrastructure sensor is performed. It may be configured not to use a vessel.

Further, a plurality of detectors mounted on a single vehicle and using a plurality of measuring instruments including a camera and a radar may be used as a plurality of asynchronous sensors. For example, when a detector using a different measuring instrument is added to the vehicle as a retrofit, it is assumed that the number of a plurality of sensors is indefinite.

The program of the present disclosure can be provided by storing it in a recording medium.

10, 210 Pedestrian state quantity estimation device 12 Communication unit 14 Sensing result acquisition unit 16 Moving object state quantity storage unit 18 Moving object state quantity prediction unit 20 Moving object state quantity matching unit 21 Moving object detection frequency prediction unit 22 Moving object state quantity Erasing unit 24 Moving object state quantity updating unit 50 Base station 60, 62 Detector 70 Network 100 Pedestrian state quantity estimating system 210 Pedestrian state quantity estimating device 220 Moving object existence probability updating unit 222 Moving object existence probability storage unit 321 Moving object Detection frequency prediction unit 322 Sensor log storage unit 323 Map 324 Detection frequency calculation unit 421 Moving object detection frequency prediction unit 422 Frequency map update unit 423 Frequency map 424 Detection frequency calculation unit

Claims (7)

A moving object state quantity storage unit that stores the state quantity and the last observed time of each of the plurality of moving objects,
For each of the plurality of moving objects, a moving object state quantity predicting unit that predicts the state quantity of the moving object at the next time using the state quantity stored in the moving object state quantity storage unit.
A detection result that detects a plurality of moving objects from any of a plurality of sensors having an indefinite number, and a sensing result acquisition unit that receives detection performance information.
Each time the detection result is received by the sensing result acquisition unit, the state amount of the moving object predicted by the moving object state amount prediction unit for each of the plurality of moving objects corresponding to the time of the detection result, and a plurality of states. The moving object state quantity that is associated with the detection result of each moving object and updates the state quantity and the last observed time of each of the plurality of moving objects stored in the moving object state quantity storage unit. Matching part and
A moving object detection frequency prediction unit that predicts the detection frequency detected by a detectable sensor among the plurality of sensors for each of the plurality of moving objects based on the detection performance information of each of the plurality of sensors.
The moving object detection frequency predicting unit predicts the detection frequency of the moving object state quantity that does not correspond to the detection result by the moving object state quantity matching unit and is determined by the detection results of detecting a plurality of moving objects. A moving object state quantity erasing unit that erases the state quantity of the moving object that is less than the detection frequency from the moving object state quantity storage unit.
A moving object state quantity estimation device including. The moving object state quantity estimation device according to claim 1, wherein the plurality of sensors include sensors mounted on different moving bodies. A moving object existence probability storage unit that stores the existence probabilities of each of the plurality of moving objects,
The existence probability of the moving object is set in the moving object existence probability storage unit according to the result of association with the detection result by the moving object state quantity matching unit or the detection accuracy predetermined for the sensor. Or, as a result of associating the existence probabilities of each of the plurality of moving objects stored in the moving object existence probability storage unit with the detection result by the moving object state quantity matching unit, from the last observed time. The elapsed time of the sensor, the detection accuracy predetermined for the sensor, or the moving object existence probability updating unit that increases or decreases according to the detection frequency by the moving object detection frequency prediction unit.
Including
The moving object state quantity erasing unit obtains the state quantity of the moving object from the moving object state quantity storage unit based on the existence probability of each of the plurality of moving objects stored in the moving object existence probability storage unit. The moving object state quantity estimation device according to claim 1 or 2, which is to be erased. The moving object detection frequency prediction unit
A detection performance information storage unit that stores the detection performance information of each of the plurality of sensors,
It was updated by the detection performance information of each of the plurality of sensors stored in the detection performance information storage unit, the moving object detectable range information in which the detectable range of the moving object is recorded, and the moving object state quantity matching unit. A detection frequency calculation unit that calculates the detection frequency based on the state quantity of the moving object,
The moving object state quantity estimation device according to claim 3. The moving object detection frequency prediction unit
A detection frequency storage unit that stores information indicating the detection frequency for each detection position of a moving object predicted by the moving object detection frequency prediction unit, and a detection frequency storage unit.
The detection position of the moving object stored in the detection frequency storage unit based on the detection performance information received by the sensing result acquisition unit and the moving object detectable range information in which the detectable range of the moving object is recorded. A detection frequency update unit that updates the detection frequency for each
A detection frequency extraction unit that extracts the detection frequency corresponding to the state quantity of the moving object from the detection frequency storage unit,
The moving object state quantity estimation device according to claim 3. The plurality of sensors are a plurality of detectors mounted on a plurality of vehicles or used as an infrastructure sensor to detect a pedestrian or a vehicle.
The state quantity of the moving object is the position and speed of a pedestrian or a vehicle.
The moving object state quantity estimation device according to claim 4 or 5, wherein the moving object detectable range information is a map showing the position of a structure that reduces the sensor detectable range. A computer including a moving object state quantity storage unit that stores the state quantity and the last observed time of each of the plurality of moving objects.
For each of the plurality of moving objects, a moving object state quantity predicting unit that predicts the state quantity of the moving object at the next time using the state quantity stored in the moving object state quantity storage unit.
Sensing result acquisition unit that receives detection results of detecting multiple moving objects from any of multiple sensors of indefinite number.
Each time the detection result is received by the sensing result acquisition unit, the state amount of the moving object predicted by the moving object state amount prediction unit for each of the plurality of moving objects corresponding to the time of the detection result, and a plurality of states. The moving object state quantity that is associated with the detection result of each moving object and updates the state quantity and the last observed time of each of the plurality of moving objects stored in the moving object state quantity storage unit. Matching part,
A moving object detection frequency prediction unit that predicts the detection frequency detected by a detectable sensor among the plurality of sensors for each of the plurality of moving objects based on the detection performance information of each of the plurality of sensors, and the moving object state. The state quantity of the moving object that does not correspond to the detection result by the quantity matching unit, and the detection frequency determined by the detection results of detecting a plurality of moving objects is higher than the detection frequency predicted by the moving object detection frequency prediction unit. A moving object state quantity erasing unit that erases a small state quantity of the moving object from the moving object state quantity storage unit,
A program to function as.