JP7207050B2 - Moving object state quantity estimation device and program - Google Patents

Description

The present invention relates to a moving object state quantity estimation device and program.

Conventionally, in order to capture the movement of pedestrians with multiple sensors, each of which is asynchronous and the number of sensors is indefinite, each time a detection result from a specific sensor is obtained, the movement that indicates the detection result, the pedestrian's position and speed, etc. is obtained. There is known a technique of associating the state quantity of an object, updating the state quantity and the last observed time, and erasing the state quantity of a moving object that has not been associated with the detection result for a certain period of time or more (Patent Reference 1).

Also, when an object is detected by integrating the detection results of each camera using multiple camera images, the detection range in the image is limited in advance based on the positional relationship between the object and each camera. There is known an object tracking device capable of improving tracking accuracy by suppressing erroneous detection of (Patent Document 2).

Japanese Unexamined Patent Application Publication No. 2018-92368 JP-A-2016-71830

In the technique disclosed in Patent Literature 1, erroneous detection is suppressed by erasing the state quantity of a moving object when a certain period of time has elapsed since the object was last observed.
However, for example, when the observation time interval changes according to the sensor performance or the detection environment such as the brightness around the moving object changes, the detection frequency of erroneous detection changes. In particular, when the frequency of erroneous detection increases in a time shorter than a certain period of time, it is difficult to distinguish between correct detection and erroneous detection, and there is room for improvement in suppressing erroneous detection.

Further, in Patent Document 2, the detection range is limited in advance based on the positional relationship between the object and the camera to suppress erroneous detection. Even if the detection range is limited based on the positional relationship between the object and the camera, it may be difficult to suppress erroneous detection.

The present invention has been made in view of the above circumstances, and even if the number of asynchronous multiple sensors is not fixed, the results of the multiple sensors are integrated in consideration of the detection frequency to obtain the state quantity of the moving object. It is an object of the present invention to provide a moving object state quantity estimating device and a program capable of accurately obtaining the moving object state quantity.

In order to achieve the above object, a moving object state quantity estimation apparatus according to a first aspect includes a moving object state quantity storage unit that stores the state quantity and the last observation time of each of a plurality of moving objects; For each moving object, either a moving object state quantity prediction unit for predicting the state quantity of the moving object at the next time using the state quantity stored in the moving object state quantity storage unit, or an indefinite number of sensors. a sensing result acquisition unit that receives a detection result of detecting a plurality of moving objects and detection performance information from the moving object, each time the detection result is received by the sensing result acquisition unit, the moving object corresponding to the time of the detection result The state quantity of the moving object predicted for each of the plurality of moving objects by the state quantity prediction unit is associated with the detection result of each of the plurality of moving objects, and stored in the moving object state quantity storage unit. , a moving object state quantity matching unit that updates the state quantity and the last observed time of each of the plurality of moving objects; A moving object detection frequency prediction unit that predicts a detection frequency detected by a detectable sensor among sensors, and a state quantity of the moving object that is not associated with the detection result by the moving object state quantity matching unit, The state quantity of the moving object whose detection frequency determined by the detection result of detecting a plurality of moving objects is smaller than the detection frequency predicted by the moving object detection frequency prediction unit is deleted from the moving object state quantity storage unit. and a moving object state quantity estimating device.

A second aspect is the moving object state quantity estimation apparatus according to the first aspect, wherein the plurality of sensors include sensors mounted on different moving bodies.

A third aspect is the moving object state quantity estimation device according to the first aspect or the second aspect, wherein the moving object existence probability storage unit stores the existence probability of each of the plurality of moving objects, and the moving object state quantity matching unit. setting the existence probability of the moving object in the moving object existence probability storage unit according to the result of association with the detection result by or the detection accuracy predetermined for the sensor, or the moving object existence probability As a result of associating the existence probability of each of the plurality of moving objects stored in the storage unit with the detection result by the moving object state quantity matching unit, the elapsed time from the last observation time, or the sensor a moving object existence probability updating unit that increases or decreases according to a predetermined detection accuracy or the detection frequency by the moving object detection frequency prediction unit, and the moving object state quantity erasing unit updates the moving object existence probability Based on the existence probability of each of the plurality of moving objects stored in the storage unit, the state quantity of the moving object is erased from the moving object state quantity storage unit.

A fourth aspect is the moving object state quantity estimation device according to claim 3, wherein the moving object detection frequency prediction unit includes a detection performance information storage unit that stores the detection performance information of each of the plurality of sensors, and the detection performance The detection performance information of each of the plurality of sensors stored in the information storage unit, the moving object detectable range information in which the detectable range of the moving object is recorded, and the moving object updated by the moving object state quantity matching unit and a detection frequency calculation unit that calculates the detection frequency based on the state quantity.

A fifth aspect is the moving object state quantity estimation device according to claim 3, wherein the moving object detection frequency prediction unit indicates the detection frequency for each detection position of the moving object predicted by the moving object detection frequency prediction unit. a detection frequency storage unit that stores information; and based on the detection performance information received by the sensing result acquisition unit and the moving object detectable range information in which the detectable range of the moving object is recorded, the detection frequency storage unit a detection frequency update unit for updating the detection frequency for each detection position of the moving object stored in the detection frequency storage unit; a detection frequency extraction unit for extracting the detection frequency corresponding to the state quantity of the moving object from the detection frequency storage unit; including.

A sixth aspect is the moving object state quantity estimation device according to the fourth aspect or the fifth aspect, wherein the plurality of sensors detect pedestrians or vehicles mounted on a plurality of vehicles or used as infrastructure sensors. wherein the moving object state quantity is the position and speed of a pedestrian or vehicle, and the moving object detectable range information is the position of a structure that reduces the sensor detectable range. It is a map showing.

In a seventh aspect, a computer including a moving object state quantity storage unit for storing the state quantity and the last observed time of each of a plurality of moving objects is stored in the moving object state quantity storage unit for each of the plurality of moving objects. a moving object state quantity prediction unit that predicts the state quantity of the moving object at the next time using the stored state quantity; a state of the moving object predicted for each of the plurality of moving objects by the moving object state quantity prediction unit corresponding to the time of the detection result each time the result acquisition unit receives the detection result by the sensing result acquisition unit; Associate the amount with the detection result of each of the plurality of moving objects, and update the state quantity and the last observed time of each of the plurality of moving objects stored in the moving object state quantity storage unit. a moving object state quantity matching unit for predicting a frequency of detection of each of said plurality of moving objects by sensors detectable among said plurality of sensors based on detection performance information of each of said plurality of sensors; and said moving object state quantity matching unit, said moving object state quantity not associated with said detection result, said moving object detection frequency predicting unit estimating a detection frequency determined by a detection result of detecting a plurality of moving objects. a moving object state quantity erasing unit for erasing from the moving object state quantity storage unit the state quantity of the moving object whose detection frequency is smaller than that predicted by the moving object state quantity erasing unit.

Also, the program of the present disclosure can be stored in a recording medium and provided.

As described above, according to the present disclosure, even if the number of asynchronous multiple sensors is indefinite, the results of the multiple sensors are integrated in consideration of the detection frequency, and the state quantity of the moving object is obtained with high accuracy. The effect of being able to

BRIEF DESCRIPTION OF THE DRAWINGS It is a block diagram which shows the pedestrian state quantity estimation system which concerns on 1st Embodiment. BRIEF DESCRIPTION OF THE DRAWINGS It is a block diagram which shows the pedestrian state quantity estimation apparatus which concerns on 1st Embodiment. FIG. 10 is an image diagram of sensing result integration based on detection frequency using detection results from asynchronous multiple sensors; It is a flowchart which shows the content of the pedestrian state quantity estimation processing routine in the pedestrian state quantity estimation apparatus which concerns on 1st Embodiment. It is a block diagram which shows the pedestrian state quantity estimation apparatus which concerns on 2nd Embodiment. FIG. 10 is an image diagram of sensing result integration based on detection frequency using detection results from asynchronous multiple sensors; It is a flowchart which shows the content of the pedestrian state quantity estimation processing routine in the pedestrian state quantity estimation apparatus which concerns on 2nd Embodiment. It is a block diagram which shows the moving object detection frequency prediction part in the pedestrian state quantity estimation apparatus which concerns on 3rd Embodiment. It is a block diagram which shows the moving object detection frequency prediction part in the pedestrian state quantity estimation apparatus which concerns on 4th Embodiment. FIG. 4 is an image diagram showing a frequency map;

Hereinafter, embodiments for implementing the technology of the present disclosure will be described in detail with reference to the drawings.

<Overview of Embodiment>
Embodiments implementing the techniques of this disclosure take the approach of not synchronizing frames between multiple sensors. At the timing when the sensing result from a certain sensor is received, the state quantity (hypothesis) of the pedestrian's position is associated with the received sensing result, the error is calculated, and the pedestrian's position is corrected while correcting the hypothetical position of the pedestrian. is detected and tracked. This processing is performed each time a sensing result from each sensor reaches the integrated processing device.

However, in this approach, the above process is performed sequentially using only the results of one sensor. Then, when the frequency of outputting results is high), the number of non-existent pedestrian hypotheses increases due to false detection, and the number of pedestrians may be erroneously estimated.

For this reason, the detection frequency of each sensor is considered, and pedestrian detection results that are highly likely to be falsely detected are discarded.

[First embodiment]

In the first embodiment, the pedestrian state quantity estimation device for estimating the state quantity of the pedestrian by integrating the detection results of the pedestrian detector mounted on the vehicle and the pedestrian detector as an infrastructure sensor. A case where the technique of (1) is applied will be described as an example.

<System configuration of pedestrian state quantity estimation system>
As shown in FIG. 1, the pedestrian state quantity estimation system 100 according to the first embodiment includes a pedestrian state quantity estimation device 10, a base station 50, a plurality of detectors 60 mounted on a plurality of vehicles, The base station 50 and the pedestrian state quantity estimation device 10 are connected by a network 70 such as the Internet, and the base station 50 and the detectors 60 and 62 are connected by wireless communication. connected by

Detectors 60 and 62 use cameras and radar to detect pedestrians at any time, and transmit the detection results to pedestrian state quantity estimation device 10 via base station 50 each time they are detected.

For example, the detector 60 extracts an image feature amount (SIFT, FIND, HOG, etc.) for each sliding window from a road image in front captured by a camera that captures an image of the front of the vehicle, and extracts an image for each sliding window. Pedestrians are detected using a feature amount and a pedestrian detection model (SVM, AdaBoost), and image coordinates representing the positions of the detected pedestrians are obtained. The detector 60 also transforms the image coordinates representing the pedestrian position into a three-dimensional position. At this time, an error distribution matrix is obtained according to the detected height of the pedestrian. Further, the detector 60 obtains the absolute three-dimensional position of the pedestrian based on the absolute coordinates of the own vehicle measured by the GPS mounted on the own vehicle and the obtained three-dimensional position.

The three-dimensional position of the pedestrian and the error variance matrix are obtained for each detected pedestrian, and the set Y _{i of the three-dimensional position of the pedestrian detected by the detector 60 of the vehicle C i =} {y _i,1 , y i _{, m} } and a set of observation error variance matrices R _i ={r _i,1 , . In addition, the detector 60 can transmit sensor information including detection performance information. An example of the detection performance information of the sensor information includes information indicating the detection time, the detection range of the detector 60 (including the position of the detector at the detection time), the operation period, and the non-detection rate of moving objects.

Detector 62, like detector 60, transmits a set of detected three-dimensional positions of pedestrians and a set of observation error variance matrices to pedestrian state quantity estimation device 10 for each observation.

A plurality of detectors 60, 62 asynchronously detect pedestrians. In addition, since the plurality of detectors 60 are mounted on different vehicles, the number of detectors 60 and 62 that transmit detection results to the pedestrian state quantity estimation device 10 is indefinite.

The pedestrian state quantity estimation device 10 is configured by, for example, a server, and the pedestrian state quantity estimation device 10 includes a CPU, a RAM, and a ROM storing a program for executing a pedestrian state quantity estimation processing routine described later. and is functionally configured as follows. As shown in FIG. 2, the pedestrian state quantity estimation device 10 includes a communication unit 12, a sensing result acquisition unit 14, a moving object state quantity storage unit 16, a moving object state quantity prediction unit 18, and a moving object state quantity A matching unit 20 , a moving object detection frequency prediction unit 21 , a moving object state quantity erasing unit 22 , and a moving object state quantity updating unit 24 are provided.

Each time the communication unit 12 receives a set of pedestrian coordinates and a set of observation error variance matrices transmitted from one of the detectors 60 and 62, the sensing result acquisition unit 14 acquires a set of pedestrian coordinates and Get a set of observed error variance matrices. Also, the sensing result acquisition unit 14 can acquire sensor information of the detectors 60 and 62 .

The moving object state quantity storage unit 16 stores the state quantities (positions and velocities of the pedestrians) updated by the moving object state quantity updating unit 24 and the last observed time for each of the plurality of observed pedestrians. .

The moving object state quantity prediction unit 18 performs the prediction step of the Kalman filter for each of the plurality of pedestrians in accordance with the time of the latest data acquired by the sensing result acquisition unit 14. The moving object state quantity stored in the moving object state quantity storage unit 16 Predicting the state quantity of the pedestrian at the next time using the state quantity is repeated, and a set of state quantities of the pedestrian at the time of the latest data X = {x ₁ , ... x _n } and state quantity , the variance-covariance matrix V={v ₁ , . . . , v _n }. For example, the state quantity of the pedestrian at the next time is predicted by constant velocity prediction or the like.

Each time the moving object state quantity matching unit 20 receives the latest data from the sensing result acquisition unit 14, the moving object state quantity prediction unit 18 predicts each of the plurality of pedestrians corresponding to the time of the latest data. The state quantity is associated with the detection results of each of the plurality of pedestrians represented by the latest data.

Specifically, a set X _{= {} x ₁ , . y _i,1 , . . . y _i,m }. For example, the correspondence that maximizes the product of the probability p(y _i,j |x _k ) of the combination of the associated state quantity x _k and the detected three-dimensional position y _i,j of the pedestrian is defined as Hungarian Calculation at high speed by methods such as

Note that the probability p(y _i,j |x _k ) of the combination of the state quantity x _k and the detected three-dimensional position y _i,j of the pedestrian is centered on the state quantity x _k and the variance of the corresponding Kalman filter _{Probability _ _ _} It suffices to find p(y _i,j |x _k ).

Also, a probability of not being associated is given as a set value, and combinations with probabilities smaller than this set value are not associated.

The moving object state quantity updating unit 24 uses the corresponding three-dimensional position _yi,j as an observed value for each state quantity associated with the latest data detection result by the moving object state quantity matching unit 20, and calculates the error Using the variance matrix r _i,j , the state quantity of the pedestrian and the covariance matrix obtained in the prediction step, the filtering step of the Kalman filter updates the state quantity and the last observed time.

In addition, based on the result of association with the detection result by the moving object state quantity matching unit 20, the moving object state quantity updating unit 24 updates pedestrians who are not associated with the state quantity among the detection results of the latest data. A new state quantity x is generated from the detection result, and the current time is stored in the moving object state quantity storage unit 16 as the last observation time. It is assumed that the new state quantity x has the same coordinates and error distribution matrix as the observed three-dimensional position y of the pedestrian.

The moving object detection frequency prediction unit 21 predicts the detection frequency of each of the plurality of pedestrians detected by the detectable detectors among the plurality of detectors 60 and 62 within a predetermined period of time between common times. That is, using the detection time, the detection range (including the position of the detector at the detection time), the operation period, and the non-detection rate of the moving object, which are the sensor information of the plurality of detectors 60 and 62, a plurality of pedestrians The detection frequency is predicted by calculating the detection frequency within a predetermined time between common times.

Specifically, using the sensor information of the detectors 60 and 62 acquired by the sensing result acquisition unit 14, the detectors 60 and 62 that can commonly detect a certain pedestrian within a predetermined time between common times. is specified, and the sum of the detection frequencies of the specified detectors is set as the predicted detection frequency. In this case, a detector whose detection range is the same coordinates as the observed three-dimensional position y of the pedestrian may be specified. For example, as shown in FIG. 3, consider a situation in which cameras are used as detectors 60, 62 to detect pedestrians. In the example shown in FIG. 3, for all cameras, the non-detection rate is 5% (0.05), the operating cycle of camera 1 is 10 Hz, and the operating cycle of camera 2 is 1 Hz. Then, when a pedestrian is detected by one observation by camera 1, it is predicted that the pedestrian is detected at a frequency of 9.5 times/second, and when a pedestrian is detected by camera 2 by one observation , predicts that pedestrians will be detected at a frequency of 0.95 times/second. In the observation of the two cameras 1 and 2, the detection timing is different, but the total detection frequency detected within a predetermined time between common times, in this case, the pedestrian is detected at a frequency of 10.45 times / second. Expect to be detected. In other words, when a pedestrian present at a certain position is observed by a plurality of cameras, the pedestrian is detected at a detection frequency that is the sum of the detection frequencies of each of the plurality of cameras within a predetermined time between common times. be done.

The moving object state quantity erasing unit 22 removes the detected detection frequency predicted by the moving object detection frequency predicting unit 21 from the pedestrian state quantities not associated with the detection result by the moving object state quantity matching unit 20 . The state quantities of moving objects whose detection frequency is smaller than the frequency are erased from the moving object state quantity storage unit 16 . The moving object state quantity erasing unit 22 sets a threshold determined by the detection frequency predicted by the moving object detection frequency prediction unit 21 as a detection frequency with a low possibility of existing as a moving object. Therefore, the moving object state quantity erasing unit 22 determines that the detection frequency predicted by the moving object detection frequency prediction unit 21 among the pedestrian state quantities not associated with the detection result by the moving object state quantity matching unit 20 is less than the threshold. The state quantity of the small moving object is erased from the moving object state quantity storage unit 16 . For example, when the threshold is set to a detection frequency of, for example, 50% of the predicted detection frequency, in the example shown in FIG. Pedestrian state quantities whose frequency is smaller than a threshold determined by 9.5 times/sec are deleted. Note that the threshold may be appropriately set according to sensor information including detection performance information.

As a simple process, the moving object state quantity erasing unit 22 removes the state quantity of the pedestrian that has not been associated with the detection result by the moving object state quantity matching unit 20, and removes the state quantity of the pedestrian that has not been associated with the detection result by the moving object state quantity matching unit 20. It is also possible to have a function of erasing the state quantity of the pedestrian that is moving from the moving object state quantity storage unit 16 .

The pedestrian state quantity estimating device 10 outputs the state quantity set X updated by the above-described series of processes as a result of integrating the pedestrian detection results.

<Operation of pedestrian state quantity estimation system 100>
Next, the operation of the pedestrian state quantity estimation system 100 according to the first embodiment will be described. First, pedestrians are sequentially detected by each of a plurality of detectors 60 mounted on a plurality of vehicles and a detector 62 as an infrastructure sensor. , is transmitted to the pedestrian state quantity estimation device 10, the pedestrian state quantity estimation processing routine shown in FIG.

In step S100, a set Y _i ={y _{i ,1} , . Upon receiving the set of observation error variance matrices R _i ={r _{i ,1} , .

In step S102, in accordance with the time of the detection result received in step S100, the prediction step of the Kalman filter is performed for each of the plurality of pedestrians, using the state quantity stored in the moving object state quantity storage unit 16 for the next time. , and a set of pedestrian state quantities X={x ₁ , . . . x _n } and its variance-covariance matrix V = {v ₁ ,..., v _n }.

In step S104, the state quantity predicted for each of the plurality of pedestrians in step S102 corresponding to the time of the detection result received in step S100, and the plurality of pedestrians represented by the detection result received in step S100 are calculated. is associated with the three-dimensional position of .

In step S105, for each of the plurality of pedestrians in step S102 corresponding to the time of the detection result received in step S100, within a predetermined time between common times, a detector that can be detected among a plurality of detectors Predict the frequency of detection detected by

In step S106, among the state quantities of pedestrians that are not associated with the detection result in step S104, state quantities of pedestrians whose observed detection frequency is smaller than a threshold value predicted in step S105 are treated as moving object state quantities. Erased from the amount storage unit 16 .

In step S108, for each state quantity associated with the detection result in step S104, the corresponding three-dimensional position y _i,j is set as an observed value, and the error variance matrix r _i,j obtained in step S102 is Using the state quantity of the pedestrian and the variance-covariance matrix, the state quantity and the last observed time are updated by the filtering step of the Kalman filter. Further, among the detection results received in step S100, a new state quantity x is generated based on the detection result of the pedestrian that is not associated with the state quantity, and the current time is used as the time of the last observation of the moving object. Stored in the state quantity storage unit 16 .

Then, in step S110, the updated state quantity set X is output as a result of integration of the pedestrian detection results, and the process returns to step S100.

As described above, according to the pedestrian state quantity estimation system according to the first embodiment, even if the number of asynchronous multiple sensors is indefinite, the results of the multiple sensors are integrated in consideration of the detection frequency. , the state quantity of the pedestrian can be obtained with high accuracy.

In addition, when sensing results are received asynchronously from unspecified multiple sensors, it is possible to eliminate the effects of unreliable detection results by taking into consideration the detection frequency, thereby preventing the effects of unreliable detection results from remaining as state quantities. Be robust to detection. In this way, when the sensing results from a plurality of sensors coming asynchronously are integrated in real time, it becomes robust against erroneous detection of each sensor.

[Second embodiment]
<System configuration of pedestrian state quantity estimation system>
Next, a pedestrian state quantity estimation system according to the second embodiment will be described. Parts having the same configuration as in the first embodiment are denoted by the same reference numerals, and descriptions thereof are omitted.

In the second embodiment, the presence probability of each of a plurality of pedestrians is held, and based on the correspondence between the detection result and the state quantity, the presence probability of each of the plurality of pedestrians is updated. The difference from the first embodiment is that when erasing, the presence probability that changes according to the predicted detection frequency is considered.

As shown in FIG. 5, the pedestrian state quantity estimation device 210 according to the second embodiment includes a communication unit 12, a sensing result acquisition unit 14, a moving object state quantity storage unit 16, and a moving object state quantity prediction unit 18. , a moving object state quantity matching unit 20, a moving object detection frequency prediction unit 21, a moving object state quantity erasing unit 22, a moving object state quantity updating unit 24, a moving object existence probability updating unit 220, and a moving object existence and a probability storage unit 222 .

The moving object existence probability storage unit 222 stores the existence probability of each of the plurality of pedestrians stored in the moving object state quantity storage unit 16 .

Based on the result of association with the detection result by the moving object state quantity matching unit 20, the moving object existence probability updating unit 220 updates the detection result of the pedestrian that is not associated with the state quantity among the detection results of the latest data. , the set value is stored in the moving object existence probability storage unit 222 as the existence probability of the pedestrian. The set value may be determined according to the predetermined detection accuracy of the detectors 60 and 62 .

In addition, the moving object existence probability updating unit 220 updates the existence probabilities of each of the plurality of pedestrians stored in the moving object existence probability storage unit 222 and updates the detection result of the latest data by the moving object state quantity matching unit 20 to correspond to the moving object existence probability storage unit 222 . The detectors 60 and 62 are updated so as to be increased according to the detection frequency for each of the attached pedestrian existence probabilities (see FIG. 6).

In addition, the moving object existence probability updating unit 220 updates the existence probabilities of each of the plurality of pedestrians stored in the moving object existence probability storage unit 222 and updates the detection result of the latest data by the moving object state quantity matching unit 20 to correspond to the moving object existence probability storage unit 222 . Each of the existence probabilities of pedestrians that has not been attached is attenuated and updated so as to become an existence probability according to the detection frequency (see FIG. 6).

The moving object state quantity elimination unit 22 removes the detection frequency predicted by the moving object detection frequency prediction unit 21 from among the pedestrian state quantities that are not associated with the detection result by the moving object state quantity matching unit 20 to a first threshold ( The state quantity of the pedestrian whose existence probability stored in the moving object existence probability storage unit 222 is smaller than the second threshold corresponding to the state quantity of the pedestrian smaller than the threshold in the first embodiment) is deleted. It should be noted that the second threshold value is a value corresponding to the probability of presence of a pedestrian, which is predetermined as a probability that the presence of a moving object is low.

Here, updating of the existence probability according to the detection frequency by the moving object existence probability updating unit 220 will be further described. Here, consider a case where cameras are used as the plurality of detectors 60 and 62 to detect pedestrians.

Regarding the state quantity of a certain moving object, if the elapsed time from the previous time is represented by Δt, and the decreasing speed of the existence probability is represented by ν, the change amount Δp of the existence probability can be expressed by the following equation.
Δp=−νΔt

When there are N cameras as the pedestrian state quantity estimation device 210, f _i is the operating frequency of the i-th camera (i=1, 2, . . . , N) at time t, and Let R _i (t) be the field of view of camera i taking into consideration a predetermined area invisible due to a structure to be reduced (for example, a blind spot due to a wall or the like). Furthermore, a function σ _i (x, t) indicating whether or not a certain place x is included in the field of view of camera i is expressed by the following equation.

Also, the non-detection rate of an object by camera i at location x is expressed as p _FN,i (x, t). The non-detection rate of moving objects can be derived from the distance from the detector, sunlight conditions, and the like. Then, when there is a moving object at the position x, the total detection frequency f(x, t) by the N cameras can be calculated by the following formula.

Therefore, the decrease amount ν _j (t) of the existence probability of the j-th moving object at the location x _j at the time t can be expressed by the following equation.
ν _j (t)=ν ₀ ·f(x, t)
By doing so, it is possible to update the existence probability according to the detection frequency.

Further, instead of calculating the detection frequency of each sensor each time the existence probability of each moving object is updated, the detection frequency calculated for each detector is stored in advance, and a value corresponding to the position of the moving object is extracted. may By updating the information stored only in the detectors whose detection frequency distribution has changed, it is possible to efficiently calculate when the number of detectors and moving objects increases.

In addition, since other configurations of the pedestrian state quantity estimation system according to the second embodiment are the same as those of the first embodiment, description thereof will be omitted.

<Operation of pedestrian state quantity estimation system>
Next, the operation of the pedestrian state quantity estimation system according to the second embodiment will be described. First, pedestrians are sequentially detected by each of a plurality of detectors 60 mounted on a plurality of vehicles and a detector 62 as an infrastructure sensor. While being transmitted to the apparatus 10, the pedestrian state quantity estimation processing routine shown in FIG. 7 is executed in the pedestrian state quantity estimation device 210. FIG. In addition, about the same process as 1st Embodiment, the same code|symbol is attached|subjected and detailed description is abbreviate|omitted.

In step S100, a set Y _i ={y _{i ,1} , . Upon receiving the set of observation error variance matrices R _i ={r _{i ,1} , .

In step S102, in accordance with the time of the detection result received in step S100, the prediction step of the Kalman filter is performed for each of the plurality of pedestrians, using the state quantity stored in the moving object state quantity storage unit 16 for the next time. , and a set of pedestrian state quantities X={x ₁ , . . . x _n } and its variance-covariance matrix V = {v ₁ ,..., v _n }.

In step S104, the state quantity predicted for each of the plurality of pedestrians in step S102 corresponding to the time of the detection result received in step S100, and the plurality of pedestrians represented by the detection result received in step S100 are calculated. is associated with the three-dimensional position of

In step S105, for each of the plurality of pedestrians in step S102 corresponding to the time of the detection result received in step S100, within a predetermined time between common times, a detector that can be detected among a plurality of detectors Predict the frequency of detection detected by

In step S106, among the state quantities of pedestrians that are not associated with the detection result in step S104, state quantities of pedestrians whose observed detection frequency is smaller than a threshold value predicted in step S105 are treated as moving object state quantities. Erased from the amount storage unit 16 .

In step S202, among the existence probabilities of each of the plurality of pedestrians stored in the moving object existence probability storage unit 222, for each of the pedestrian existence probabilities associated with the detection result of the latest data in step S104, , the detectors 60 and 62 are updated so as to be increased according to the detection frequency predicted in step S105. Further, among the existence probabilities of each of the plurality of pedestrians stored in the moving object existence probability storage unit 222, for each of the pedestrian existence probabilities not associated with the latest data detection result in step S104, Attenuate and update so that the existence probability corresponds to the detection frequency predicted in step S105.

In step S204, among the state quantities of pedestrians that are not associated with the detection result in step S104, the state quantities of pedestrians whose existence probability stored in the moving object existence probability storage unit 222 is smaller than the second threshold are deleted. do.

In step S108, for each state quantity associated with the detection result in step S104, the corresponding three-dimensional position y _i,j is set as an observed value, and the error variance matrix r _i,j obtained in step S200 is Using the state quantity of the pedestrian and the variance-covariance matrix, the state quantity and the last observed time are updated by the filtering step of the Kalman filter. Further, among the detection results received in step S100, a new state quantity x is generated based on the detection result of the pedestrian that was not associated with the state quantity, and the current time is used as the time of the last observation of the moving object. Stored in the state quantity storage unit 16 . Also, the predetermined existence probability is stored in the moving object existence probability storage unit 222 .

Then, in step S110, the updated state quantity set X is output as a result of integration of the pedestrian detection results, and the process returns to step S100.

As described above, according to the pedestrian state quantity estimation system according to the second embodiment, when sensing results are asynchronously received from a plurality of unspecified sensors, the detected state quantity is determined by the presence probability determined from the detection frequency. , the influence of unreliable detection results does not remain as state quantities, making it robust against false detections of each sensor.

[Third Embodiment]
<System configuration of pedestrian state quantity estimation system>
Next, a pedestrian state quantity estimation system according to the third embodiment will be described. In addition, the same reference numerals are given to the parts having the same configuration as those of the first embodiment and the second embodiment, and the description thereof is omitted.

In the third embodiment, when updating the presence probability of pedestrians from the result of predicting the detection frequency of moving objects, the latest sensor information of each of the detectors 60 and 62 stored in advance and the structure such as a wall This differs from the second embodiment in that the frequency of detection of moving objects is predicted using a map showing the position of .

As shown in FIG. 8, the moving object detection frequency prediction unit 321 in the pedestrian state quantity estimation device according to the third embodiment includes a sensor log storage unit 322, a map 323, and a detection frequency calculation unit 324. .

The sensor log storage unit 322 stores the latest sensor information of each of the detectors 60 and 62 . Map 323 also stores map information indicating the locations of structures such as walls that reduce the detectable range of detectors 60 and 62 .

The detection frequency calculation unit 324 stores the latest sensor information of the detectors 60 and 62 in the sensor log storage unit 322, and the map 323 stores the positions of structures such as walls that reduce the detectable range of the detectors 60 and 62. The detection frequency is calculated using the map information shown.

Specifically, for example, when cameras are used as the plurality of detectors 60 and 62 to detect pedestrians, the detection frequency calculation unit 324 calculates the detection position (including the detection direction) and the detection range of each camera. , and the position of a structure such as a wall, a blind spot or an undetectable area caused by the structure such as a wall (an area undetectable by the detector due to the structure) is obtained. The area excluding the undetectable area is defined as the field of view R _i (t) of the camera i, and a function σ _i (x, t) indicating whether or not a certain location x is included in the field of view of the camera i is determined.

Also, the latest non-detection rate p _FN,i (x, t) based on sensor information is used to calculate the total detection frequency f(x, t) by the N cameras. Therefore, by obtaining the reduction amount ν _j (t) of the existence probability of the j-th moving object at the location x _j at the time t using the above formula, it is possible to update the existence probability according to the detection frequency. .

The rest of the configuration and action of the pedestrian state quantity estimation system according to the third embodiment are the same as those of the second embodiment, so description thereof will be omitted.

In this way, by updating the presence probability of pedestrians using the range in which detection is impossible by the detector determined by the map as a constraint, it is possible to appropriately eliminate the state quantity of non-existing pedestrians.

[Fourth Embodiment]
<System configuration of pedestrian state quantity estimation system>
Next, a pedestrian state quantity estimation system according to a fourth embodiment will be described. In addition, the same reference numerals are given to the parts having the same configuration as those of the first embodiment and the second embodiment, and the description thereof is omitted.

The fourth embodiment differs from the second embodiment in that the moving object detection frequency predicted for each position is stored, and the stored detection frequency is used to predict the moving object detection frequency. there is

As shown in FIG. 9, the moving object detection frequency prediction unit 421 in the pedestrian state quantity estimation device according to the fourth embodiment includes a frequency map update unit 422, a map 323, a frequency map 423, and a detection frequency calculation unit 424. and have.

The frequency map 423 uses the map information stored in the map 323 and the latest sensor information of the detectors 60 and 62 to store the detection frequency for each position in the pedestrian detectable range. Specifically, for example, when the plurality of detectors 60 and 62 are two cameras 1 and 2, as shown in FIG. Make it possible. It is also assumed that a wall exists within the detectable range according to the map information. In this case, the entire area where pedestrians may exist is an area where pedestrians cannot be detected by camera 1 and camera 2 (the number of cameras Nc = 0), and pedestrians can be detected only by camera 1 or camera 2. area (the number of cameras Nc=1) and an area where the pedestrian can be detected by both camera 1 and camera 2 (the number of cameras Nc=2). Therefore, the number of detectable cameras differs depending on the position of the pedestrian, and the detection frequency changes according to the number of detectable cameras. Therefore, as shown in FIG. 10, an area indicating the number of cameras for each position is used as a frequency map. With this frequency map, the detection frequency of each camera can be calculated from the operation cycle of the camera corresponding to each area, and by adding the number of cameras, the detection frequency of the corresponding area can be derived. Therefore, the frequency map updating unit 422 updates the frequency map to the latest state using the sensor information and the map information stored in the map 323 .

The detection frequency calculation unit 424 extracts the detection frequency of the position indicated by the state quantity of the pedestrian from the detection frequencies stored in the frequency map 423, thereby calculating and outputting the detection frequency.

Other configurations and actions of the pedestrian state quantity estimation system according to the fourth embodiment are the same as those of the second embodiment, and therefore description thereof is omitted.

In this way, by updating the frequency map from the sensor information using the range where detection is impossible by the detector determined by the map as a constraint, the detection frequency corresponding to the position of the pedestrian's state quantity can be quickly obtained. is possible, and the state quantity of non-existent pedestrians can be appropriately eliminated.

In the above-described embodiment, a case where a pedestrian is targeted as a moving object to be detected has been described as an example, but the object is not limited to this. may be

In addition, the case where a plurality of detectors mounted on a plurality of vehicles and a detector as an infrastructure sensor are asynchronous sensors has been described as an example. It may be configured so as not to use a device.

Also, multiple detectors mounted on a single vehicle and using multiple instruments, including cameras and radar, may be asynchronous multiple sensors. For example, if a detector using a different measuring instrument is retrofitted to the vehicle, it is assumed that the number of multiple sensors will be indefinite.

Note that the program of the present disclosure can be provided by being stored in a recording medium.

10, 210 Pedestrian state quantity estimation device 12 Communication unit 14 Sensing result acquisition unit 16 Moving object state quantity storage unit 18 Moving object state quantity prediction unit 20 Moving object state quantity matching unit 21 Moving object detection frequency prediction unit 22 Moving object state quantity Elimination unit 24 Moving object state quantity update unit 50 Base stations 60, 62 Detector 70 Network 100 Pedestrian state quantity estimation system 210 Pedestrian state quantity estimation device 220 Moving object existence probability update unit 222 Moving object existence probability storage unit 321 Moving object Detection frequency prediction unit 322 Sensor log storage unit 323 Map 324 Detection frequency calculation unit 421 Moving object detection frequency prediction unit 422 Frequency map update unit 423 Frequency map 424 Detection frequency calculation unit

Claims (7)

a moving object state quantity storage unit that stores the state quantity and the last observed time of each of a plurality of moving objects;
a moving object state quantity prediction unit that predicts the state quantity of the moving object at the next time using the state quantity stored in the moving object state quantity storage unit for each of the plurality of moving objects;
a sensing result acquisition unit that receives detection results of detecting a plurality of moving objects and detection performance information from any of an indefinite number of multiple sensors;
Each time the sensing result acquisition unit receives the detection result, the state quantity of the moving object predicted for each of the plurality of moving objects by the moving object state quantity prediction unit corresponding to the time of the detection result; and the moving object state quantity for updating the state quantity and the last observed time of each of the plurality of moving objects stored in the moving object state quantity storage unit. a matching unit;
a moving object detection frequency prediction unit that predicts, based on detection performance information of each of the plurality of sensors, a frequency of detection of each of the plurality of moving objects by sensors that can be detected among the plurality of sensors;
The state quantity of the moving object which is not associated with the detection result by the moving object state quantity matching unit, and the detection frequency determined by the detection result of detecting a plurality of moving objects is predicted by the moving object detection frequency prediction unit. a moving object state quantity erasing unit for erasing from the moving object state quantity storage unit the state quantity of the moving object smaller than the detection frequency;
A moving object state quantity estimator including 2. A moving object state quantity estimation apparatus according to claim 1, wherein said plurality of sensors include sensors mounted on different moving objects. a moving object existence probability storage unit that stores the existence probability of each of the plurality of moving objects;
The moving object existence probability is set in the moving object existence probability storage unit according to the result of association with the detection result by the moving object state quantity matching unit or according to the predetermined detection accuracy of the sensor. or, as a result of associating the existence probability of each of the plurality of moving objects stored in the moving object existence probability storage unit with the detection result by the moving object state quantity matching unit, from the last observation time a moving object existence probability updating unit that increases or decreases according to the elapsed time of the sensor, or the predetermined detection accuracy of the sensor or the detection frequency by the moving object detection frequency prediction unit;
further comprising
The moving object state quantity erasing unit removes the state quantity of the moving object from the moving object state quantity storage unit based on the existence probability of each of the plurality of moving objects stored in the moving object existence probability storage unit. 3. A moving object state quantity estimating device according to claim 1 or 2, wherein the moving object state quantity estimating device is erased. The moving object detection frequency prediction unit,
a detection performance information storage unit that stores the detection performance information of each of the plurality of sensors;
The detection performance information of each of the plurality of sensors stored in the detection performance information storage unit, the moving object detectable range information in which the detectable range of the moving object is recorded, and updated by the moving object state quantity matching unit a detection frequency calculation unit that calculates the detection frequency based on the state quantity of the moving object;
The moving object state quantity estimation device according to claim 3, comprising: The moving object detection frequency prediction unit,
a detection frequency storage unit that stores information indicating a detection frequency for each detection position of a moving object predicted by the moving object detection frequency prediction unit;
The detection position of the moving object stored in the detection frequency storage unit based on the detection performance information received by the sensing result acquisition unit and the moving object detectable range information in which the detectable range of the moving object is recorded. a detection frequency update unit that updates the detection frequency for each
a detection frequency extraction unit that extracts a detection frequency corresponding to the state quantity of the moving object from the detection frequency storage unit;
The moving object state quantity estimation device according to claim 3, comprising: The plurality of sensors is a plurality of detectors that detect pedestrians or vehicles mounted on a plurality of vehicles or used as infrastructure sensors,
The state quantity of the moving object is the position and speed of the pedestrian or vehicle,
6. A moving object state quantity estimating apparatus according to claim 4, wherein said moving object detectable range information is a map indicating the position of a structure for reducing the sensor detectable range. a computer including a moving object state quantity storage unit that stores the state quantity and the last observed time of each of a plurality of moving objects,
a moving object state quantity prediction unit that predicts the state quantity of the moving object at the next time using the state quantity stored in the moving object state quantity storage unit for each of the plurality of moving objects;
A sensing result acquisition unit that receives detection results of detecting a plurality of moving objects from any of an indefinite number of multiple sensors;
Each time the sensing result acquisition unit receives the detection result, the state quantity of the moving object predicted for each of the plurality of moving objects by the moving object state quantity prediction unit corresponding to the time of the detection result; and the moving object state quantity for updating the state quantity and the last observed time of each of the plurality of moving objects stored in the moving object state quantity storage unit. matching unit,
a moving object detection frequency prediction unit that predicts a frequency of detection of each of the plurality of moving objects by sensors capable of detecting the plurality of sensors based on detection performance information of each of the plurality of sensors; and the state of the moving object. The state quantity of the moving object that is not associated with the detection result by the quantity matching unit, and the detection frequency determined by the detection result of detecting a plurality of moving objects is higher than the detection frequency predicted by the moving object detection frequency prediction unit. a moving object state quantity erasing unit for erasing the small state quantity of the moving object from the moving object state quantity storage unit;
A program to function as

Images