JP2001229488A - Vehicle tracking method and traffic state tracking device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a traffic state tracking device which can continue vehicle tracking and obtain a stable travel track if vehicles possibly over each other or are hidden in an image. SOLUTION: An image processor 2 inputs an image photographed by a television camera 1, detects a vehicle newly appearing on a screen at time (t), tracks a vehicle which was present on the screen at time t-1 on the screen at the time (t), and outputs the vehicle position and information on whether the tracking can be carried out as tracking results. A tracking complementing device 3 performs parallel processing with the image processor 2 and predicts the vehicle position at the time (t) from the vehicle position at the time t-1. Further, the tracking result at the time (t) obtained by the image processor 2 is checked and the image processor 2 is instructed to detects a vehicle which can not be tracked again according to predicted coordinate values at the time (t). When the vehicle can be detected through the redetection, the vehicle position is registered in a travel track storage part 305 as determined coordinates at the time (t). If the vehicle can not be detected even by the redetection, the predicted coordinate values at the time (t) are registered.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

The present invention relates to a traffic situation tracking device using an image processing device.

[0002]

2. Description of the Related Art A method of measuring traffic volume and abnormal traffic by capturing a vehicle trajectory by processing a video image of a camera installed on a road has been used for a long tunnel or the like. Japanese Unexamined Patent Application Publication No. 8-147591 provides a function of predicting the coordinates of the next cycle in the image processing apparatus, and calculates the vehicle position detected by the image processing and the position and speed of each vehicle in the previous cycle stored in advance. A traffic flow measuring device that performs tracking for each vehicle by taking correspondence with a predicted position is disclosed.

[0003] On the other hand, in recent years, needs for analysis of vehicle running behavior have been increasing. Japanese Patent Application Laid-Open No. H11-96494 assumes that a wide monitoring area is monitored by a plurality of cameras,
There is disclosed a method of complementing a vehicle traveling in a blind spot between a plurality of cameras and displaying a state of the entire monitoring area. Here, parameters such as the initial speed at the time of vehicle detection in image processing are input to the vehicle running simulation unit, vehicle running is simulated by the vehicle following model, and vehicle running in the blind spot between cameras is complemented by the above following model. I do.

[0004]

In traffic flow monitoring and vehicle running behavior analysis using images, it is important to continuously track traffic conditions, that is, to improve the accuracy of vehicle tracking using images. However, there is a problem that the tracking is interrupted due to the overlap of vehicles on the screen, and there is a problem with respect to tracking in terms of accuracy as compared with the conventional sensor.

The method for predicting the vehicle position in Japanese Patent Application Laid-Open No. Hei 8-147591 estimates the vehicle position in the next cycle from the running position of the vehicle in the previous cycle and the instantaneous speed, and cannot cope with a sudden change in speed. There is a problem with the prediction accuracy. In Japanese Patent Application Laid-Open No. 11-96494, the running behavior of the vehicle in a section where the camera cannot be photographed is limited to prediction, and no consideration is given to complementing the detailed running state of the vehicle within the field of view of the camera. In this case, vehicles that could not be detected due to the overlap cannot be displayed.

[0006] Separating the overlap of vehicles in the field of view of the camera requires more accurate predictions of vehicle behavior. For this purpose, it is necessary to model the characteristics of the distance between one vehicle and one preceding vehicle, acceleration / deceleration, and the like in detail. The addition of these running model calculations in addition to the detection and tracking processes conventionally performed by image processing devices is limited in general-purpose image processing devices, and expensive systems including high-performance image processing devices are limited. Will be needed.

An object of the present invention is to solve the above problems,
It is an object of the present invention to provide a vehicle tracking method and a traffic condition tracking device that can continue vehicle tracking even when it is difficult to separate the vehicles by image processing alone, in which overlapping and hiding of vehicles occur, and a stable traveling trajectory can be obtained.

[0008]

SUMMARY OF THE INVENTION In order to achieve the above object, according to the present invention, a vehicle is detected by image processing from an image on a screen periodically taken in by a television camera installed on a road, and the vehicle is detected at the previous time (t-1). In the vehicle tracking method for searching for the vehicle detected in the next time (t) on the screen at the next time (t) and tracking the running state of each vehicle, when the tracking by the image processing is not possible, the complementing process based on the predicted position To continue vehicle tracking.

In the complementing process, a predicted vehicle position at the time t is obtained for each vehicle detected at the time t-1 in parallel with the image processing at the time t-1, and exists at the time t-1. For the vehicle that could not be searched on the screen at time t, a re-search is performed by the image processing based on the predicted vehicle position. If the vehicle was detected, the detected position is set as the vehicle position at time t. If no vehicle is detected, the predicted vehicle position is regarded as the vehicle position at time t.

According to the present invention, even when a characteristic portion of a vehicle for tracking a vehicle by image processing is hidden by overlapping of the vehicle on a screen, another portion that is not hidden is detected again to track the vehicle. Processing can continue. At time t−
Among the vehicles detected at 1 and that could not be searched at time t, those that passed outside the detection area of the television camera can be excluded from the predicted vehicle position.

Further, the predicted vehicle position is obtained by maximum, minimum and average moving conditions from t-1 to t based on a change in vehicle speed obtained from the image processing.
When it is not possible to search again at the predicted vehicle position under a certain moving condition, the search is performed again at the predicted vehicle position under another moving condition. Thereby, the accuracy of the re-search can be improved.

Further, the re-search based on the predicted vehicle position may be performed by searching for the vicinity of the predicted vehicle position or the vicinity of a position where the predicted vehicle position is moved on a screen in a predetermined direction so as to reduce overlap with a following vehicle. Do. For example, when the vehicle appears from the lower part of the screen and passes to the upper part of the screen, the detection window set as the periphery of the predicted vehicle position moves a predetermined distance to the upper left of the predicted position so that the following vehicle does not overlap.

A traffic situation tracking device to which the vehicle tracking method of the present invention is applied is a television camera for periodically photographing a road condition, a screen shot is input to detect a vehicle, and a vehicle is detected in response to the next screen input. An image processing device that tracks a vehicle detected on the previous screen, a vehicle position prediction unit that obtains a predicted vehicle position on the next screen from the vehicle position for each vehicle detected on the previous screen, and a next screen by the image processing device A tracking complement device having a vehicle position correcting means for instructing the image processing device to perform retracking based on the predicted vehicle position for a vehicle that cannot be tracked by the vehicle is provided.

[0014] The tracking complementing device determines whether tracking or retracking on the next screen has been performed by the image processing device. If tracking or retracking has been performed, the vehicle position of the detected vehicle and the retracking can be performed. If there is no such vehicle, any of the predicted vehicle positions is determined as the vehicle position on the next screen (in the embodiment, the tracking determination unit 303 and the vehicle position correction /
And a means for storing the determined vehicle position as a time-series traveling locus.

According to this, this can be realized by adding the tracking complementing device to a general-purpose image processing device having a normal function, and it is not necessary to use an expensive image processing device having a high function. Further, the tracking supplementing device can be inexpensively configured by a personal computer or the like and a storage medium storing its processing functions as programs.

Further, an animation display means is added to the traffic condition tracking device of the present invention, and the running locus of each vehicle stored in the running locus storing means is displayed in animation in synchronization with the input cycle from the television camera, thereby monitoring. The traffic condition of the area can be displayed from a viewpoint other than the angle of view of the camera.

[0017]

Embodiments of the present invention will be described below in detail with reference to the drawings. FIG. 1 is a block diagram showing a configuration of a traffic situation tracking device according to one embodiment of the present invention. The traffic situation tracking device includes one or a plurality of television cameras 1 for capturing road images and an image processing device 2 for inputting an image captured by the television camera 1 and performing image processing.
In order to predict the position of the next sampling time of the vehicle existing on the image in parallel with the image processing device 2 and enable re-tracking when the vehicle tracking is interrupted, the vehicle position is determined in parallel with the image processing. It consists of a tracking supplementary device 3 that makes predictions.

The image processing device 2 has a vehicle detection unit 201, a vehicle tracking unit 202, and a tracking vehicle management unit 203. The vehicle detection unit 201 is means for detecting a vehicle newly appearing on the screen at the latest time t by the image processing function. The vehicle tracking unit 202 is means for individually tracking vehicles existing on the screen at the previous time t-1 from the screen using the image processing function. The tracking vehicle management unit 203 includes a vehicle detection unit 201.
And means for managing information such as the position and speed of the vehicle existing on the screen at time t, obtained by the vehicle tracking unit 202.

The tracking supplementing device 3 includes a vehicle position predicting unit 301,
It has a predicted coordinate management unit 302, a tracking determination unit 303, a vehicle position correction / determination unit 304, and a traveling locus storage unit 305. The vehicle position prediction unit 301 is a unit that predicts the vehicle position at time t from the vehicle position information at time t-1. The predicted coordinate management unit 302 is a unit that manages the coordinate values predicted by the vehicle position prediction unit 301 for each vehicle. Tracking determination unit 303
Shows the respective vehicles existing on the screen at time t by the vehicle tracking unit 20.
2 is means for determining, for each vehicle, whether or not the vehicle 2 has been tracked.

The vehicle position correction / determination unit 304 determines, based on the predicted coordinate values calculated by the vehicle position prediction unit 301, for the vehicle determined to be unable to be tracked by the tracking determination unit 303.
A position where a vehicle is predicted to be present is instructed to the image processing device 2. Also, a signal indicating whether or not the corresponding vehicle has been detected around the predicted position is received from the image processing device 2, and if detected, the coordinates of the detected vehicle are determined as the position of the vehicle that could not be tracked by the image processing device 2. If the position is not detected, the predicted coordinate value itself is regarded as the position of the vehicle that could not be tracked, and is determined. The traveling locus storage unit 305 includes:
The vehicle position determined by the vehicle position correction / determination unit 304 is added to the traveling locus of each vehicle and stored as the vehicle position at time t.

Next, the configuration and operation of each part of the image processing device 2 and the tracking complement device 3 will be described in detail. The road image captured by the television camera 1 is input to the image processing device 2. The vehicle detection unit 201 and the vehicle tracking unit 202 realized by the image processing function of the image processing device 2 are described in “Real-time measurement of traffic flow and congestion by image processing” in the 2nd Image Sensing Symposium Proceedings, pp. 293-296. In a manner similar to the method described in (1), the detected characteristic part of the vehicle is registered as a template, and vehicle tracking is performed by template matching.

The vehicle detection unit 201 provides a detection area on the screen, and detects a characteristic portion such as a horizontal edge of the vehicle in the detection area at a timing when the vehicle newly appears in the detection area by an image processing function. The region where the horizontal edge exists is registered as a vehicle tracking template. The timing at which a vehicle newly appears on the screen is determined, for example, by storing a background (road surface) pattern, correlating the pattern with the pattern each time an image is input from a television camera, It can be realized by a method such as determining that it has passed. The vehicle passage determination and template registration processing will be specifically described with reference to FIGS.

FIG. 2 shows an example of a screen in which no vehicle is present. A window 22 for determining the timing for determining whether or not the vehicle has passed is set on a screen 21 in the detection area. The position of the window 22 is determined based on the height, angle, and the like of the television camera that captures the image, so that the vehicle enters the screen. And
The background pattern 23 is stored in advance from the window 22 of the screen 21 where no vehicle exists. When the vehicle detection unit 21 starts processing, a window 22 is displayed for each screen input from the TV camera.
The correlation value between the input image and the background pattern 23 is calculated.

FIG. 3 shows an example of a change in the correlation value before and after the vehicle passes. The horizontal axis in the figure is time, and the vertical axis is a correlation value. The correlation values 31, 33, and 34 are values at times t1, t2, and t3, respectively. The time zone 32 is a time zone in which the vehicle is passing through the window 22 and the correlation value is decreasing.

FIG. 4 shows an example of a screen corresponding to each correlation value in FIG. In each of the screens (a), (b) and (c), the window 22 is a window for measuring the timing at which the vehicle passes as in FIG. 2, and the window 43 in (c) is a vehicle for registering a template. Is a detection window for detecting the characteristic portion of. At the moment when the vehicle passes from the window 22, the characteristic portion at the lower rear part of the vehicle is detected in the detection window 43 set immediately above the window 22.

Since the screen shown in (a) has no vehicle, the correlation value 31 between the pattern 42 of the window 22 and the background pattern 23 is also high. (B) is a screen in which the vehicle 41 is passing,
Since the pattern 42 of the window 22 is clearly different from the background pattern 23, the correlation value 33 also decreases as shown in FIG. Since the vehicle 41 has passed through the area of the window 22 in the screen of (c), the correlation value 34 between the pattern 42 and the background pattern 23 becomes high again.

At the timing of the screen shown in FIG.
Assuming that a vehicle is present above 22, a characteristic portion of vehicle 41 is searched for in detection window 43. Here, the horizontal edge of the vehicle 41 is detected by the image processing function, and as a result, the area 44 where the horizontal edge exists is registered as the tracking template 50 of the vehicle 41.
The registered tracking template 50 is passed to the vehicle tracking unit 202, and performs a vehicle tracking process described below for each vehicle.

Further, at the time of vehicle detection, a vehicle length and a vehicle type can also be detected. As for this detection method, for example, as described in the traffic flow monitoring device of JP-A-10-307988, the vehicle length is calculated from the transit time and speed of a vehicle in a certain area, and a large vehicle is calculated based on the vehicle length. Or small car.

Although the vehicle length and vehicle type information are not essential to the configuration of the present invention, once the vehicle length information is obtained, when the vehicle position prediction unit 301 calculates the predicted position of the vehicle, the vehicle length information is obtained. The detailed prediction in consideration of can be performed. When large / small vehicle type information is obtained, a vehicle position correction / determination unit described later
In the process of 304, it is also possible to change the size of the detection window for re-detecting the vehicle depending on the size of the large / small vehicle.

Next, the vehicle tracking unit 202 of the image processing device 2 will be described. FIG. 5 shows an example of a screen of a vehicle tracking process by image processing, showing running vehicles 51 and 52 and templates 53 and 54 for tracking the respective vehicles. In the case of this example, templates 53 and 54 having two kinds of different characteristics are stored, and when an image whose time progresses and changes in the next cycle is input, the similarity between the templates 53 and 54 is high. An area is searched from the screen, and the area is registered as the positions of the vehicle 51 and the vehicle 52, respectively. This is tracking processing using the image processing function. The coordinate system on the screen is an xy coordinate system as shown in the figure, with the origin (0, 0) at the upper left corner.

A method of setting a template search range in the tracking process will be described with reference to FIG.
This is to reduce the load on the processing by limiting the range in which the template is searched by using the fact that the range in which the template exists can be limited based on the processing cycle interval and the traveling direction of the vehicle on the screen. Therefore, the setting of the range is not necessarily required when the processing capacity of the apparatus has a margin, and the setting for searching the entire screen may be used.

FIG. 6 shows an example of a screen at time t. It is assumed that the tracking template of the vehicle 61 exists at the position 62 on the screen at the previous time t-1. Since the traveling direction on the road is the upper left direction of the screen, the position of the template on the screen at time t is moving in the minus direction in both the x and y coordinates on the screen. Accordingly, the range for searching for the template is set to a range extended in the x and y directions from the position (dotted frame) of the template 62 at the time t-1, as indicated by 63 in the drawing. The range to be widened is set by assuming a maximum value when moving at a time interval between time t-1 and time t. For example, the time between time t-1 and t is 100 msec.
If the vehicle ran at a speed of 100 km / h
At 100msec, it will travel about 278m. Therefore, a range of 278 m from the position at the time t-1 is converted into coordinates on the screen, and a range of 63 is set.

The tracking processing is performed for each image processing cycle, and the coordinate values obtained as a result of the tracking are stored in a tracking vehicle management unit 203 which is a work memory of the image processing apparatus 2. FIG. 7 shows the structure of data stored in the tracked vehicle management unit n.
1,. . . ), Time of cycle, coordinate value, speed, acceleration, tracking impossible flag, vehicle length, vehicle type, etc. are stored in time series.

The positions of the coordinates on the screen obtained by the image processing are converted into coordinates on a plane as seen from the overhead view in FIG. The method of this conversion is described in, for example, "3D Computer Graphics" by Shoichiro Nakamae (Shokodo) pp. 14-47 and "3D Image Measurement" by Seiji Iguchi (Shokodo) pp. 91-99. It can be performed by a method such as

In the case of a road, there are many cases where a marking is used as a reference for the distance on the road, for example, 55 in FIG.
There are poles on the center line as shown from to 60. Usually, such markings are provided at regular intervals, so that the coordinate position on a plane can be converted according to the correspondence between the position of the marking and the vehicle position on the surface. As described above, the instantaneous speed can be obtained by converting the coordinates into the coordinates on the plane, and the running status can be displayed from a bird's-eye view by the animation display means described later.

Template positions 81 and 82 in FIG.
Are the templates 5 of the vehicles 51 and 52 in FIG. 5, respectively.
It corresponds to the position on the plane of 3,54. The coordinate values are, for example,
As shown in FIG. 8, it is assumed to be unified at the lower part of the rear end of the vehicle. As the speed, an instantaneous speed is calculated from a coordinate value on a plane obtained as a result of tracking for each image processing cycle and the image processing cycle. As the acceleration, an average acceleration is calculated from the instantaneous velocities before and after and the image processing cycle.

FIG. 9 shows a flow of processing realized by each means of the image processing apparatus 2. First, an image at time t is input (s101), a vehicle newly appearing in the vehicle detection area is detected (s102), and it is determined whether a new vehicle has been detected (s103). If a new vehicle can be detected, the new vehicle is registered in the tracking vehicle management unit 203 (s10
After 4), if a new vehicle is not detected, a tracking process is immediately performed (s105).

In the tracking process at step s105, the template information of the vehicle being tracked is sequentially read from the tracked vehicle management unit 203, and the tracking is performed by performing the matching process in the image at time t. The coordinates of the tracking result at time t are
It is stored in the tracking vehicle management unit 203. If the vehicle with the ID managed at the previous time (t-1) is not found this time, a tracking impossible flag "1" is set in the corresponding vehicle data of the tracking vehicle management unit 203. After the tracking process is completed, the process timer is advanced to the next time (t + 1) (s106), and the process waits for the next cycle of image capture.

As described above, the image processing apparatus 2 registers a characteristic portion of a vehicle as a template, and tracks a portion having a high degree of similarity with the template by detecting the portion from the input image in each cycle. In this tracking method, if the vehicle portion corresponding to the template is hidden by a subsequent vehicle or the like, the tracking is interrupted. In the present embodiment, a tracking complement device 3 is provided for continuing the tracking when the tracking is interrupted.

The vehicle position predicting unit 301 of the tracking complement device 3
Is the position and speed of each vehicle at the previous time (t−1) stored in the traveling locus storage unit 305 at the same cycle as the image processing device 2 for all vehicles existing on the screen input from the television camera 1 This is a process of calculating a predicted coordinate value at the next time (t) based on information such as

Here, data stored in the traveling locus storage unit 305 will be described. The traveling locus storage unit 305 records the coordinate values of each vehicle determined by the tracking determination unit 303 and the vehicle position correction / determination unit 304 together with the passing time. Since it is a traveling locus including the positional relationship between vehicles obtained from the measured traffic flow, it can be used as a highly useful database for vehicle behavior analysis and the like.

FIG. 10 shows the data structure stored in the traveling locus storage unit. The traveling locus storage unit 305 stores information for each vehicle ID,
Time, coordinates, speed (m / s), acceleration (m
/ s ² ) and the type and length of each vehicle. Further, in order to determine whether or not the vehicle is currently being tracked on the screen, a tracking end flag is set for the vehicle whose tracking has been completed. If the tracking is possible, the tracking vehicle management unit 203
Is the same as The coordinates when tracking is impossible are calculated based on the predicted vehicle position as described later.

The vehicle position predicting section 301 reads out the above data for the vehicle stored at the time t-1 among the vehicles stored in the traveling locus storage section 305 and calculates a predicted coordinate value.

There are various methods for predicting the coordinates of the next cycle. In the present embodiment, since the current instantaneous speed and position of each vehicle and the relative speed and positional relationship with the preceding vehicle can be obtained by the image processing apparatus 2, for example, the 14th traffic It is calculated by the method described on pages 17-20 of the Journal of Engineering Research Presentations.

The present prediction method is based on a follow-up traveling model that travels while keeping a distance between the preceding vehicle and the preceding vehicle. In other words, the running behavior of the vehicle is classified into three patterns, that is, a single action, a following action, and a stopping action, and the predicted position of the vehicle at time t is calculated by applying a calculation formula corresponding to each pattern.

FIG. 11 is a flowchart showing one embodiment of the vehicle prediction method. The process starts at time t, and first, the previous time t
The vehicle IDs existing on the screen of the detection area at −1 are sequentially read from the traveling locus storage unit 305 from the top (s20).
1). The read first vehicle ID is set as a vehicle i, and data relating to a vehicle i-1 which is a preceding vehicle of the vehicle i is read (s
202).

Next, the running behavior (behavior pattern) of the vehicle i is determined from the inter-vehicle distance between the vehicle i and the vehicle i-1 and the speed of the preceding vehicle i-1 as follows (s203). The speed of the preceding vehicle i-1 is fv, the position of the preceding vehicle is fx, the position of the target vehicle i (the first vehicle) is bx, and the free travel distance is d. The running behavior of the vehicle i is fv = 0.
Is determined as a stop action, fv> 0 and fx−bx> d as a single action, and fv> 0 and fx−bx <d as a follow-up action.

In step s203, the number 1 is selected when it is determined to be the single action (s204), the number 2 is selected when it is determined to be the following action (s205), and the number 3 is determined when it is determined to be the stop action. Based on the selected formula, the moving distance of the vehicle i from time t-1 is calculated, the predicted coordinates at time t are calculated (s207), and the predicted coordinates are stored in the predicted coordinate management unit 302. (S20
8).

[0049]

(Equation 1)

[0050]

(Equation 2)

[0051]

(Equation 3)

Next, assuming that i = i + 1, the vehicle ID to be processed is advanced to the next vehicle (s209).
At -1, it is determined whether the vehicle has reached the end of the vehicle existing on the screen (s210). If the end has been reached, the process is terminated. If the end has not been reached, the process returns to s202, and the above series of processes is repeated.

In the above calculation method, the instantaneous speed and the acceleration are used. However, the speed actually obtained from the vehicle tracking by using the image processing is unstable. FIG. 12 illustrates the vehicle speed as a result of the vehicle tracking. As shown in the figure, since the speed 121 obtained from the vehicle tracking fluctuates, a history of the speed characteristics is obtained for each vehicle, and the value 122 obtained by smoothing the speed change characteristics at the time t is used to obtain a more stable predicted value. Can be obtained.

Alternatively, in consideration of the above-mentioned unstable fluctuation of the speed, the maximum value V
It is also possible to calculate t-1max and the minimum value Vt-1min, calculate the moving distance using the obtained maximum value and minimum value, and give a range to the predicted value.

[0055]

(Equation 4)

As described above, the predicted coordinate values calculated for each vehicle are stored in the predicted coordinate management unit 305.

FIG. 13 shows data stored in the predicted coordinate management unit. The predicted coordinate management unit 305 determines, for each vehicle, Condition 0: average value of vehicle speed V (condition 0), maximum value Vmax (condition 1),
Each predicted vehicle position (coordinate value) obtained from the minimum value Vmin (condition 2) is stored. If the vehicle cannot be tracked by the vehicle tracking unit 202, the vehicle position correction / determination unit 3
03 performs vehicle re-detection using the predicted coordinate values of the condition 0, the condition 1, and the condition 2. Note that the average, maximum, and minimum values of the vehicle speed are assumed as conditions, but are not necessarily limited to these three values.

Next, the tracking determination unit 303 will be described. The tracking determination unit 303 is a vehicle tracking unit 202 of the image processing device 2,
It is determined whether or not each of the vehicles existing on the screen at time t-1 has been tracked at time t. If the vehicle cannot be tracked, the process proceeds to the process of vehicle position correction / determination section 304. For the vehicle that has been tracked, the coordinates of the tracking result are stored in the travel locus storage unit 305.

FIG. 14 shows the flow of processing of the tracking determination unit.
First, vehicle data being tracked is read from the tracked vehicle management unit 203 (s301), and it is determined whether or not the read vehicle i can be tracked at time t (s302). Specifically, if the coordinates at time t have been written and the tracking impossible flag is not 1, it is determined that tracking has been performed.

When the tracking is possible, the coordinate value of the vehicle i at the time t written in the tracking vehicle management unit 203 is stored in the traveling locus storage unit 305 as the determined coordinate value at the time t (s303). Then, the process proceeds to the process for the next vehicle (i + 1) (s304).

If the tracking is impossible, that is, if the coordinates at time t are not written and the tracking impossible flag is set, the predicted coordinate value of the vehicle i is read from the predicted coordinate management unit 302, and the vehicle i is displayed on the screen. Is determined (s305). If the predicted coordinate value is outside the screen, it is considered that the vehicle i has passed out of the detection area, and the tracking process of the vehicle i is completed, and a tracking end flag is set for the vehicle i (s306).

When the predicted coordinate value is within the screen, the vehicle i
Is still present in the screen, but the tracking by the image processing device 2 has failed.
The process is passed to the determination unit 303 (s308), and the process moves to the next vehicle (s309).

Next, the vehicle position correcting / determining section 304 will be described. When the vehicle i present on the screen cannot be tracked, the image processing device 2 detects the vehicle I again based on the predicted coordinate value of the vehicle i stored in the predicted coordinate management unit 302. The position of the detection window for handover is passed, and re-detection is instructed.

The image processing device 2 performs the detection processing of the vehicle i again in the designated detection window, and if it is detected, re-registers the template for tracking, and stores the coordinate values of the vehicle i. The coordinates are stored in the travel locus storage unit 305 as fixed coordinates. If the detection cannot be performed again, the predicted coordinates obtained by the vehicle position prediction unit 301 are stored in the traveling locus storage unit 305 as temporary fixed coordinates of the vehicle i.

The vehicle position correction / determination unit 304
Even when the vehicle overlaps on the screen and the tracking processing by the image processing device 2 is interrupted, the tracking can be continued based on the predicted coordinates. In addition, the vehicle position prediction unit 301 corrects the predicted coordinate value using the actual measurement value of the vehicle detected in the detection processing of the image processing apparatus 2, and performs the prediction of the next time using the value, thereby achieving higher accuracy. High prediction can be realized.

The point of this means is that the vehicle existence range on the screen is converted and estimated from the coordinate values on the plane of each vehicle handled by the tracking complement device 3. As already described with reference to FIGS. 5 and 8, the coordinates on the plane can be converted into the coordinates on the screen, and therefore, as shown in FIG.
From the coordinate values 1503, 1504 on the plane (a) of 2, the existence range on the screen (b) of a vehicle of a standard size with the coordinate values at the lower end is like 1505, 1506, respectively. Can be estimated.

The purpose of this means is to set the position of the detection window of the vehicle i to be re-detected from the predicted coordinates of the vehicle I and pass it to the image processing device.
It is highly likely that the preceding template is hidden in the succeeding vehicle because the tracking process could not be performed in the following tracking process. Therefore, as shown in FIG.
Then, from the estimated area 1505 on the screen, the vehicle 1502 (1501
Area 1507 excluding the estimated area 1506 on the screen of the following vehicle
To the image processing apparatus 2 as a detection window. Here, for the following vehicle, if the coordinate position at the time t has already been obtained in the tracking processing by the image processing, the coordinate is used, and if not, the predicted coordinate value is used.

When the image processing device 2 is configured to obtain vehicle length / vehicle type information, the size of the detection window is set to one of a normal vehicle and a large vehicle based on the obtained vehicle type information. be able to.

FIG. 16 shows the flow of the process performed by the vehicle position correction / determination unit 304. This processing is started when it is determined in the tracking determination processing 302 that the vehicle i has been lost. Processing s4
01 to s409 are the processing flow of the vehicle position correction / determination unit 304, and the processing s501 to s506 are the processing flow in the image processing apparatus 2 that is restarted in response to the instruction from the vehicle position correction / determination unit 304. is there.

First, the condition of the predicted coordinates instructed to the image processing apparatus 2 is set to condition 0 (j = 0) (s401), and the coordinate value of the vehicle i read from the predicted coordinate management unit 302 and the predicted coordinate management unit The position of the detection window on the screen is calculated from the coordinate values of the following vehicle i + 1 read out from 302 or the running locus storage unit 305 (s402), and the calculated position of the detection window is transmitted to the image processing device 2 (s403). And
In step s404, the process waits for a template registration signal from the image processing apparatus 2.

The image processing device 2 provides a detection window at the position of the detection window specified by the vehicle position correction / determination unit 304 (s501). Next, vehicle detection is performed by detecting a characteristic portion of the vehicle such as a horizontal edge in the set detection window (s502). The processing here is the same as the vehicle detection processing described using FIG.

Next, it is determined whether or not the vehicle has been detected (s503). If it cannot be detected, an “unregistered” signal is transmitted to the vehicle position correction / determination unit 304 (s50).
4). If it is determined that the detection is successful, the “registration” signal and the coordinates of the detected vehicle are used as the vehicle position correction / determination unit 304.
(S505). Then, the template is re-registered at the position where the vehicle can be detected (s506), and the re-detection processing of the image processing device 2 is terminated.

Upon receiving the signal from the image processing device 2, the vehicle position correcting / determining section 304 restarts the processing (s40).
4) It is determined whether the received signal is a “registered” signal for which the template has been registered or an “unregistered” signal for which the template has not been registered (s405). In the case of "registration", the coordinate values of the new template are registered in the traveling locus storage unit 305 (s406), and the process ends. In the case of “unregistered”, one condition j to be instructed to the image processing apparatus is recommended (s407), and processing s402 is performed.
And the above series of processing is repeated. When the condition j is larger than the maximum condition number (jmax = 3 in FIG. 12), the predicted coordinate value of the vehicle i (for example, the average value of the condition 0) is set as the coordinate value of the time t from the predicted coordinate management unit 302. It is stored in the traveling locus storage unit 305.

As described above, the vehicle position correction / determination unit 30
In No. 4, the predicted coordinates of the plurality of conditions of the vehicle i stored in the predicted coordinate management unit 302 are sequentially read and transmitted to the image processing device 2, so that the image processing device cannot perform tracking based on the transmitted predicted coordinate values. The detected vehicle may be detected again, and if detected, the coordinate value of the new template may be registered in the travel locus storage unit 305 as the coordinate value of the vehicle i at the time t. it can.

FIG. 17 shows an image of the re-detection processing performed by the image processing apparatus based on the predicted coordinate values. (A) is the screen at time t-1, and (b), (c), and (d) are the same screen at the next time t.

On the screen (a), a tracking template 172 is set for an arbitrary vehicle 171 at time t-1. In the normal tracking process, the screen (b) at time t
Attempt to detect similar regions using 172. However, a similar area cannot be detected because the corresponding area overlaps the following vehicle 173 on the screen. In that case, the vehicle position correction / determination unit 303 calculates the detection window position on the screen from the predicted coordinate value of the vehicle 171 and passes it to the image processing device 2, and the image processing device 2 performs the operation as indicated by 174 in the screen (c). The passed detection window is set, and vehicle detection processing such as detecting a horizontal edge is performed again. If the detection is possible, the template 175 in the detected area is re-registered as a new template of the vehicle 171.

By performing the above processing, the area of the template registered for an arbitrary vehicle i is hidden, but if the vehicle i itself exists on the screen, the area without occlusion is deleted. By re-registering as a template, vehicle tracking can be continued. As described above, the template region registered by the following vehicle or the like is hidden and tracking by image processing is interrupted, but the vehicle detection region at an arbitrary position is reset based on the coordinates predicted by the vehicle prediction unit, and the hidden region is hidden. The feature area without the feature can be re-registered as a vehicle tracking template.

FIG. 18 is an explanatory diagram of the processing of the image processing apparatus and the tracking complementing apparatus based on specific data examples. In the figure,
The times are t, t + 1, t + 2 in order from left to right, and the processing shown in the figure is the processing in the image processing device 2 in the upper part and the processing in the tracking complementing device 3 in the lower part.

(A) to (c) in the figure correspond to times t to t + 2
184 to 186 are obtained as a result of tracing of the image processing device 2 from time t to t + 2, and the tracked vehicle management unit 2
03 is a coordinate value and a tracking impossible flag stored in the same. Reference numerals 195, 196, and 197 denote unique numbers of the vehicle, respectively.

(D) to (f) in the figure are obtained by projecting the position of the vehicle predicted by the tracking complement device as an overhead view image. 190 to 192 are coordinate values predicted by the vehicle position prediction unit from time t to t + 2 and stored in the predicted coordinate management unit 302. 193 and 194 are vehicle position correction / determination units 30.
The coordinates stored in the travel locus storage unit 305 as a result of the process of 3.

The description will be made according to the flow of actual processing. First, when the apparatus is started up, an image (a) at time t is input. In the initial state, since there is no stored template, only the detection processing is performed from the screen, and the coordinate values (50, 80) and (50, 120) of the vehicle 195 and the vehicle 196 at time t are obtained. Store. This coordinate value 184 is the tracking complement device 3
And the tracking determination unit 303 sets the coordinate value at time t to 0 because the tracking impossible flag is 0, and stores the coordinate value in the traveling locus storage unit 305.

The vehicle position estimating unit 301 of the tracking complementing device then calculates the time t based on the coordinate value, speed, acceleration, etc. at time t.
Predict the vehicle position at +1. The predicted coordinate values of the vehicle 195 and the vehicle 196 are shown at 191. Next, in response to the input of the image (b) at time t + 1, the image processing device 2 performs a process of detecting a new vehicle and a process of tracking the vehicles 195 and 196. Since there is no new vehicle in the image (b), the coordinates 185 obtained by tracking the vehicles 195 and 196 are input to the tracking complement device 3. In the tracking determination unit 303 of the tracking complement device 3, the vehicle 195, the vehicle 1
Since the tracking coordinates are obtained for all 96 and the tracking disabled flag is also 0, the tracking coordinates 185 are stored in the traveling locus storage unit 304 as the coordinate values at time t + 1.

Similarly, at time t + 2, the vehicle position predicting section 301 of the tracking complement device 3 also stores the traveling locus storing section 30
5 from the coordinate value at time t + 1 to the predicted coordinate value 19 at time t + 2
Calculate 2. From the calculated predicted coordinate values, y of the vehicle 195 is calculated.
It is predicted that the coordinates will be negative and will be off the screen.

Next, upon receiving the image input at time t + 2, the image processing device 2 detects the new vehicle and performs the vehicle 195 and vehicle 1
Perform 96 tracking operations. Since the new vehicle 197 is detected and the vehicle 195 cannot be tracked in the tracking process, the tracking disabled flag of the vehicle 195 is set to 1 and the coordinates 186 of the other two vehicles are input to the tracking complement device 3. Tracking determination unit 303 of tracking complementing device 3
Then, since the tracking impossible flag of the vehicle 195 is 1 and the predicted coordinate value is also outside the screen, it is determined that the vehicle 195 has passed the outside of the screen, and the tracking end flag of the vehicle 195 in the traveling locus storage unit 305 is set to 1.
Since the vehicle 196 can be tracked and the tracking impossible flag is 0, the tracking coordinate value is stored in the travel locus storage unit 305, and the vehicle 197 is a newly detected vehicle and the coordinate 186 is the travel locus storage unit 305 as the coordinate value at time t + 2. To be stored.

As described above, the position of the vehicle at each time is obtained by repeating the image input ⇒ tracking process ⇒ coordinate determination process ⇒ prediction process, so that the trajectory of the vehicle is sequentially stored in the trajectory storage unit 305. Will be.

Next, another embodiment of the present invention will be described. FIG. 19 illustrates a configuration of the traffic situation tracking device according to the second embodiment.
An animation display unit is added to the configuration of the first embodiment. The image from the TV camera 1 installed on the road is
Usually it depends on a certain height and angle. For this reason, the actual speed, the distance between vehicles, and the like are often difficult to understand intuitively only from the video.

The animation display means 4 of the present embodiment
In synchronization with the input of the moving image from the television camera 1, the vehicle on the road is displayed as a bird's-eye view based on the coordinates of each vehicle stored in the travel locus storage unit 304 at each time. At this time, the vehicle displays an animation in a shape prepared in advance. As a result, vehicles on the road can be perceived as a bird's-eye view running condition, and a plurality of vehicles that appear to be overlapped on the image can be separated and the running position can be visually displayed. In addition, by using an interface with the locus data stored in the traveling locus storage unit 304, it is also possible to reproduce past traveling conditions.

The traffic condition tracking devices of the first and second embodiments have been described by way of example using one television camera and one image processing device for simplicity. The present invention can also be realized by a configuration in which a plurality of television cameras and a plurality of image processing devices are combined. In fact, a monitoring device for a long tunnel to which the present invention is applied captures an image from a television camera installed at intervals of several hundred meters on one line.

FIG. 20 shows a system configuration using a plurality of television cameras and a plurality of image processing devices. From the images taken by the television cameras 1a, 1b, 1c, the image processing devices 2a, 2b, 2c detect vehicles running in the respective photographing areas and perform tracking processing. The configuration of the image processing device 2 here is the same as that of the first embodiment.

The tracking supplementing device 3 manages information on the vehicle detected by each of the image processing devices 2a to 2c, predicts the vehicle position, determines whether or not tracking is possible, corrects the vehicle position when tracking is not possible, and fixes it. The processing is performed in common. In this case, since one vehicle position prediction unit 301 commonly predicts information on vehicles obtained from a plurality of image processing apparatuses, it is possible to take correspondence between vehicles tracked in each image processing apparatus. . Thereby, the traveling state of each vehicle on the entire route can be grasped at a time, and furthermore, the entire situation can be visually grasped by displaying the entire route by the animation display means.

FIG. 21 shows an example of a screen on which images from a plurality of television cameras are displayed as an animation. Screens (a), (b), and (c) are TV camera 215,
This is a screen area captured by the television camera 216 and the television camera 217. The screen (d) is an example in which the vehicle running in the screen area captured by the plurality of television cameras is displayed as an animation by the animation display unit 4. The vehicles 218 to 225 on the shooting screens (a), (b), and (c) correspond to the vehicles 218 to 225 on the animation screen (d), respectively.

Although only the positional relationship of the vehicle within the area can be detected from the image of each TV camera, the coordinates detected by the image processing device and the coordinates predicted by the vehicle position predicting unit are mutually corrected. In addition, after eliminating the overlap between vehicles such as the vehicle 218 and the vehicle 219 on the screen 211, it is possible to visually capture the coordinate information of the vehicle group existing on the continuous route.

[0093]

According to the present invention, even when the vehicle overlaps or hides on the screen, the vehicle position is predicted by the traffic flow prediction function having the vehicle running model, and the vehicle is controlled based on the predicted position. Since the tracking is continued by re-detecting or supplementing, it is possible to provide a traffic condition tracking method and device capable of obtaining a stable traveling trajectory.

[Brief description of the drawings]

FIG. 1 is a configuration diagram showing one embodiment of a traffic situation tracking device of the present invention.

FIG. 2 is an explanatory diagram of a detection window of vehicle passage and template registration processing.

FIG. 3 is an explanatory diagram showing a change in a correlation value before and after a vehicle passes.

FIG. 4 is an explanatory diagram showing an example of a screen before and after a vehicle passes.

FIG. 5 is an explanatory diagram showing a screen example of a vehicle tracking process by image processing.

FIG. 6 is an explanatory diagram showing a method of setting a range in which a template is searched in a tracking process.

FIG. 7 is an explanatory diagram of an information storage format in a tracking vehicle management unit.

FIG. 8 is an overhead view showing a positional relationship when a coordinate position on a screen is converted into a coordinate position on a plane.

FIG. 9 is a flowchart of vehicle detection and vehicle tracking processing by the image processing apparatus.

FIG. 10 is an explanatory diagram of an information storage format in a traveling locus storage unit.

FIG. 11 is a flowchart of a prediction process performed by a vehicle position prediction unit.

FIG. 12 is an explanatory diagram showing a change in speed obtained by vehicle tracking in image processing.

FIG. 13 is an explanatory diagram of an information storage format in a predicted coordinate management unit.

FIG. 14 is a flowchart of a determination process performed by a tracking determination unit.

FIG. 15 is an explanatory diagram showing a setting method of a detection window.

FIG. 16 is a flowchart of processing by a vehicle position correction / determination unit and reprocessing by an image processing device.

FIG. 17 is an explanatory diagram of a process in which the image processing device performs vehicle detection again around the predicted coordinate value.

FIG. 18 is an explanatory diagram showing a specific example of parallel processing of the image processing device and the tracking complement device.

FIG. 19 is a configuration diagram showing another embodiment of the traffic situation tracking device of the present invention.

FIG. 20 is a configuration diagram that realizes another embodiment (FIG. 19) with a plurality of television cameras and a plurality of image processing apparatuses.

FIG. 21 is an explanatory view showing a screen example of an animation display of a traffic situation in which a plurality of vehicles are tracked over a long distance.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 ... TV camera, 2 ... Image processing apparatus, 3 ... Tracking complement apparatus, 4 ... Animation display means, 201 ... Vehicle detection part, 202 ... Vehicle tracking part, 203 ... Tracking vehicle management part, 3
01: vehicle position prediction unit, 302: predicted coordinate management unit, 30
3: tracking determination unit, 304: vehicle position correction / determination unit, 30
5. Running trajectory storage unit.

 ──────────────────────────────────────────────────続 き Continuing on the front page (72) Inventor Takayoshi Yokota 7-1-1, Omika-cho, Hitachi City, Ibaraki Prefecture Inside the Hitachi Research Laboratory, Hitachi, Ltd. (72) Inventor Yoshiyuki Sato 5-chome, Omika-cho, Hitachi City, Ibaraki Prefecture No. 1 F term in Hitachi Ltd. Omika Works (reference) 5H180 AA01 BB15 CC04 CC30 EE01 EE02 5L096 BA04 CA04 FA06 FA34 FA66 FA69 HA03 HA05 JA09

Claims (7)

[Claims] 1. A vehicle is detected by image processing from an image on a screen taken in time series from a television camera installed on a road, and the vehicle detected at the previous time (t-1) is detected at the next time (t). In the vehicle tracking method of searching and tracking on the screen of (1), the predicted vehicle position at time t of the detected vehicle is obtained in parallel with the image processing at time t-1, and the vehicle cannot be tracked on the screen at time t. The vehicle is searched again by the image processing on the screen at time t based on the predicted vehicle position based on the predicted vehicle position. If the vehicle can be detected, the detected position is determined as the vehicle position at time t, and the vehicle is detected. A vehicle tracking method characterized by considering the predicted vehicle position as the vehicle position at time t when the vehicle position cannot be determined. 2. The vehicle according to claim 1, wherein the predicted vehicle position is determined by maximum, minimum, and average movement conditions from time t-1 to time t based on speed fluctuations obtained from the image processing. A vehicle tracking method characterized in that when a re-search at a predicted vehicle position based on a condition is not possible, the vehicle is re-searched by changing to another moving condition. 3. The re-search based on the predicted vehicle position according to claim 1, wherein the re-search based on the predicted vehicle position is a position around the predicted vehicle position or a position where the predicted vehicle position is moved on the screen in a direction to reduce overlap with a subsequent vehicle. Tracking method for the surrounding area. 4. A television camera for photographing road conditions,
A traffic condition tracking device including an image processing device that periodically inputs a captured screen to detect a vehicle and tracks the vehicle detected in the previous screen in response to the input of the next screen, wherein the detection is performed by the image processing device. Vehicle position prediction means for obtaining a predicted vehicle position on the next screen from a vehicle position on the previous screen for each vehicle, and re-tracking based on the predicted vehicle position for a vehicle that cannot be tracked on the next screen by the image processing device. A traffic condition tracking device, comprising a tracking complement device having a vehicle position correcting means for instructing the image processing device. 5. The tracking complementary device according to claim 4, wherein the tracking complementing device determines whether tracking or retracking on the next screen has been performed by the image processing device, and if tracking or retracking has been performed, a vehicle of the detected vehicle. Means for determining one of the predicted vehicle positions as the vehicle position of the next screen when the re-tracking is not performed, and means for storing the determined vehicle position as a time-series running locus. A traffic condition tracking device characterized by the following. 6. The vehicle according to claim 4, wherein an animation display means for displaying a vehicle having an animation shape prepared in advance is provided at a vehicle position for each vehicle on an overhead view screen for overhead view of the road condition. Traffic condition tracking device. 7. A plurality of television cameras provided apart from each other along the road, and a plurality of image processing devices for inputting images of the corresponding cameras and performing image processing, respectively. A traffic condition tracking device provided with one of the tracking complement devices that complements the processing results of these image processing devices.