JPH11272989A - Traffic flow monitor device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To precisely trace a vehicle group or vehicle even when photography is wide in field angle and one screen needs to cover high resolution to low resolution. SOLUTION: This monitor device provides >=2 image analytic areas in a road elevation image picked up by a TV camera and cyclically processes a vehicle which travels from far away from the camera to nearby the camera through an image analyzing process 102. When the vehicle (vehicle group) which is far away from the camera passes through the initial registration area, template initial registration is performed (103) and tracing is carried out (104) by pattern matching. When the vehicle passes through the next initial registration area, the template which is traced so far is reset, a template is newly registered in this initial registration area (105), and the vehicle is traced (107) up to near by the camera. The speed of the vehicle is calculated from the movement of the template and the needed time, the length by which vehicles or vehicle groups which are slower than a specific speed succeed is measured by lanes, and queuing or congestion occurrence is decided (108).

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a traffic flow monitoring device for measuring the length of a queue of vehicles running on a road and determining the occurrence of congestion.

[0002]

2. Description of the Related Art Generally, at the entrance of an intersection, every time a traffic light turns red, a queue of vehicles trying to pass through the intersection occurs. Since the length of the queue is considered to indicate the degree of congestion at the intersection, the degree of congestion at each intersection is determined from this length and provided to ordinary households and drivers as traffic information. . In addition, if there is a queue even though the traffic light is green, it is said that there is a congestion phenomenon, and considerable congestion that the traffic congestion at a lower intersection extends to the intersection. . In addition, the length of the queue at the intersection is compared with the length of the queue in a direction orthogonal to the intersection, and the length of the signal in the direction of greater congestion is extended, so that it can be used for signal control. Have been done. Also, when the queue length for each lane is measured,
Depending on the length of the queue, you can control the blue turn of the right turn dedicated lane,
We realize efficient intersections. By the way, there are techniques disclosed in the following publications and documents as a conventional technique for detecting a queue and a traffic jam using an image by a TV camera. Japanese Patent Application Laid-Open No. Hei 7-210595 describes a technique for separating a vehicle from a road surface to detect a queue and a traffic jam by binarizing images. A technique for detecting a vehicle moving to a vehicle and detecting a vehicle based on a difference from the background with respect to a portion having a good resolution in the vicinity is described. Also, the IEICE General Conference: 1995 D-423: p14
9 ”“ Study of matrix length measurement algorithm by image processing ”
Describes a technique of detecting a vehicle from a differential value of an image and detecting stop of the vehicle from a difference between a current frame and a next frame. On the other hand, Japanese Patent Application Laid-Open
Japanese Patent Application Laid-Open No. 97335 discloses that the number of vehicles passing is counted by extracting and tracking vehicles from a portion having a high resolution from the center of the screen to the lower part of the screen. In this method, since the purpose is to count the number of vehicles passing, it is difficult to detect the congestion state of the road over the entire screen. Also, in Japanese Patent Application No. 8-119911,
A vehicle is detected at a lower part of the screen, that is, a high-resolution part near the camera, and the upper part of the screen, that is, a distant part of the camera is tracked by pattern matching. Has been advocated. However, in this method, it is impossible to cut out the vehicles one by one in a place where the resolution is poor in a distant place of the camera, and therefore, this method cannot be applied to an angle of view in which the vehicle approaches from a distant place of the camera.

[0003]

The above-mentioned prior art has the following problems. That is, JP-A-7-21
According to the technology disclosed in Japanese Patent Application Publication No. 0795, the brightness and resolution of a wide-angle image covering a wide range from the vicinity of the camera to the distant place, which shows all the vehicles forming a queue, vary greatly depending on the location. It is difficult to determine the threshold. In the technique disclosed in Japanese Patent Application Laid-Open No. 8-69596, it is difficult to accurately detect a vehicle from a camera having a low resolution, and it is difficult to measure a queue length extending over 100 m or more. Also, in the technology for performing queue and traffic congestion detection described in "IEICE General Conference: 1995 D-423: p149""Examination of matrix length measurement algorithm by image processing", the vehicle Since a stop is detected, when processing a wide-angle image over a range of 150 m, the moving distance per pixel is significantly different between the lower part and the upper part of the screen. However, it is impossible to accurately grasp the speed of each vehicle existing at a wide angle, and therefore, it is not possible to measure a queue length extending over 100 m. Further, in the technologies proposed in Japanese Patent Application Laid-Open No. 9-97335 and Japanese Patent Application No. 8-119911, both of the vehicles capture the vehicle at a high resolution portion. A camera with a poor resolution approaches from a distance and cannot catch the vehicle immediately after entering the screen.

SUMMARY OF THE INVENTION An object of the present invention is to track a vehicle group or a vehicle with high accuracy even in a wide-angle shooting mode and a high resolution to a low resolution in one screen, and to perform traffic jam determination and queue length measurement. To make it possible.

[0005]

The above object is achieved by providing image analysis areas at two or more locations in a bird's-eye view image and tracking the image analysis areas provided at two or more locations with a template captured at a distance. When a vehicle or group of vehicles approaching a more nearby image analysis area, resolve the template captured from a distance, register a new template in the nearby image analysis area, and use the new template to This is solved by tracking the vehicle to the image analysis area. In addition, image analysis areas are provided at two or more locations in the road overhead view image, and a vehicle or a vehicle group tracked by a template captured from a distance is closer to the image analysis area among the image analysis areas provided at two or more locations. When entering the area, capture the video of the vehicle entering the nearby image analysis area as a template, calculate the distance from the template captured at a distance, and if this distance is smaller than the width or total length of the vehicle, By eliminating the captured template and tracking the vehicle further to the vicinity with the template captured in the nearby image analysis area,
Will be resolved. Also, an image analysis area is provided for each lane, and the image differential value of a vehicle or a group of vehicles for each lane is projected in the direction perpendicular to the lane, and a portion having a high projection value is registered as a template, and is already registered. Calculates the distance from the template, and if this distance is smaller than the width or the total length of the vehicle, resolves the template captured in a distant place, and solves the problem by tracking the vehicle further to the vicinity with the template captured in the nearby image analysis area. Is done.

According to the present invention, the front of the vehicle is registered as a template in an image analysis area that crosses the road of the image input for each cycle, and the vehicle is tracked while updating this template, so that the resolution of a distant image with low resolution is reduced. By the way 1
A single vehicle or multiple vehicles are captured and tracked as a group of vehicles without distinction, and captured and tracked for each vehicle again in a nearby area with good resolution, so from wide-angle shooting to high-resolution to low-resolution in one screen Can be tracked with high accuracy, and highly accurate traffic congestion determination and queue length measurement can be performed.

[0007]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 4 is a block diagram showing an example of a system configuration of a traffic flow monitoring system to which the present invention is applied.
The traffic flow monitoring device 401 performs image processing on a bird's-eye view image of a road taken by a TV camera 402 installed on, for example, a support standing on a road, and measures the queue length of vehicles for each lane. Then, the measurement result is sent to the traffic control room 404 via the traffic event communication means 403, and is displayed on the display means 405. Alternatively, an intersection signal (not shown) is controlled according to the measurement result.

FIG. 3 is a block diagram showing a configuration of a traffic flow monitoring device according to one embodiment of the present invention. The traffic flow monitoring device 401 of the present embodiment includes a CPU 301 and a CPU 30.
1, a man-machine interface 302 for the operator to operate, a main memory 303 serving as a memory for storing a program of the CPU 301 and serving as a working memory, and an image processing means for processing an image under the control of the CPU 301 304, image input means 306
And an image memory 305 which also serves as a working memory of the image processing means 304. The image processing means 304 performs image analysis of an image analysis area, which will be described in detail later, tracks a light / dark pattern of the vehicle by shading pattern matching, and measures the length of the queue based on the position and speed of the vehicle.

FIG. 1 shows a first processing procedure of the traffic flow monitoring device according to the present invention. Basically, the TV image input 101
And the image analysis processing 102 are cyclically repeated from image analysis 1 to image analysis n. In the meantime, the template initial registration that registers the front of the vehicle, such as the front grill, windshield, and headlight, as a template
This is referred to as “TP initial registration”. ) 103, template tracking equivalent to vehicle tracking or vehicle group tracking (hereinafter,
Called "TP tracking". ) 104, a template update (hereinafter referred to as “TP update”) 105, an instantaneous speed measurement 106, and a tracking end process 107. This processing cycle is the same as that of the vehicle being tracked. Is executed when a predetermined condition is satisfied. In the processing cycle of the TV image input 101 and the image analysis processing 102, a queue length measurement processing 108 and a traffic index (number of passing vehicles per lane, passing speed, occurrence and resolution of traffic jam, queue length) measurement / transmission / accumulation processing This processing is executed every cycle or every fixed time. The conditions for executing each processing will be described later in detail together with the processing of the traffic flow monitoring device.

FIG. 5 shows an example of an overhead road image road of a four-lane road. The overlooked road is divided into four lanes by lane dividing lines 507 to 511, and is referred to as a first lane, a second lane, a third lane, and a fourth lane from the right side in the drawing. Initial registration area 50 across the bird's-eye view road
1 to 503 are provided. Initial registration areas 501 to 503
Then, a differentiation process is performed, for example, as disclosed in JP-A-9-97335. The analysis waveforms 504 to 506 are
The results of the differentiation process are vertically projected in the initial registration areas 501 to 503, respectively, and become larger when the front of the vehicle is in the initial registration areas 501 to 503. This is because the front of the vehicle includes a light and dark pattern that increases the differential value. Rectangular frames 512 to 5
Reference numeral 28 denotes a pattern matching portion of a tracked vehicle. Rectangular frames 512 to 520 are vehicles that are initially registered in a template in the initial registration area 501 and are tracked, and rectangular frames 521 to 524 are initial registration areas. 50
2, the vehicles are initially registered in the template and are being tracked, and the rectangular frames 525 to 528 are vehicles that are initially registered in the initial registration area 503 and are being tracked.

[0011] A vehicle traveling from the far side to the near side of the camera on the overlooked road is cyclically processed by the TV image input 101 and image analysis processing 102 such as image analysis 1 to image analysis n. First, when a vehicle traveling from a distant location to a nearby location of the camera passes through the initial registration area 501, the template is initially registered (TP initial registration 10).
3) Tracked by pattern matching (TP tracking 104). Next, when passing through the initial registration area 502,
The template that has been tracked so far is released, and the template is newly registered in the initial registration area 502 (T.
P. Update 105). Then, similarly, when passing through the initial registration area 503, the template that has been tracked so far is released, the template is registered again in the initial registration area 503, and the vehicle is tracked to the nearest camera (tracking end processing). 107). If it is not possible to cut out and track one car at a time in a part of the camera with a poor resolution at a distant location, the method described above tracks multiple vehicles as a group of vehicles, Measure the speed.
When the resolution of the vehicle has increased due to the approach of the camera, the template is initially registered again for each vehicle, and the location and speed of each vehicle are measured. Details of the measurement of the moving speed of the vehicle or vehicle group will be described later. Immediately before re-registering a template, as a first method for eliminating a template that has been tracked so far, for example, tracking cutoff lines 530 to 531 are provided, and rectangular frames 512 to 520 initially registered in the initial registration area 501 are tracked. When reaching the cutoff line 530, the template is canceled, and the rectangular frames 521 to 524 initially registered in the initial registration area 502 are replaced with the tracking cutoff line 531.
When it reaches, the template is canceled. A second method for resolving a template that has been tracked immediately before registering a template again will be described later.

Next, details of speed measurement of a vehicle or a group of vehicles and details of queue length measurement for each lane will be described. If the positional relationship between the camera and the road and the focal length of the camera lens are known, the coordinates in the road overhead view image can be associated with the actual distance from the camera on the road. Therefore, Japanese Patent Application Hei 8
As described in -254546, the movement on the screen of the rectangular frame during the vehicle tracking is converted into the moving distance on the actual road,
By dividing by the time required for the movement, the instantaneous speed of the vehicle or vehicle group is measured (instantaneous speed measurement 106). In addition, the speed is measured from a moving distance of several tens of milliseconds near the camera, and the speed is measured from a moving distance of several seconds far from the camera, so that the accuracy is stably maintained.

FIG. 6 superimposes and displays the instantaneous speed on a rectangular frame relating to vehicle tracking. Then, a distance from a camera installation position or an intersection entrance to a vehicle or a series of vehicles whose instantaneous speed is equal to or lower than a predetermined speed is measured for each lane and is set as a queue length (queue length measurement processing 108). In FIG. 6, the vehicle 601 is in a state where the second lane is 20 km / h or less.
Therefore, the area from the intersection entrance to the vehicle 601 is the position where the queue exists. The queue length becomes longer as the number of vehicles with lower vehicle speeds increases, and as the positions of vehicles with lower vehicle speeds extend farther. When vehicles or groups of vehicles whose instantaneous speeds are equal to or lower than a predetermined speed for each lane are consecutive, it is determined that traffic congestion has occurred.

FIG. 7 shows a method for checking the distance between vehicles in the initial TP registration 103. A rectangular frame 703 tracking the vehicle 701 to check the distance between the vehicles
And the rectangular frame 704 tracking the vehicle 702
Is set as a square SPQR with the lower center as the diagonal. The extension of the line segment PS and the line segment QR intersect at the vanishing point. On a straight road, it can be easily obtained as the intersection of the dividing line 508 and the dividing line 510. The line segment SR and the line segment PQ may be in the horizontal direction as long as the TV camera 402 is almost in the road direction. When the TV camera 402 is inclined with respect to the road direction, the directions of the line segment SR and the line segment PQ are directions of the N vector conjugate to the vanishing point. The “vanishing point” and the “conjugate N vector” are described in detail in the document “Image understanding—Mathematics of three-dimensional recognition—” by Kenichi Kanaya (Morikita Shuppan Co., Ltd.) P3 to P21. Although the line segment SR and the line segment PQ are apparently shorter on the screen than the line segment SR, the distances on the road are equal.
Here, the distance between the SR and the PQ on the road is referred to as a distance in the road crossing direction. In addition, the distance on the road between the line segment PS and the line segment QR is substantially equal, and is referred to as a distance in the road traveling direction. FIG. 7A shows a case where the vehicle 701 and the vehicle 702 are running side by side, and the distance in the road crossing direction is secured by about one lane width, but the traveling direction of the road is necessarily secured by the entire length of the vehicle. Not. On the other hand, FIG.
(B) is a case where the vehicle 701 and the vehicle 702 are running longitudinally, and the distance in the road crossing direction is not one lane, but the traveling direction of the road is more than the full length of the vehicle.

FIG. 8 shows a second method for canceling a template that has been tracked just before re-registering a template.
The method of is shown. The vehicle 801 is being tracked by the rectangular frame 802 and approaching the initial registration area 502. Vehicle 80
Since the front surface of No. 1 has entered the initial registration area 502, the analysis waveform 505 rises, and a rectangular frame 803 as a template initial registration candidate is generated. At this time, a square SPQR is set between the rectangular frame 802 and the rectangular frame 803 to calculate the distance in the road traveling direction and the road crossing direction. If both are shorter than a predetermined distance, the two vehicles run in parallel. It is determined that there is no possibility of vertical running. As a result, the rectangular frame 802 is deleted. Then, the vehicle 801 is newly tracked by the rectangular frame 803. Since the rectangular frame 803 is initially registered closer to the camera than the rectangular frame 802, a portion more suitable for tracking the vehicle 801 than the rectangular frame 802 is registered. The above operation is the same when the vehicle 801 passes through the initial registration area 503, and the rectangular frame is updated again.

As described above, the traffic flow monitoring device according to the present embodiment tracks one or a plurality of vehicles as a vehicle group in a low-resolution portion far from the camera by the first processing procedure, and approaches the camera to obtain a resolution. As the vehicle goes up, the vehicle is detected again accurately, so it is suitable for processing wide-angle images from near to the camera to a distant place, even if one screen contains high-resolution to low-resolution images. Can be accurately tracked. In addition, since the instantaneous speed is measured by accurately tracking the vehicle or vehicle group, the presence or absence of congestion and the length of the queue are determined by the magnitude of this instantaneous speed,
It is most suitable for monitoring intersections with dynamic changes where there are vehicles that do not stop completely or vehicles that stop and start repeatedly.

FIG. 2 shows a second processing procedure of the traffic flow monitoring device according to the present invention. The second processing procedure differs from the first processing procedure in that waveform analysis is performed for each lane and initial registration is performed. Hereinafter, the template initial registration will be described taking the processing of the second lane as an example. FIG. 9 is a view obtained by cutting out the second lane of the road overhead view image. FIG. 10 illustrates the second lane segmentation unit in FIG. 9 in which only the horizontal edges are emphasized by the same processing as the initial registration area 503.
That is, the image differential value of the vehicle or vehicle group in the second lane is projected in a direction perpendicular to the second lane. Of course, the second lane portion may be cut out after emphasizing only the horizontal edge on the entire screen by the same processing as the initial registration area 503. The projected waveform 1001 projects the emphasized horizontal edge in the horizontal direction (projects the image differential value of a vehicle or a group of vehicles in the vertical direction with respect to the second lane), normalizes it with the road width on the screen, and The size is adjusted to the upper and lower directions. Next, T. P. A place where the projected waveform 1001 takes the maximum value in the initial registration 103 is set as a template candidate. Then, the distance from the already registered template is calculated by the method shown in FIG. 8, and if the distance is smaller than the width or the total length of the vehicle, the template candidate at the lower part of the screen is registered as a template, and Cancel the upper template. The rectangular frame 110 of FIG.
Reference numerals 1 to 1109 denote rectangular frames for vehicle tracking obtained in this manner. The detection of the template registration candidates described above is performed by the lane-specific image analysis 202. In this way, the traffic flow monitoring device of the present invention tracks one or more vehicles as a group of vehicles in the low-resolution portion far from the camera by the second processing procedure, and approaches the camera to increase the resolution. As a result, the vehicle is accurately detected again, so that the same effect as in the first processing procedure is exhibited.

[0018]

As described above, according to the present invention,
In the low-resolution part far away from the camera, one or more vehicles are tracked as a group of vehicles, and as the resolution increases as you approach the camera, the vehicle is re-detected accurately again, so wide-angle images from near the camera to the distance Even if one screen includes high resolution to low resolution, the vehicle group or the vehicle can be tracked with high accuracy. In addition, the instantaneous speed is measured by accurately tracking the vehicle or vehicle group, and the magnitude of this instantaneous speed determines the presence or absence of congestion and the length of the queue, so some vehicles do not stop completely or repeat stop and start Ideal for monitoring intersections with dynamic changes where vehicles are present.

[Brief description of the drawings]

FIG. 1 is a first processing procedure of a traffic flow monitoring device according to the present invention.

FIG. 2 shows a second processing procedure of the traffic flow monitoring device according to the present invention.

FIG. 3 is a block diagram showing a configuration of a traffic flow monitoring device according to an embodiment of the present invention.

FIG. 4 is a block diagram showing a system configuration of a traffic flow monitoring system to which the present invention is applied;

FIG. 5 is a diagram showing an example of an overhead road image road of a four-lane road;

FIG. 6 is a diagram showing an instantaneous speed superimposed on a tracking frame for vehicle tracking according to the present invention.

FIG. 7 is a diagram showing a method for checking the distance between vehicles according to the present invention.

FIG. 8 is a diagram showing a method of re-registering a template according to the present invention.

FIG. 9 is a diagram showing a second lane of a bird's-eye view image of a road according to the present invention.

FIG. 10 is a diagram in which only horizontal edges are emphasized in a second lane cutout according to the present invention.

FIG. 11 is a diagram of a rectangular frame for tracking a vehicle according to the present invention.

[Explanation of symbols]

101: image input means, 102: image analysis processing, 103: template initial registration, 104:
Template tracking, 105 ... template update, 1
06: Instantaneous speed measurement, 107: Tracking end processing,
108: queue length measurement processing, 109: traffic index measurement / transmission / accumulation processing, 202: image analysis by lane, 207: tracking end processing

 ──────────────────────────────────────────────────続 き Continuing from the front page (72) Inventor Yoshiyuki Sato 5-2-1 Omikacho, Hitachi City, Ibaraki Prefecture Inside the Omika Plant, Hitachi, Ltd. (72) Inventor Toru Nitta 2477 Takaba, Hitachinaka City, Ibaraki Prefecture Inside Hitachi Car Engineering Co., Ltd.

Claims (5)

[Claims] A TV camera image input unit; and an image processing unit, wherein an image analysis region is provided in a road overhead view image by the TV camera image input unit, a vehicle image in the image analysis region is captured as a template, In a traffic flow monitoring device that tracks a vehicle by a pattern matching process using a template, an image analysis area is provided at two or more locations, and the image analysis area provided at the two or more locations is tracked by a template captured from a distance. When a vehicle or a group of vehicles enters a more nearby image analysis area, the template captured at a far distance is resolved, and then a new template is registered in the nearby image analysis area, and the new template is used for further registration. A traffic flow monitoring device for tracking a vehicle to a nearby image analysis area. 2. A TV camera image input means, comprising: an image processing means, an image analysis area is provided in a road overhead view image by the TV camera image input means, and a vehicle image in the image analysis area is captured as a template; In a traffic flow monitoring device that tracks a vehicle by a pattern matching process using a template, an image analysis area is provided at two or more locations, and the image analysis area provided at the two or more locations is tracked by a template captured from a distance. When a vehicle or a group of vehicles enters a more nearby image analysis area, a vehicle image that has entered the nearby image analysis area is captured and used as a template, and the distance to the template captured at a greater distance is calculated. If the width is smaller than the width or the total length of the vehicle, the template captured in the distant place is canceled and the template captured in the nearby image analysis area is removed. A traffic flow monitoring device characterized in that the vehicle is tracked further to the vicinity by a template. 3. A TV camera image input means and an image processing means, wherein an image analysis area is provided in a road overhead view image by the TV camera image input means, and a vehicle image in the image analysis area is captured as a template, In a traffic flow monitoring device that tracks a vehicle by a pattern matching process using a template, an image analysis area is provided for each lane, and an image differential value of a vehicle or a group of vehicles for each lane is projected in a direction perpendicular to the lane. Is registered as a template, a distance from the already registered template is calculated, and if the distance is smaller than the width or the total length of the vehicle, the already registered template is canceled. Traffic flow monitoring device. 4. The vehicle according to claim 1, wherein a speed of a vehicle or a group of vehicles tracked by the template is calculated from the movement of the template and a required time, and the speed of a vehicle or a vehicle slower than a predetermined speed for each lane is calculated. A traffic flow monitoring device that measures the length of a series of vehicles and forms a queue for each lane. 5. The vehicle according to claim 1, wherein a speed of a vehicle or a group of vehicles tracked by the template is calculated from the movement of the template and a required time, and the speed of a vehicle or a vehicle group slower than a predetermined speed is determined for each lane. A traffic flow monitoring device characterized by determining that traffic congestion has occurred when a group of vehicles are connected.