JPH103595A - Road monitoring device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve the recognizing ratio of a road object in a device for monitoring a road surface by distributing a camera so that image pickup areas can be mutually overlapped. SOLUTION: Cameras A and B are arranged on a road, and the road is photographed at a prescribed tilt angle ϕ. The image pickup areas of the both cameras are set so that one part can be overlapped. When an object 100 such as a vehicle is present in the overlapped area, it is image picked-up by the both cameras, and only an extracted result obtained by one camera is adopted for the objective data in the overlapped area. The object extracted result is stored in a memory, and the lane entrance and exit of the vehicle is discriminated from the past history. Also, when the object can be extracted from only one camera although the object is present in the overlapped area, it is judged that the object is also present in the other camera, and the processing is executed.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a road monitoring device, and more particularly to a device for monitoring by arranging a plurality of imaging means such as cameras along a traveling direction of a road.

[0002]

2. Description of the Related Art Hitherto, a monitoring system has been developed which monitors the state of a road, particularly the running state of a vehicle, using a monitoring camera on the road. For example, Japanese Patent Publication No. 63-57840 discloses a technique in which a plurality of television cameras are arranged along a traveling direction of a road, and a moving speed of a vehicle is calculated from an obtained image to detect traffic congestion. ing.

[0003]

However, in order to reliably monitor the road surface, it is necessary to arrange the imaging areas of adjacent cameras without any gaps, sometimes overlapping each other. With such an arrangement, for example, if a vehicle exists in the overlapping area, an adjacent camera will photograph the vehicle together. Will be erroneously detected if it exists on the road.

[0004] The present invention has been made in view of the above-mentioned problems of the prior art, and an object of the present invention is to overlap an image pickup area of a plurality of cameras with each other to reliably capture an object.
It is an object of the present invention to provide a road monitoring device having an excellent recognition rate by eliminating erroneous detection of an object.

[0005]

In order to achieve the above object, a first aspect of the present invention is to monitor a road by a plurality of image pickup means arranged so that image pickup areas thereof overlap with each other along a traveling direction of the road. A road surveillance device, comprising: an extraction unit for extracting an object present in each imaging region from image data obtained by each imaging unit; and extracting one of the extracted objects when the extracted object is present in an overlapping region of an adjacent imaging region. Processing means for selectively outputting the result of extraction of the overlapping area by the means. As a result, it is possible to effectively prevent erroneous recognition of an originally single object as two different objects.

According to a second aspect of the present invention, in the first aspect, there is further provided a detecting means for detecting an obstacle on the road from a plurality of extraction results at different times.

In a third aspect based on the first or second aspect, the image pickup means is provided on a road side to take an image in the width direction of the road surface, and the processing means comprises an image pickup area adjacent to the extracted object. In the case where the object exists in the overlapping area, the other adjacent image pickup means performs the processing assuming that the object has been photographed. Thus, even if an object cannot be photographed by one image pickup device due to environmental factors such as backlight or shade, if the object can be photographed by another image pickup device, it is processed as if the object is present. The object recognition rate is improved regardless of the situation.

[0008]

BEST MODE FOR CARRYING OUT THE INVENTION

<First Embodiment> An embodiment of the present invention will be described below with reference to the drawings.

FIG. 1 shows the arrangement of a camera as an image pickup means and its image pickup area. Cameras A, B, C,
D is arranged at intervals d along the traveling direction of the road so that D looks down on the road surface, and each camera photographs the road surface at an angle of view θ. In the figure, dotted lines indicate the imaging regions of the respective cameras, and adjacent imaging regions are set so as to partially overlap each other.

FIG. 2 shows a photographing situation of the cameras A and B in FIG. FIG. 2A is a plan view showing only the cameras A and B extracted in FIG.
(B) is a diagram of the camera A and the camera B viewed from the side. The photographing center axes of the cameras A and B form an angle of φ with respect to the vertical plane (tilt angle φ), and photograph the front road surface at the angle of view θ as described above. When the object 100 such as a vehicle exists within the angle of view of the camera B, the object 100 is captured by the camera B. However, when the object 100 exists in the overlapping range of the imaging area of the camera A and the camera B, not only the camera B but also the camera B A is also captured. For example, if the object 100 exists on the photographing center axis of the camera B, the object A is photographed by the camera A at a position θA from the center axis.

FIG. 3 shows an object 100 at a position as shown in FIG.
The image of the camera A and the image of the camera B in the case where is present is shown in a simplified manner. FIG. 3A shows an image obtained by the camera A, and FIG. Object 10
0 is located at a position φA from the center of the camera A, and is located at the center of the camera B. The area indicated by the dotted line in each image is the overlapping range 200 of the imaging area.

As described above, when an object exists in the overlapping range of the imaging region, the object is photographed by two adjacent cameras at the same time. Therefore, an image obtained by each camera is simply processed. If an object is extracted, it is erroneously detected that there are two different objects that are originally the same object.

Therefore, in the present embodiment, when an object exists in an overlapping range in an image that is known in advance, only the result of extracting the overlapping area by one of the adjacent cameras is selectively used. By adopting it, erroneous detection is effectively prevented.

FIG. 4 is a block diagram showing the configuration of the road monitoring apparatus according to this embodiment. In FIG. 1, cameras A, B, C, and D are used. However, since the number of cameras is arbitrary, generalized cameras n, n + 1, n + 2,.
(N is a natural number of 1 or more). An object extraction unit 10 is connected to each camera, and the obtained image is supplied to these object extraction units 10. The object extracting unit 10 includes an A / D converter and an image processing computer, and extracts an object from an input image. Objects include a stationary object and a moving object, but a stationary object can be extracted by acquiring a background image in which no object exists in advance and calculating the difference between the input image and the background image. It can be extracted by calculating the optical flow. Note that the optical flow is a relative velocity vector in a moving image, and can be extracted by associating two temporally continuous images using a template of an appropriate size and calculating the displacement. The direction of the optical flow vector indicates the moving direction of the object, and the magnitude of the vector indicates the magnitude of the velocity of the object. The position and size data of the stationary object and the position, size and velocity data of the moving object extracted by the object extracting unit 10 are supplied to the overlap determining unit 12. The overlap determination unit 12 includes a microcomputer, and determines whether the input stationary object or moving object is an object existing in the overlapping area. Specifically, the overlap determination unit 12 includes not only the object data extracted from the image obtained by its own camera,
Object data extracted from images obtained by two adjacent cameras is supplied, and it is determined in advance whether these objects exist in a known overlapping area in the image based on the position data.

For example, the overlap determination unit (n + 1) connected to the camera n + 1 has its own camera n + 1.
, The object data from the object extraction unit of camera n, and the object data from the object extraction unit of camera n + 2. Then, the position data P (known) of the overlapping range of the imaging area of the camera n and the camera n + 1 is compared with the position data of the object from the object extraction unit of the camera n and the position data of the object from the object extraction unit of the camera n + 1, If the position data of these objects is included in P, it is determined that they are the same object, the object data from the object extraction unit of camera n is not adopted, and the object data from the object extraction unit of camera n + 1 is not used. Selectively output data only. Further, the position data P ′ (known) of the overlapping range of the imaging area of the camera n + 1 and the camera n + 2 is compared with the position data of the object from the object extraction unit of the camera n + 1 and the position data of the object from the object extraction unit of the camera n + 2. When these position data are included in P ′, it is determined that the objects are the same, but the object data from the object extracting unit of camera n + 1 is not output, and this data is output from the overlap determining unit n + 2. . That is, the overlapping range of camera n−1 and camera n → the extraction result of camera n is output by the overlap determination unit n. The overlapping range of camera n and camera n + 1 → camera n + 1
Is output by the overlap determination unit n + 1. The overlapping range of camera n + 1 and camera n + 2 → camera n + 2
Is output by the overlap determination unit n + 2. As a result, the data of the object existing in the overlapping range is output from the camera in the direction of travel among the adjacent cameras, and only one object data is output in response to the fact that the camera is originally one object. it can. It goes without saying that object data in an area that does not overlap with an imaging area of an adjacent camera is output as it is. Such object data includes, for example, the position data P of the overlapping area with the camera n and the overlapping area P ′ with the camera n + 2 in the imaging area of the camera n + 1 for the camera n + 1.
The object data is not included in any of the position data. All the object data from the overlap determination unit 12 is supplied to the incoming / outgoing line vehicle recognition unit 14. The incoming / outgoing line vehicle recognition unit 14 includes a microcomputer and a memory, and sequentially stores input data in the memory. Then, the movement status of each object is tracked based on the correlation between these time-series object data. For example, if there is a moving object at a certain time t even though it does not exist in the past, the object is recognized as a vehicle that has newly entered the driving lane where the camera is installed, and conversely, Is recognized as a vehicle that has departed from the lane when the moving object that has moved and no longer exists after a certain time t. In addition, a stationary object that does not exist in the past but exists at a certain time t is recognized as a falling object on the lane. The result recognized in this way is sent to a management center or the like, and is appropriately notified to each vehicle traveling on the lane.

FIG. 5 is a flowchart showing the overall processing of the road monitoring apparatus according to this embodiment. First, an image is input from each camera (S101), and a stationary object and a moving object are recognized by the object extraction unit 10 (S102). The object data is output to the overlap determination unit 12, and the overlap processing is performed (S103).

FIG. 6 shows a detailed flowchart of the overlap processing in S103. First, the recognition result of the nth camera and the recognition result of the (n + 1) th camera are read (S201). Then, a recognition object of the nth camera and a recognition object of the (n + 1) th camera located in the overlap region of the nth camera and the (n + 1) th camera are extracted (S202). After extracting the object in the overlap area, the object in the overlap area, in which the recognition by the nth camera and the recognition by the (n + 1) th camera overlap, is deleted from the recognition result by the nth camera, and only the recognition object of the (n + 1) th camera is removed. Selectively output (S20
3).

Returning to FIG. 5, when the overlap processing is completed as described above, the recognized object data is supplied to the incoming / outgoing line vehicle recognition unit 14. The incoming / outgoing line vehicle recognition unit 14 extracts the position and speed data of the recognized object (S104), reads out the previous object position and speed stored in the memory (S105), and compares the previous and current object data. (S106).

FIG. 7 shows a detailed flowchart of the matching processing in S106. First, the previous position and speed data of the recognized object are read from the memory (S3).
01). Next, the current position is estimated from the previous position and speed data (S302). This estimation is performed by multiplying the current time interval and the previous time interval by the previous object velocity data. Then, the current recognition object at the estimated position and the previous object are associated with each other based on the shape data (S30).
3) It is determined whether the present object exists at the estimated position (surviving object), whether it exists last time and does not exist this time (disappearing object), or whether it does not exist last time and exists this time (occurring object) (S304). . Further, the position and the vehicle speed of the surviving object and the generated object are stored in the memory (S305). Note that the disappearing object is considered to be a vehicle that has deviated from the lane as described above, and thus the tracking ends without being stored in the memory.

Returning to FIG. 5 again, when the object matching process is completed, the process proceeds to an obstacle / failed vehicle determination process (S107). FIG. 8 shows a flowchart of this processing. First, the position and velocity of the surviving object are read (S401). Then, the history of the position and the vehicle speed of the past surviving object is read from the memory (S402), and the surviving object that is determined to be decelerating from the past vehicle speed history and that is currently stationary is determined to be a malfunctioning vehicle (S403). . Next, when the generated object is present, the position and speed of the object are read out (S404), and the currently stopped generated object is determined as an obstacle (S405).

When the obstacle and the malfunctioning vehicle are determined in this way, the vehicle is judged to enter or exit (S108). FIG.
Is a detailed flowchart of this entry / exit determination. First, the position and velocity data of the disappearing object are read (S501).
Then, the past position and vehicle speed history of the disappearing object are read out (S502), and the out-of-lane determination of the disappearing object and the off-lane position are estimated (S503). The off position is estimated from the position and speed of the disappearing object. On the other hand, the position and speed data of the generated object are read out (S504), and the lane entry determination and the entry position of the generated object are estimated (S505).
The entry position is estimated from the position and speed of the generated object.

When the entry / exit determination of the vehicle is completed as described above, the lane condition is transmitted to the road control system (S109). In the traffic control system that obtained the lane status,
Appropriate information according to the situation is reported to each vehicle traveling on the lane. For example, if there is a faulty vehicle, a warning speed is given to the following vehicle and an appropriate vehicle speed for traveling in line with the lane flow is given to the incoming line vehicle requesting the dispatch of a predetermined repair vehicle. And so on.

<Second Embodiment> In the above embodiment,
Although the camera is arranged along the traveling direction of the road, the camera may be arranged on one side of the road to photograph the width direction of the road surface.

FIG. 10 shows a case where a plurality of cameras are arranged on one side of the road. The cameras A, B, and C having the same focal length are arranged at equal intervals on the road side, and photograph the width direction of the road surface at the same tilt angle. Each imaging region is set so that a part thereof overlaps each other. Even in such an arrangement, erroneous detection is reliably performed for an object existing in the overlap region as in the first embodiment by selectively outputting only one recognition result of an adjacent camera. Can be prevented.

In addition, in such an arrangement, there is an advantage that it is hardly affected by sunlight. That is, aside from the case where the sun is near the zenith in the daytime, if the sun is near the horizon, the shadow of the roadside structure may extend to the road surface, and the object may enter the shadow. FIG.
0 indicates such a situation, and the roadside structure 3
The shadow of 00 extends to the road surface, and the object 100 on the road surface is in the shadow. Then, the camera arranged in the direction of extension of the shade cannot capture the object sufficiently clearly, making it difficult to extract the object. However, other cameras can take an image without being affected by the shadow, so that the object can be taken if the object exists in the overlapping range of the imaging region.

FIG. 11 schematically shows images obtained by adjacent cameras A and B when an object exists in the overlapping area. FIG. 11A is an image of camera A,
FIG. 11B is an image of the camera B. The contrast of the object 100 is not sufficient because the camera B is in the extension direction of the shadow (so-called backlight direction), but the camera A can photograph the object 100 because the direction is different. Therefore, even if the object cannot be extracted by the adjacent camera B, if the object can be extracted by the camera A, the camera B performs the processing by assuming that the object should exist. An object can be reliably detected.

FIG. 12 shows a processing flowchart in this embodiment. The configuration of the present embodiment is similar to the configuration of the first embodiment shown in FIG.
In addition, for convenience of explanation, processing of images A and B obtained by cameras A and B will be described. First, an image A and an image B are input, and an object is extracted from each image (S60).
1). The extracted object data is supplied to the overlap determination unit, and it is determined whether or not the object exists in the overlapping area of the image A and the image B (S602). The determination method is the same as in the first embodiment, and it can be determined based on whether or not the position data of the object belongs to a known overlapping range in advance. If the object exists in the overlapping range, it is determined whether the object exists at each corresponding position of the images A and B (S603). Specifically, from the positional relationship between the position of the camera A and the position of the camera B, there is a 1: 1 correspondence between the position on the image A and the position on the image B.
It is determined whether or not the position of the extracted object in the image A and the position in the image B have this correspondence. Then, when a corresponding object exists in both images (S60
In the case of YES in step 4), the objects extracted from the images A and B are recognized as the same object (S610), and it is recognized that there is an object at the detection position and one recognition result is deleted (S61).
1). If an object cannot be extracted from either image A or image B due to the influence of sunlight as shown in FIG. 10, for example, NO is determined in S604, and it is determined which image has the object. (S605). That is, when there is an object in the image A and no object in the image B despite being present in the overlapping area, it is estimated that the object existing in the image A should also exist in the image B (S608). The result of extracting the image A is added to the result of extracting the image B, and it is recognized that an object is present at the detection position of the image A (S60).
9). Then, the overlap determination unit outputs only one of the extraction results of the two images, and deletes the other extraction result. Note that instead of adding the extraction result of the image A to the image B, only the extraction result of the image A may be output.
If there is an object in image B and no object in image A despite being in the overlapping area, it is estimated that the object present in image B should also be present in image A (S60).
6) The result of extracting the image A is added to the result of extracting the image B, and it is recognized that there is an object at the detection position of the image A (S607).

On the other hand, if no object exists in the overlapping area of the image A and the image B, NO is determined in S602, and then it is determined whether an object exists in the unique area of each image (S612). . The unique region is a region that is not an overlapping region in the imaging region, and means a region that can be photographed only by the camera A or a region that can be photographed only by the camera B. If an object exists in this unique region, it is recognized that there is an object at that position (S613). If there is no object in the unique region, the object is located in the imaging region of camera A and camera B. It is determined that there is not (S614).

As described above, in the present embodiment, even if an object is extracted by only one camera even though the object is in the overlapping range of the imaging region, it is determined that the object exists at the corresponding position of the other camera. Since the processing is performed assuming that the object is on the road, it is possible to reliably detect the road object regardless of the shooting situation.

In the above embodiment, the cameras having the same focal length are arranged at equal intervals and the road surface is photographed at the same tilt angle. However, it is needless to say that the same processing can be performed even when the cameras are different. .

Further, when the camera is arranged on one side of the road as in the second embodiment, there is a blind spot area which cannot be photographed on the road surface due to the angle of view (particularly near the road between the cameras). As shown in FIG. 13, the cameras are alternately arranged on both sides of the roadside, such as the camera A on one roadside, the next camera B on the other roadside, the next camera C again on one roadside, and so on. It is also possible to eliminate the blind spot area,
This makes it possible to more reliably detect a road object.

[0032]

As described above, according to the road monitoring apparatus of the present invention, the image pickup areas of a plurality of cameras can be overlapped with each other to reliably capture an object, and erroneous detection of the object can be eliminated to increase the recognition rate. be able to.

[Brief description of the drawings]

FIG. 1 is an explanatory diagram of a camera arrangement according to a first embodiment of the present invention.

FIG. 2 is an explanatory diagram of an imaging area of a camera A and a camera B;

FIG. 3 is an explanatory diagram of an image of a camera A and a camera B and an explanatory diagram of an overlapping area.

FIG. 4 is a configuration block diagram of an embodiment of the present invention.

FIG. 5 is an overall processing flowchart of the embodiment of the present invention.

FIG. 6 is a flowchart of an overlap process.

FIG. 7 is a flowchart of an object matching process.

FIG. 8 is a flowchart of a process for determining an obstacle and a failed vehicle.

FIG. 9 is a flowchart of lane entry / exit determination processing.

FIG. 10 is an explanatory diagram of a camera arrangement according to a second embodiment of the present invention.

FIG. 11 is an explanatory diagram of images of cameras A and B.

FIG. 12 is an overall processing flowchart of the second embodiment.

FIG. 13 is an explanatory view of another camera arrangement according to the present invention.

[Explanation of symbols]

10 object extraction unit, 12 overlap judgment unit, 14
Incoming / outgoing line vehicle recognition unit, 100 objects (vehicles), 200 overlapping areas, 300 roadside structures.

Claims (3)

[Claims] An on-road monitoring apparatus for monitoring a road by a plurality of image pickup means arranged so that image pickup areas thereof overlap with each other along a traveling direction of a road. Extracting means for extracting an object present in the imaging area; and processing means for selectively outputting an extraction result of the overlapping area by one of the extracting means when the extracted object is present in an overlapping area of an adjacent imaging area. A road surveillance device, comprising: 2. The road monitoring apparatus according to claim 1, further comprising detection means for detecting an obstacle on the road from a plurality of extraction results at different times. 3. An image pickup means provided on a road side for photographing a width direction of a road surface, and said processing means, when an extracted object is present in an overlapping area of an adjacent image pickup area, another adjacent image pickup means. The road monitoring apparatus according to claim 1 or 2, wherein the processing is performed assuming that the object is photographed.