JP2004056497A - Image processing apparatus and method therefor, and vehicle supervision system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processing apparatus and method therefor, and vehicle supervision system, enabling easy tracing of a vehicle. <P>SOLUTION: An image processing apparatus 5 acquires converted images which are analog-to-digital converted from video signals obtained by a plurality of supervisory cameras 2 installed along the vehicle movement direction of a road 10. Thereafter, the image processing apparatus 5 sets division areas with regard to the deformed images, and obtains divided images based on predetermined sampling points. The image processing apparatus 5 synthesizes these divided images so that the road 10 becomes continued, and obtains a synthesized image. Vehicles are detected based on the synthesized image. Thereafter, vehicle trace processing is performed, and after a vehicle image and background image is corrected, an image data output from the image processing apparatus 5 is displayed on a monitor 61. <P>COPYRIGHT: (C)2004,JPO

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to an image processing device, an image processing method, and a vehicle monitoring system used for traffic monitoring and the like.
[0002]
[Prior art]
Conventionally, for example, a monitoring target section such as an expressway or a general road is divided into a plurality of photographing areas, a camera is installed for each of the plurality of photographing areas, and image processing is performed based on a photographed video signal to automatically start a vehicle. An image processing device for detecting is known. In this image processing apparatus, vehicle detection is performed for each of a plurality of photographing regions, and an ID and a vehicle image are transferred in order to track a specific vehicle over the plurality of photographing regions.
[0003]
[Problems to be solved by the invention]
However, with the conventional image processing apparatus, it is still difficult to track a specific vehicle over a plurality of photographing areas. This will be described with reference to FIG. First, the particular vehicle 100 at the position of _{P 1} at time _{t 1} is moved at time _{t 2} proceeds to the position of _{P 2.} In the conventional image processing apparatus, for example, stores vehicle image of particular vehicle 100 at time t _1, then the detection of the vehicle when the time reaches t _2. The particular vehicle 100 is determined to have moved to the position of P ₂ by the detected template matching or the like and an image with a stored vehicle image of the vehicle. In this manner, the conventional image processing apparatus tracks the vehicle.
[0004]
Then particular vehicle 100 has moved to the position of P _3. In this case, the image processing apparatus performs the template matching or the like as in the case where a specific vehicle 100 has moved to the position of P _2. However, the particular vehicle 100 at the position of P ₃ and extends over the first and second imaging regions 300a, 300b, to determine whether a particular vehicle 100 even when subjected to template matching, etc. Has become difficult. For this reason, it is difficult for the image processing apparatus to assign the ID or the like to the vehicle 100 even if the ID or the like is transferred to the second imaging area 300b, and it is difficult to track the vehicle.
[0005]
SUMMARY An advantage of some aspects of the invention is to provide an image processing apparatus, an image processing method, and a vehicle monitoring system that can easily track a vehicle.
[0006]
[Means for Solving the Problems]
In order to achieve such an object, an image processing apparatus according to the present invention receives a plurality of video signals obtained by continuously photographing a road by a plurality of photographing means installed along a vehicle traveling direction on the road. A / D conversion means for A / D converting the video signals, and a divided area obtained by dividing the road in the vehicle traveling direction with respect to a converted image obtained by A / D conversion by the A / D conversion means are set. Extracting means for extracting a divided image in each divided region; synthesizing means for synthesizing the divided images extracted by the extracting means so that the road is continuous in the vehicle traveling direction; and a synthesizing image synthesized by the synthesizing means. And a vehicle detecting means for detecting the vehicle.
[0007]
In addition, the image processing method according to the present invention is configured such that a plurality of video signals obtained by continuously photographing a road by a plurality of photographing means installed along a vehicle traveling direction on a road are input, and the video signals are converted to A An A / D conversion step of performing A / D conversion, and a divided area obtained by dividing the road in the vehicle traveling direction with respect to the converted image obtained by the A / D conversion in the A / D conversion step are set. Extraction step, a synthesis step of synthesizing the divided images extracted by the extraction step so that the road is continuous in the vehicle traveling direction, and a vehicle detection step of detecting a vehicle based on the synthesis image synthesized by the synthesis step It is comprised including.
[0008]
Further, a vehicle monitoring system according to the present invention includes a plurality of photographing means installed along a vehicle traveling direction on a road, and the above-described image processing device.
[0009]
Further, the vehicle monitoring system according to the present invention is characterized by further comprising a display means for displaying image data from the image processing device as an image.
[0010]
According to these inventions, the detection of the vehicle is performed based on the synthesized image synthesized so that the road is continuous in the vehicle traveling direction, so that the vehicle detection is performed on one wide image. For this reason, even if a vehicle exists at the joint of the photographing areas, there is no joint in the composite image itself, and tracking of the vehicle does not become difficult due to the joint of the image itself in the tracking processing performed after the vehicle detection. . Therefore, tracking of the vehicle can be facilitated.
[0011]
Further, in the image processing apparatus according to the present invention, the synthesizing unit performs the synthesizing so that the road in the synthesized image is oriented in the vertical direction or the horizontal direction.
[0012]
Further, the image processing method according to the present invention is characterized in that in the combining step, the combining is performed such that the road faces in the vertical direction or the horizontal direction in the combined image.
[0013]
In these cases, the composite image is as viewed from directly above the road, so that it is possible to provide an image that is easy for a monitor or the like to see.
[0014]
Further, in the image processing device according to the present invention, when the vehicle detected by the vehicle detection unit is newly detected from the composite image, the vehicle image of the vehicle is corrected and stored, and the vehicle image is corrected according to the movement of the vehicle. And displaying the corrected vehicle image at the moving position.
[0015]
Also, in the image processing method according to the present invention, when the vehicle detected in the vehicle detection step is newly detected from the composite image, the vehicle image of the vehicle is corrected and stored, and the vehicle image is corrected according to the movement of the vehicle. And displaying the corrected vehicle image at the moving position.
[0016]
In these cases, the stored vehicle image is displayed at the vehicle movement position, so that it is not necessary to correct the vehicle image every time a vehicle is detected. Therefore, the processing can be performed quickly.
[0017]
Further, an image processing apparatus according to the present invention is characterized in that a background image is corrected in accordance with a correction of a vehicle image.
[0018]
Further, an image processing method according to the present invention is characterized in that a background image is corrected according to a correction of a vehicle image.
[0019]
In these cases, the inconsistency with the background image, etc., generated when the vehicle image is corrected is eliminated. Therefore, at the time of display, it is possible to remove a sense of discomfort due to the correction of the vehicle image.
[0020]
Further, in the vehicle monitoring system according to the present invention, at least two or more photographing means are provided for each region for photographing each region of the road.
[0021]
In this case, since each area of the road is photographed from different angles by two or more photographing means, for example, it is possible to detect an ordinary car that is hidden behind a shadow of a large car or the like. Therefore, the detection accuracy of the vehicle can be improved.
[0022]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In each of the drawings, the same elements are denoted by the same reference numerals, and redundant description will be omitted. Also, the dimensional ratios in the drawings do not always match those described.
[0023]
An image processing device, an image processing method, and a vehicle monitoring system according to an embodiment will be described.
[0024]
FIG. 1 shows a configuration diagram of a vehicle monitoring system according to the present embodiment. As shown in FIG. 1, the vehicle monitoring system is a system that monitors the traffic condition of a road, and is used for a system that detects a traffic abnormality in the tunnel 1, for example. The vehicle monitoring system includes a plurality of monitoring cameras 2.
[0025]
The monitoring camera 2 is a plurality of photographing units installed along the vehicle traveling direction on the road 10, and photographs the vehicle 11 traveling on the road 10 in the tunnel 1. These surveillance cameras 2 are installed at regular intervals in the tunnel 1 so as to continuously photograph the road 10. The installation interval of the monitoring camera 2 is, for example, about 150 to 200 m.
[0026]
For example, the surveillance camera 2 is installed in the upper part of the tunnel 1 and photographs the vehicle 11 traveling on the road 10 from obliquely forward or rearward. Note that not all of the surveillance cameras 2 need be installed above, and the surveillance cameras 2 existing at the entrance of the monitoring target area may be installed above and other surveillance cameras 2 may be installed on the side.
[0027]
Each monitoring camera 2 is connected to a distributor 3. The distributor 3 is installed in a facility 4 built near the tunnel 1, and inputs a captured image of each monitoring camera 2 and outputs the captured image to the image processing device 5.
[0028]
The image processing device 5 receives the image of the surveillance camera 2 and performs image processing, and transmits the processed image to the control center 6. The image processing device 5 includes, for example, an analog-to-digital (A / D) converter, a video frame memory, an image processing processor, and a signal output interface.
[0029]
The A / D converter is an A / D converter for inputting video signals of the monitoring camera 2 and A / D converting the video signals. For example, a video A / D converter is used. The video frame memory stores the converted image data of a plurality of areas. The image data is a set of pixels having a grayscale value and a color signal for each of the x-axis and y-axis coordinate points, and the grayscale of the pixel is, for example, 8-bit 256 gradations.
[0030]
The image processing processor performs image processing on image data stored in the video frame memory. The signal output interface is for outputting image data subjected to image processing to the outside.
[0031]
In the control center 6, the image data transmitted from the image processing device 5 is input to the monitor 61, and the captured image in the tunnel 1 is displayed through the monitor 61. Through this image, the monitoring state of the vehicle in the tunnel 1 can be visually monitored by the observer. The monitor 61 is a display unit that displays image data from the image processing apparatus 5 as an image, and preferably has a horizontally or vertically long display area, for example, a projector.
[0032]
A vehicle height sensor 7 may be provided in the tunnel 1. The vehicle height sensor 7 is a vehicle height detection unit that detects the height of the vehicle 11 and is installed above the road 10. As the vehicle height sensor 7, for example, an ultrasonic sensor is used. The ultrasonic sensor emits ultrasonic waves to the vehicle 11 from above and detects a reflected wave reflected by the vehicle 11 to detect an upper end position of the vehicle 11.
[0033]
The vehicle height sensor 7 is installed at the entrance of the monitoring target section in the vehicle monitoring system. For example, when there is no confluence in the monitoring target section, it is installed only at the entrance of the monitoring target section.
[0034]
As the vehicle height sensor 7, a sensor that performs stereo image processing using two photographing cameras, a laser range finder, a radio range finder, or the like may be used. Any device other than the ultrasonic sensor may be used as long as the height can be detected.
[0035]
Next, operations of the vehicle monitoring system and the image processing device 5 will be described.
[0036]
In FIG. 1, a vehicle 11 traveling on a road 10 is photographed by a monitoring camera 2, and a video signal is input from each monitoring camera 2 to a distributor 3. The distributor 3 inputs the video signal of the monitoring camera 2 to the image processing device 5 as appropriate.
[0037]
On the other hand, when the vehicle height sensor 7 is provided, the vehicle height of the vehicle 11 traveling on the road 10 is detected by the vehicle height sensor 7, and the detection signal is input to the image processing device 5. Then, the image processing device 5 performs image processing of the video signal.
[0038]
FIG. 2 shows a flowchart of the image processing in the image processing device 5.
[0039]
As shown in S1 of FIG. 2, each video signal input to the image processing device 5 is A / D converted. A / D conversion is a process of converting an analog image of a video signal into a digital image, and the converted image data after conversion is stored in a video frame memory. The converted image is, for example, an image itself when the road 10 is viewed from the position of the monitoring camera 2 as shown in FIG. In the converted image 20, the vehicles 11a to 11c traveling on the road 10 are photographed obliquely from behind.
[0040]
After the A / D conversion, the processing shifts to S2 in FIG. 2, and the extraction processing is performed. The extraction process is a process of setting a divided area obtained by dividing the road 10 in the vehicle traveling direction for each A / D converted image, and extracting a divided image in each divided area. As shown in FIG. 4, the divided area 30 is set by dividing the road 10 into a predetermined length along the vehicle traveling direction. Further, one divided area 30 is set for each converted image of each monitoring camera 2. Then, as shown in FIG. 5, each of the divided areas 30 is set without overlapping each other. Note that the divided areas 30 do not necessarily need to be set without overlapping, and even if they are overlapped, the overlapped portion may be cut in a later process. In the following description, it is assumed that the divided areas 30 are set without overlapping.
[0041]
The divided image is formed by extracting pixel data of sampling points set in advance in the divided area 30. FIG. 6 is a schematic explanatory diagram of the sampling points set in the divided area 30. As shown in FIG. 6, the sampling points 31 are set in a grid pattern along the vehicle traveling direction and the road crossing direction of the road 10. By setting the sampling points 31 in this manner, an image of the road 10 in a plan view can be obtained by simply arranging the pixel data at the sampling points 31 as pixel data in a coordinate system in which the vehicle traveling direction and the road crossing direction are orthogonal. Can be easily created.
[0042]
After the extraction of the divided images, the processing shifts to S3 in FIG. 2, and the combining processing is performed. The synthesizing process is a process of synthesizing the divided images extracted by the extracting process so that the road 10 is continuous in the vehicle traveling direction. Also, here, the composition is performed such that the road 10 faces in the vertical direction or the horizontal direction in the composite image.
[0043]
FIG. 7 shows an example of a combined image obtained by combining the road 10 in the horizontal direction. The composite image 40 is obtained by connecting the divided images of the respective divided regions 30 in the vehicle traveling direction into one image. For example, the composite image 40 is obtained by setting and representing pixel data of each divided image created based on different video signals as pixel data of the same coordinate system. It is desirable that the composite image data is configured by storing each divided image data in one memory. Since the divided areas 30 are set without being overlapped with each other, in the combining processing here, the combining can be easily performed without considering the overlapping portion.
[0044]
Further, as shown in FIG. 7, the composite image 40 looks like the road 10 is viewed from directly above. In order to obtain such a combined image 40, it is necessary not only to combine the divided regions 30 shown in FIG. 4 but also to deform each divided region 30 by calculation as viewed from directly above. However, in the present embodiment, since the sampling points 31 are set as shown in FIG. 6, the transformation is performed together with the extraction, and the transformation processing is unnecessary.
[0045]
After the combination, the process proceeds to S4 in FIG. 2, and a vehicle detection process for detecting the vehicle 11 based on the combined image combined by the combination process is performed. The vehicle detection process is performed by, for example, a background difference method in which a background image provided in the image processing device 5 and a composite image 40 are compared in advance.
[0046]
Then, the flow shifts to S5, where it is determined whether or not the vehicle 11 detected by the vehicle detection processing is newly detected from the composite image. If it is determined that the detected vehicle 11 is not newly detected from the composite image, the process proceeds to S8. On the other hand, when it is determined that the vehicle 11 is newly detected from the composite image, the process proceeds to S6. As long as there is no merging in the monitoring target section, the vehicle 11 is newly detected near the entrance of the monitoring target section in the composite image. Therefore, the vehicle 11 that has newly entered the monitoring target section may be detected based only on the data from the monitoring camera 2 provided at the entrance. Further, when there is a merger in the monitoring target section, a new vehicle 11 may be detected based only on data from the monitoring camera 2 installed at the entrance and the merger point.
[0047]
In S6, a vehicle correction process for correcting the newly detected vehicle image of the vehicle 11 according to the vehicle height is performed. If the correction is not performed here, the vehicle 11 will be as shown in FIG. As shown in FIG. 8, the vehicles 11b and 11c traveling in a location far from the imaging position tend to have larger distortion than the vehicle 11a traveling in a location near the imaging position. In particular, the distortion of the vehicle 11c such as a tall truck is large.
[0048]
For this reason, the correction is performed in the process of S6 in FIG. The vehicle correction process is performed, for example, based on the vehicle height data detected by the vehicle height sensor 7, and is performed such that the higher the vehicle height of the vehicle 11, the lower the vehicle height of the vehicle 11. By performing the vehicle correction process in this manner, each of the vehicles 11a to 11c shown in FIG. 8 is corrected as shown in FIG.
[0049]
After the correction, the process proceeds to S7. In S7, the corrected vehicle image is stored. That is, as shown in FIG. 9, vehicle images of the vehicles 11a to 11c from which distortion has been removed are stored.
[0050]
Then, the process proceeds to S8 in FIG. 2, and the tracking process is performed. The tracking process determines, for example, the position of the vehicle 11 stored as the vehicle image by matching the image of the vehicle 11 currently traveling in the monitoring target section with the previously stored vehicle image. This is the processing to be performed. The tracking processing is performed on the vehicles 11a to 11c stored as the vehicle images in S7 when they move and the next position data is obtained.
[0051]
Thereafter, in S9, the vehicle image stored at the position of the vehicle 11 is pasted. As a result, the vehicle image stored at the position of the vehicle 11 is displayed. In addition, regarding the vehicles 11a to 11c stored as the vehicle images in S7, when they later move to another position, the vehicle images are pasted at the moved positions.
[0052]
Then, the flow shifts to S10, where a background correction process is performed. The background correction process is a process of correcting a background image according to a correction of a vehicle image. For example, in the case of the vehicle 11c shown in FIG. 9, the background portion in the area 50 occupied by the vehicle 11c before correction is corrected. This is because, if only the vehicle image is corrected and the background image is not corrected, when the image is displayed, the background image may feel uncomfortable.
[0053]
Thereafter, image data obtained through a series of these processes is transmitted to the control center 6 and displayed on the monitor 61 of the control center 6 as an image. A series of processing in FIG. 2 is repeatedly performed at predetermined time intervals.
[0054]
As described above, according to the present embodiment, since the detection of the vehicle 11 is performed based on the composite image 40 composed so that the road 10 is continuous in the vehicle traveling direction, the vehicle detection is performed for one wide image. Will do. For this reason, even if the vehicle 11 exists at the joint of the photographing areas, the joint does not exist in the composite image 40 itself, and the tracking of the vehicle becomes difficult due to the joint of the image itself in the tracking processing performed after the vehicle detection. There is no. Therefore, tracking of the vehicle 11 can be facilitated.
[0055]
In addition, since the composite image 40 looks as if the road 10 is viewed from directly above, it is possible to provide an image that is easy for a monitor or the like to see.
[0056]
Further, since the stored vehicle image is displayed at the vehicle moving position, it is not necessary to correct the vehicle image every time the vehicle 11 is detected. Therefore, the processing can be performed quickly.
[0057]
In addition, the inconsistency with the background image that occurs when the vehicle image is corrected is eliminated. Therefore, at the time of display, it is possible to remove a sense of discomfort due to the correction of the vehicle image.
[0058]
In addition, since the sampling points 31 in the divided area 30 are arranged in a grid pattern along the vehicle traveling direction and the road crossing direction of the road 10, the sampling points 31 are extracted and deformed. This eliminates the need for deformation processing.
[0059]
Further, according to the present embodiment, since each of the divided areas 30 is set without overlapping each other, it is possible to easily combine the divided areas 30 without considering the overlapping portion.
[0060]
As described above, the invention made by the inventor has been specifically described based on the embodiment. However, the invention is not limited to the above embodiment. For example, when the vehicle is not accurately detected in the vehicle detection process, the vehicle may be processed as one of the background images to display an uncorrected vehicle image, or the vehicle The position of a vehicle that has not been detected based on the past motion vector and the movements of the preceding and following vehicles may be predicted, and the corrected and stored vehicle image may be displayed at that position. Furthermore, at least two or more surveillance cameras 2 may be provided for each region in order to photograph each region of the road 10. In this case, since each area of the road 10 is photographed from different angles by the two or more surveillance cameras 2, for example, it is possible to detect a normal car hidden behind a large car or the like. Therefore, the detection accuracy of the vehicle 11 can be improved.
[0061]
【The invention's effect】
As described above in detail, according to the present invention, it is possible to provide an image processing apparatus, an image processing method, and a vehicle monitoring system that can easily track a vehicle.
[Brief description of the drawings]
FIG. 1 is a configuration diagram of a vehicle monitoring system according to an embodiment.
FIG. 2 is a flowchart illustrating a process of the image processing apparatus illustrated in FIG. 1;
FIG. 3 is a diagram illustrating an example of a deformed image.
FIG. 4 is a diagram illustrating an example of a divided area.
FIG. 5 is a schematic explanatory diagram of sampling points set in a divided area.
FIG. 6 is a diagram showing a state of sampling points set in the divided areas shown in FIG.
FIG. 7 is a diagram illustrating an example of a combined image obtained by combining roads in a horizontal direction.
FIG. 8 is a diagram illustrating an example of a case where a vehicle is displayed without performing a vehicle correction process.
FIG. 9 is a diagram illustrating an example of a case where a vehicle is displayed by performing a vehicle correction process.
FIG. 10 is a diagram illustrating an example of a conventional vehicle tracking process.
[Explanation of symbols]
2 surveillance camera, 5 image processing device, 7 vehicle height sensor, 10 road, 11 vehicle, 20 converted image, 30 divided area, 31 sampling point, 32 divided image, 40 composite image, 61 Monitor.

Claims (11)

A / D conversion means for inputting a plurality of video signals obtained by continuously photographing the road by means of a plurality of photographing means installed along the vehicle traveling direction on the road, and A / D converting the video signals; ,
Extracting means for setting divided areas obtained by dividing the road in the vehicle traveling direction with respect to the converted image obtained by A / D conversion by the A / D converting means, and extracting divided images in each divided area;
Combining means for combining the divided images extracted by the extracting means so that the road is continuous in the vehicle traveling direction;
Vehicle detection means for detecting the vehicle based on the synthesized image synthesized by the synthesis means,
An image processing apparatus comprising: The image processing apparatus according to claim 1, wherein the combining unit performs the combining such that the road is oriented in a vertical direction or a horizontal direction in the composite image. When the vehicle detected by the vehicle detection means is newly detected from the composite image, the vehicle image of the vehicle is corrected and stored,
According to the movement of the vehicle, to display the corrected vehicle image at the moving position,
The image processing apparatus according to claim 2, wherein: The image processing apparatus according to claim 3, wherein a background image is corrected according to the correction of the vehicle image. An A / D conversion step of inputting a plurality of video signals obtained by continuously photographing the road by a plurality of photographing means installed along the traveling direction of the road, and A / D converting the video signals; ,
An extraction step of setting a divided region obtained by dividing the road in the vehicle traveling direction with respect to a converted image obtained by A / D conversion in the A / D conversion process, and extracting a divided image in each divided region;
A combining step of combining the divided images extracted in the extracting step so that the road is continuous in the vehicle traveling direction;
A vehicle detection step of detecting the vehicle based on the synthesized image synthesized by the synthesis step,
An image processing method comprising: 6. The image processing method according to claim 5, wherein in the combining step, the combining is performed such that the road faces in a vertical direction or a horizontal direction in the composite image. When the vehicle detected by the vehicle detection step is newly detected from the composite image, the vehicle image of the vehicle is corrected and stored,
According to the movement of the vehicle, to display the corrected vehicle image at the moving position,
7. The image processing method according to claim 6, wherein: 8. The image processing method according to claim 7, wherein a background image is corrected according to the correction of the vehicle image. A plurality of photographing means installed along the vehicle traveling direction on the road;
An image processing apparatus according to claim 1,
A vehicle monitoring system comprising: The vehicle monitoring system according to claim 9, wherein at least two photographing units are provided for each region for photographing each region of the road. The vehicle monitoring system according to claim 9, further comprising a display unit configured to display image data from the image processing device as an image.

Images