JP6385851B2 - Road monitoring device, road monitoring system, method and program - Google Patents

Description

  Embodiments described herein relate generally to a road monitoring device, a road monitoring system, a method, and a program.

  Conventionally, in street traffic, the presence of road parking is often the cause of traffic jams. On the other hand, the police are conducting the control based on the results detected by looking around with human hands and the monitoring cameras and sensors installed on the road. In patent document 1, the structure which reads a vehicle number with the surveillance camera installed in the parking prohibition area vicinity and collects violation vehicle information is proposed.

JP 2002-197589 A

  However, in the prior art, monitoring is possible only at the place where the surveillance camera is installed (pinpoint), and monitoring in real time on the entire road to be controlled is also from the viewpoint of equipment installation cost, etc. It was difficult.

  The present invention has been made in view of the above, and an object thereof is to provide a road monitoring device, a road monitoring system, a method, and a program capable of performing real-time monitoring over the entire road to be controlled. Yes.

The road monitoring apparatus according to the embodiment is mounted on a vehicle, and the imaging unit performs imaging in at least a front direction or a rear direction of the vehicle, and sequentially outputs imaging data. The position / time information calculation unit outputs position information and time information of the vehicle corresponding to the imaging data. The determination unit determines whether or not a predetermined monitoring target is included in the image corresponding to the imaging data. When a predetermined monitoring target is included in the image, the extraction unit extracts the imaging data corresponding to a timing that is a predetermined time after the time when the monitoring target disappears from the image . Transmission unit, the extracted imaging data was in association with the position information and time information, and transmits by radio to the central unit.

FIG. 1 is a schematic configuration block diagram of a road monitoring system according to an embodiment. FIG. 2 is a schematic functional configuration block diagram of a road monitoring device and a central device mounted on each vehicle. FIG. 3 is a process flowchart of the road monitoring apparatus. FIG. 4 is a processing flowchart of the central apparatus.

Next, embodiments will be described with reference to the drawings.
FIG. 1 is a schematic configuration block diagram of a road monitoring system according to an embodiment.
As shown in FIG. 1, the road monitoring system 10 is connected to a central apparatus (management server) 11 and the central apparatus 11 via a radio base station 12 and a communication network 13 so that communication is possible. A road monitoring device 15 mounted on a plurality of vehicles 14 such as a taxi or a private car.

FIG. 2 is a schematic functional configuration block diagram of a road monitoring device and a central device mounted on each vehicle.
The road monitoring device 15 is a pair of imaging devices (cameras) 21 that captures images in front of the traveling direction in synchronization with each other while the host vehicle 14 is moving, and outputs imaging data constituting a stereo image (stereo video), and imaging data On the road monitoring of other parked vehicles, construction sites, road falling objects, etc., based on the imaging data constituting the stereo image (stereo video) input from the memory unit 22 and the pair of imaging devices 21 An image processing unit 23 that performs processing for detecting the object X, a probe information transmission unit 24 that transmits the processing result of the image processing unit 23 as probe information to the central apparatus 11 via the radio base station 12 and the communication network 13, And a GPS / clock unit 25 that outputs position data and timing data of the host vehicle 14.
In this embodiment, a pair of imaging devices functioning as a stereo camera is provided as the imaging device. However, although the distance measurement accuracy is slightly reduced, a single imaging device may be used. Further, instead of or in addition to imaging in the front in the traveling direction, an imaging device that performs imaging in the rear in the traveling direction can be provided.

  Here, the image processing unit 23 includes a monitoring target determination unit 31 that detects a road monitoring object X from a pair of images (video) corresponding to imaging data constituting a stereo image, imaging position information corresponding to the imaging data, and A position / time information calculation unit 32 that calculates imaging time information, a vehicle number reading unit 33 that reads a vehicle number (number plate) when the road monitoring object X is another parked vehicle 14X, and imaging data Among them, a video extraction unit 34 that extracts an image (video) to be transmitted as probe information to the central apparatus 11 is provided.

  The central device 11 includes a probe information receiving unit 41 that receives probe information from the road monitoring device 15, a probe information database unit 42 that manages the received probe information, road construction information that has been reported, and the like as a database, a probe A search unit 43 that searches the information database unit 42 and a determination unit 44 that determines road monitoring based on the search result of the search unit 43 are provided.

  In the above configuration, the central device 11 constitutes a part of a road information providing system and a traffic control system.

Next, the operation of the embodiment will be described.
FIG. 3 is a process flowchart of the road monitoring apparatus.
First, the imaging device 21 of the road monitoring device 15 starts imaging and accumulates imaging data, and thus an image, in the memory unit (step S11).

Thereby, the monitoring target determination unit 31 of the image processing unit 23 determines whether or not the road monitoring target X is detected from a pair of images (videos) corresponding to the imaging data constituting the stereo image (step S12). .
Here, the on-road monitoring object X is detected by pattern matching, feature extraction, or the like.

  When the road monitoring object X is not detected in the determination in step S12 (step S12; No), the monitoring target determination unit 31 repeats the determination process in step S12 and effectively enters a standby state.

  If the road monitoring object X is detected in the determination in step S12 (step S12; Yes), the position / time information calculation unit 32 acquires time information and position information from the GPS / clock unit 25, and performs imaging. The position and time are calculated (step S13).

Subsequently, the monitoring target determination unit 31 determines whether the road monitoring object X is a vehicle, a road construction site, or a road falling object (step S14).
Specifically, as a vehicle detection process, for example, the technique shown in ““ A Trainable System for Object Detection ”, Papageorgiou et al., IJCV-2000” is used.

  In this case, since the parked vehicle generally exists on the left side of the own vehicle in the case of left-hand traffic and on the right side of the own vehicle in the case of right-hand traffic, only the detected vehicle corresponding to such a position in the image is present. It is also possible to configure so as to be calculated. As a result, the amount of calculation can be suppressed.

Further, when the road monitoring object X is a road construction site, the road construction site guide plate and the like may be detected by feature extraction in the same manner as the feature extraction of the road sign.
Furthermore, when the road monitoring object X is a road falling object, patterns for pattern matching of assumed road falling objects are prepared, and it is determined by matching with them, or to some extent such as wood and old tires. It is possible to configure a fallen object on the road having a size to be detected as a fallen object on the road from the size and the point of not moving.

  If it is determined in step S14 that the road monitoring object X is a vehicle (step S14; vehicle), a license plate is extracted from the image corresponding to the imaging data, and the vehicle number is read (step S15).

Subsequently, it is determined whether or not the (identical) vehicle as the road monitoring object X has disappeared from the (image) frame (step S16).
In the determination in step S16, when the (identical) vehicle that is the road monitoring object X has not yet disappeared from the (image) frame (step S16; No), the process proceeds to step S13 again. Similar processing is performed.

  In the determination in step S16, when the (identical) vehicle that is the road monitoring target X disappears from the (image) frame (step S16; Yes), the video extraction unit 34 of the image processing unit 23 performs the central device 11. The vehicle image to be transmitted is extracted (step S17). In this case, the image processing unit 23 is an image including a license plate among the imaging data including the vehicle, and is a timing (for example, a time that goes back a predetermined time from the time when the vehicle disappears from the frame). The imaging data corresponding to the image 1 second before) is extracted.

  This is because a license plate is necessary when determining parking violations later, and it is necessary to include a license plate more clearly, and imaging data closer to the vehicle is preferable. is there.

  Note that the video extraction unit 34 of the image processing unit 23 may extract imaging data corresponding to a plurality of images as necessary. For example, the imaging data corresponding to the image including the whole image and the imaging data corresponding to the image in which the license plate is clearly displayed as described above are included, or the parking position is also determined from the image. Imaging data corresponding to an image in which the background of the parked vehicle is also reflected may be included so as to be specified.

  Subsequently, the probe information transmission unit 24 transmits the processing result of the image processing unit 23 as probe information to the central apparatus 11 via the radio base station 12 and the communication network 13 (step S18). Then, the process proceeds to step S25.

  In the determination in step S14, when the road monitoring object X is a road construction site (step S14; construction site), the (same) road construction site that is the road monitoring object X is extracted from the (image) frame. It is determined whether or not it has disappeared (step S19).

  If it is determined in step S19 that the (same) road construction site as the road monitoring object X is continuously detected and has not yet disappeared from the (image) frame (step S19; No), the process is re-stepped. The process proceeds to S13, and the same processing is performed thereafter.

  In the determination of step S19, when the road construction site (estimated to be the same) that is the road monitoring object X is no longer detected (image), becomes a normal road scenery and disappears from the frame (step S19). ; Yes), the image processing unit 23 extracts a road construction site image to be transmitted to the central apparatus 11 (step S20). In this case, the image processing unit 23 extracts the imaging data corresponding to the respective images including the road construction start position and the road construction end position from the imaging data including the road construction site. It is preferable to do this.

  Subsequently, the probe information transmission unit 24 captures image data corresponding to each image including the road construction start position and the road construction end position, which are the processing results of the image processing unit 23, the road construction site start position, and the road construction. The on-site end position and the time when these images are taken are transmitted as probe information to the central apparatus 11 via the radio base station 12 and the communication network 13 (step S21). Then, the process proceeds to step S25.

Further, in the determination in step S14, when the road monitoring object X is a road falling object (step S14; road falling object), the road monitoring object X (estimated to be the same) is the road monitoring object X. It is determined whether or not the image has disappeared from the frame (step S22).
In this case, since the road monitoring device 15 includes a stereo camera, the distance to the falling object on the road can be grasped, so that the size of the falling object on the road can be easily grasped.

  In the determination of step S22, when the road fall object which is the road monitoring object X is continuously detected and has not yet disappeared from the (image) frame (step S22; No), the end position of the construction site is detected. Therefore, the process again proceeds to step S13, and the same process is performed thereafter.

  In the determination of step S22, when a road fall object that is the road monitoring object X is not detected and becomes a normal road scenery and disappears from the frame (step S22; Yes), the image processing unit 23 The road falling object image to be transmitted to the central device 11 is extracted (step S23). In this case, the image processing unit 23 converts the image data including the falling object on the road to an image at a timing (for example, one second before) that is a predetermined time after the time when the falling object on the road disappears from the frame. Corresponding imaging data may be extracted, or imaging data corresponding to an image having the largest amount of falling objects on the road (the largest area in the image) may be extracted.

Subsequently, the probe information transmitting unit 24 wirelessly uses the imaging data corresponding to the image including the falling object on the road as the processing result of the image processing unit 23, the position of the falling object on the road, and the time when the image is captured as the probe information. It transmits to the central apparatus 11 via the base station 12 and the communication network 13 (step S24). Then, the process proceeds to step S25.
In this case, when falling objects on the road are detected in a wide range, the imaging data corresponding to the falling start position and falling end position where the falling of the falling objects on the road is detected can be transmitted. .

Subsequently, the image processing unit 23 determines whether or not the engine has been stopped by the operation of the driver (step S25).
If it is determined in step S25 that the engine has not stopped yet, the process proceeds to step S12 again in order to continue road monitoring, and the same process is repeated thereafter.

  If it is determined in step S25 that the engine has stopped, all the processes are terminated. If the engine is stopped during transmission of probe information, if there is a lot of probe information to be transmitted, transmission is temporarily stopped, and if there is little probe information to be transmitted, transmission is performed with power from the battery. It is also possible to configure so as to end the processing after continuing.

Next, processing of the central device will be described.
FIG. 4 is a processing flowchart of the central apparatus.
First, when receiving the probe information from the road monitoring device 15, the probe information receiving unit 41 of the central device 11 performs a process of storing the received probe information in the probe information database unit 42 (step S31).

Subsequently, the search unit 43 of the central apparatus 11 determines a monitoring target corresponding to the received probe information (step S32).
If it is determined in step S32 that the monitoring target is a vehicle (step S32; vehicle), the search unit 43 searches the probe information database unit 42 and searches for probe information of the same vehicle number (step S33). .

Subsequently, the determination unit 44 determines whether there is probe information of the same vehicle number based on the search result of the search unit 43 (step S34).
In the determination in step S34, when there is no probe information of the same vehicle number (step S34; No), the determination unit 44 ends the process.

If there is probe information of the same vehicle in the determination in step S34 (step S34; Yes), the determination unit 44 compares the time between the probe information and detects it as a parking time (step S35).

Next, the determination unit 44 compares the detected parking time with the determination reference time, and determines whether or not the determination reference time has elapsed (step S36).
As a result of the determination in step S36, when the detected parking time is less than the determination reference time (step S36; No), the determination unit 44 ends the process.

If it is determined in the step S36, if the parking time has been detected it is equal to or higher than the determination reference time (step S36; Yes), determination unit 44 determines that the vehicle of the vehicle number is a parking violation vehicle, police A process of notifying a predetermined place such as the above is performed (step S37). Then, the process associated with the reception of the current probe information is terminated.

  As a result, it becomes possible to quickly control the illegally parked vehicle that is causing the traffic jam.

  If it is determined in step S32 that the monitoring target is a road construction site (step S32; construction site), the search unit 43 searches the probe information database unit 42 and searches for road construction reports (step S38). ).

Then determination section 44 determines whether or not there is a construction notification contents match (step S39). Here, the content matches means that the location, date, and time zone of the road work match (= actually within the time zone).
As a result of the determination in step S39, if there is a construction report with the same content (step S39; Yes), the determination unit 44 terminates the process as having no problem.

As a result of the determination in step S39, when the contents do not match, for example, when construction is being performed even after the scheduled time has elapsed (step S39; No), the determination unit 44 records the mismatch. At the same time, a construction notification (extension) process for changing the display on the electric bulletin board or the like or notifying the related department is performed (step S40), and the process associated with the reception of the probe information is terminated.

  Thus, in the road monitoring system 10 constituting the traffic control system, the actual time and place (especially where the place moves with time) with respect to the road construction time and place registered in advance. Traffic controllers can quickly grasp (almost in real time) and provide necessary information and monitoring.

If it is determined in step S32 that the monitoring target is a fallen object on the road (step S32; a fallen object on the road), the search unit 43 searches the probe information database unit 42, and the determination unit 44 has already been treated. It is determined whether or not the object falls on the road (step S41). Here, the handling completed means that the object has fallen on the road for which display on an electric bulletin board or the like, notification to a related department, or the like has already been performed.

As a result of the determination in step S41, if it is a fallen object on the road that has already been dealt with (step S41; Yes), the determination unit 44 terminates the process with no problem. In this case, when the scattering range is widened, it can be handled as another fallen object on the road.

If the result of determination in step S41 is a fallen object on the road that has not yet been dealt with (step S41; No), the determination unit 44 changes the display of the electric bulletin board or the like, or notifies the related departments. Or the like (step S42), and the process associated with the reception of the current probe information is terminated.

  As described above, according to the road monitoring system 10, falling objects on the road can be detected in real time, and the traffic controller can be used for preventing traffic accidents and removing the cause of traffic jams.

In the above description, the case where road monitoring is performed in real time has been described. However, the time axis to be compared is set to a long time (month level), for example, the planting is extended, and it becomes an obstacle to travel or hides signs / signals. It is also possible to adopt a configuration for detecting that the situation has occurred.
As a result, the planting can be cut quickly, which can be used for preventing traffic accidents and eliminating the cause of traffic jams.
In the above description, a configuration is adopted in which image storage is performed simultaneously with the start of imaging (step S11). However, in step S13, the imaging position / time is calculated and then stored in association with this. It is also possible to do. By adopting such a configuration, it is possible to reduce the capacity of the stored image and store only more important images.

  The road monitoring device of the present embodiment includes a control device such as a CPU, a storage device such as a ROM (Read Only Memory) and a RAM, an external storage device such as an HDD and a CD drive device, a display device such as a display device, It has an input device such as a keyboard and a mouse, and has a hardware configuration using a normal computer.

  The program executed by the road monitoring apparatus according to the present embodiment is an installable or executable file, and is a computer such as a CD-ROM, a flexible disk (FD), a CD-R, or a DVD (Digital Versatile Disk). It is recorded on a readable recording medium and provided.

  Further, the program executed by the road monitoring apparatus of the present embodiment may be provided by being stored on a computer connected to a network such as the Internet and downloaded via the network. Further, the program executed by the road monitoring device of the present embodiment may be provided or distributed via a network such as the Internet.

  Further, the program of the road monitoring apparatus of the present embodiment may be configured to be provided by being incorporated in advance in a ROM or the like.

  Although several embodiments of the present invention have been described, these embodiments are presented by way of example and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the scope of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are included in the invention described in the claims and the equivalents thereof.

DESCRIPTION OF SYMBOLS 10 Road monitoring system 11 Central apparatus 12 Wireless base station 13 Communication network 14 Vehicle 15 Road monitoring apparatus 21 Imaging device 22 Memory part 23 Image processing part (determination part, extraction part)
24 Probe information transmitter (transmitter)
25 GPS / clock part 31 monitoring object discrimination part ( discrimination part)
32 Position / Time Information Calculation Unit 33 Vehicle Number Reading Unit 34 Video Extraction Unit (Extraction Unit)
41 Probe information receiving unit 42 Probe information database unit 43 Search unit 44 Discriminating unit X On-road monitoring object

Claims (11)

A road monitoring device mounted on a vehicle,
An imaging unit that performs at least forward or rear imaging of the vehicle and sequentially outputs imaging data;
A position / time information calculation unit that outputs position information and time information of the vehicle corresponding to the imaging data;
A determination unit for determining whether contains a predetermined monitoring object in the image in the corresponding prior SL imaging data,
An extraction unit that extracts the imaging data corresponding to a timing that is back a predetermined time from the time when the monitoring object disappears from the image when the predetermined monitoring object is included in the image ;
The extracted the imaging data is in association with the positional information and the time information, and a transmitting section that transmits by radio to the central unit,
Road monitoring device with The determination unit, a vehicle, a road construction site, among 及 beauty road falling objects, to determine at least one as the predetermined object to be monitored,
The road monitoring device according to claim 1. The predetermined monitoring object is a vehicle ,
The extraction unit extracts the imaging data corresponding to an image in which the license plate of the vehicle is reflected;
The road monitoring device according to claim 1. The predetermined monitoring object is a road construction site,
The extraction unit extracts the imaging data corresponding to a start position and an end position of the road construction site;
The road monitoring device according to claim 1. The predetermined monitoring object is a falling object on the road,
The extraction unit extracts the imaging data corresponding to the position where the falling object on the road is detected;
The road monitoring device according to claim 1. A probe information database unit which stores the received probe information data,
A central unit that performs processing on the basis of the probe information data store,
An imaging unit that is mounted on a vehicle and captures at least forward or rearward imaging of the vehicle and sequentially outputs imaging data; and position / time information that outputs position information and time information of the vehicle corresponding to the imaging data a calculation unit, before Symbol a determination section for determining whether or not a predetermined monitoring object in the image corresponding to the imaging data is included, if that contained pre Ki監 visual object in the image When the monitoring object disappears from the image corresponding to the imaging data, the extraction unit extracts the imaging data corresponding to a timing that is a predetermined time after the time when the monitoring object disappears from the image A road monitoring device comprising: a transmitter that wirelessly transmits the extracted imaging data as the probe information data to the central device in association with the position information and the time information;
A road monitoring system with The predetermined monitoring object is a vehicle ,
The extraction unit of the road monitoring device extracts the imaging data corresponding to an image in which a license plate of the parked vehicle is reflected,
The central device calculates a parking time from the time information and discriminates the extracted vehicle as a parking violation vehicle when the probe information database includes the probe information data of the same numbered vehicle. ,
The road monitoring system according to claim 6. The predetermined monitoring object is a road construction site,
The extraction unit of the road monitoring device extracts the imaging data corresponding to a start position and an end position of the road construction site;
The central device determines whether the road construction site is made according to a predetermined notification,
The road monitoring system according to claim 6. The predetermined monitoring object is a falling object on the road,
The extraction unit of the road monitoring device extracts the imaging data corresponding to the detection position of the falling object on the road,
The central device determines whether or not a countermeasure corresponding to the falling object on the road has been taken.
The road monitoring system according to claim 6. It is a method executed in a road monitoring device mounted on a vehicle, including an imaging unit that captures at least forward or backward imaging of a mounted vehicle and sequentially outputs imaging data,
A process of outputting position information and time information of the vehicle corresponding to the imaging data;
A process for determining whether contains a predetermined monitoring object in the image corresponding to the imaging data,
A step of extracting the imaging data corresponding to a timing that is back a predetermined time from a time when the monitoring object disappears from the image when the predetermined monitoring object is included in the image ;
A process of wirelessly transmitting the extracted imaging data to the central device in association with the position information and the time information;
With a method. A program for performing imaging of at least a forward direction or a backward direction of a mounted vehicle, including an imaging unit that sequentially outputs imaging data, and for controlling a road monitoring device mounted on the vehicle by a computer,
The computer,
Means for outputting position information and time information of the vehicle corresponding to the imaging data;
It means for determining whether contains a predetermined monitoring object in the image corresponding to the imaging data,
Means for extracting the imaging data corresponding to a timing that is back a predetermined time from the time when the monitoring object disappears from the image when the predetermined monitoring object is included in the image ;
Means for wirelessly transmitting the extracted imaging data to the central device in association with the position information and the time information;
Program to make it work.

Images