JP5931662B2 - Road condition monitoring apparatus and road condition monitoring method - Google Patents

Description

  Embodiments described herein relate generally to a road condition monitoring apparatus and a road condition monitoring method that detect a vehicle on a road and monitor the road condition.

  In general, a road condition monitoring apparatus is known as an apparatus that detects a vehicle on a road and monitors the road condition.

  The conventional road condition monitoring device cannot deal with an image such as a curve in which the traveling direction of the vehicle changes, so that the applicable road is limited to a straight line. Thereby, since it is necessary to designate the template creation range for each road surface on the input image, there is a problem that the vehicle detection accuracy is lowered.

Japanese Patent No. 3345623

  As described above, the conventional road condition monitoring device cannot deal with an image in which the traveling direction of the vehicle changes, such as a curve, so that applicable roads are limited to straight lines. Thereby, since it is necessary to designate the template creation range for each road surface on the input image, there is a problem that the vehicle detection accuracy is lowered.

  Accordingly, the object is to provide a road condition monitoring device capable of stably extracting and tracking a vehicle, such as a curve, in which the traveling direction of the vehicle changes, and improving vehicle detection accuracy, and a road. It is to provide a situation monitoring method.

According to the present embodiment, the road condition monitoring apparatus includes a photographing means, an acquisition means, a first detection means, a creation means, a second detection means, and a road condition detection means. The photographing means photographs the monitoring area. The acquisition unit acquires an image of the monitoring area from the photographing unit. The first detection means detects a vehicle that appears in the monitoring area based on the frequency of the feature amount in the local region in the acquired image. Creating means, the positional information and its characteristic information of the vehicle of the upper monitor area which is image or we detect acquired by the acquisition unit is calculated to create a template for identifying the vehicles from the calculated results . The second detecting means detects the same vehicle by sequentially comparing and collating the templates created by the creating means against the image obtained by the obtaining means. The road condition detection means includes a detection result of the vehicle based on the frequency of the feature quantity in the local region obtained by the first detection means , an image of the monitoring area obtained by the second detection means , and the template. The road condition is detected on the basis of the detection result of the vehicle based on the comparison comparison .

The block diagram which shows the structure of the road condition monitoring apparatus which concerns on this embodiment. The figure which shows the extraction result of the vehicle area | region in the vehicle area extraction part shown in FIG. The figure which shows the template image produced in the template production part shown in FIG. The figure which shows the process of the raster scan in the matching process part shown in FIG. The figure which shows the process of the vehicle tracking in the matching process part shown in FIG. The figure which shows multiple extraction of the candidate area | region which concerns on this embodiment.

  Hereinafter, embodiments will be described with reference to the drawings.

  FIG. 1 is a block diagram showing the configuration of the road condition monitoring apparatus according to the present embodiment.

  The road condition monitoring apparatus includes an imaging device, an image acquisition unit 1, a vehicle detection unit 2, a discriminator construction unit 3, a template creation unit 4, a vehicle tracking unit 5, a template update unit 6, and a road condition detection. Part 7.

  The image acquisition unit 1 receives the image data of the monitoring area from the imaging device in units of frames and transmits the image data to the vehicle detection unit 2. Note that the imaging apparatus of the present embodiment is, for example, an ITV camera, is installed at a predetermined position on the side of the road or above the road, captures an image of the road to be monitored, and outputs image data in units of frames. To do.

  The vehicle detection unit 2 detects a vehicle on the road that appears in the monitoring area based on the frequency of the feature amount in the local region for each frame of image data obtained from the image acquisition unit 1. The feature amount is an amount representing a numerical change in image data such as luminance. The vehicle detection unit 2 includes a search window setting unit 21, a first higher-order feature amount calculation unit 22, a likelihood calculation unit 23, a discriminator selection unit 24, and a vehicle region extraction unit 25.

  The search window setting unit 21 sets a search area on the obtained image data.

  The first higher-order feature quantity calculation unit 22 receives information on the search area from the search window setting unit 21 and calculates a higher-order feature quantity from the received search area.

  The likelihood calculating unit 23 is based on the high-order feature amount calculated by the first high-order feature amount calculating unit 22 and the vehicle information held in the plurality of discriminators 271 to 27N. The likelihood expressed as a vehicle or a numerical value is calculated.

  The discriminator selection unit 24 stores the discriminators 271 to 27N that are held in the vehicle detection unit 2 and classified into a plurality of categories such as ordinary vehicles and large vehicles when the likelihood calculation unit 23 calculates the likelihood. The optimum classifier is selected from among them. At this time, the discriminator selecting unit 24 refers to numerical values such as the brightness of the vehicle held in the plurality of installed dictionaries 261 to 26N, and selects the discriminator that is optimal for calculating the likelihood.

  The vehicle region extraction unit 25 extracts a vehicle region in which the vehicle is considered to be detected based on the likelihood information calculated by the likelihood calculation unit 23.

  In the vehicle area extracted by the vehicle detection unit 2, the template creation unit 4 calculates the position information on the monitoring area of the vehicle detected from the image acquired by the image acquisition unit 1 and its feature information, and from this calculation result Create a template to identify the vehicle. The created template is stored in the template information storage unit 63.

  The vehicle tracking unit 5 detects and tracks the same vehicle by using the template created by the template creation unit 4 and sequentially comparing and collating the template with the image acquired by the image acquisition unit 1. The vehicle tracking unit 5 includes a template information reading unit 51 and a matching processing unit 52.

  The template information reading unit 51 reads the template created by the template creation unit 4 and the upper left coordinates and lower right coordinates of the search area set in the template. Here, the search area in the template image is set based on the position information of the vehicle area at the time of creating the template, and the vehicle tracking unit 5 searches for the surrounding area.

  The matching processing unit 52 performs raster scanning using the template image read by the template information reading unit 51, the upper left coordinates, and the lower right coordinates of the search area. The matching processing unit 52 calculates the sum of the luminance differences between the template and the search area by raster scanning, and detects and tracks the vehicle from the calculated sum of the luminance differences.

  Further, the vehicle tracking unit 5 may calculate the comparison between the template image and the search area in the matching processing unit 52 not only by the sum of the luminance differences but also by the similarity as in the normal correlation.

  The template update unit 6 updates the template for each vehicle image data detected and tracked by the vehicle tracking unit 5. The template update unit 6 includes an overlap rate calculation unit 61 and a template update determination unit 62.

  The overlapping rate calculating unit 61 compares the coordinates obtained from the vehicle region extracting unit 25 with the coordinates obtained by the matching processing unit 52, and calculates the overlapping rate of the two rectangular regions.

  The template update determination unit 62 determines whether the overlap rate calculated by the overlap rate calculation unit 61 is greater than a certain value. If the overlap rate is greater than a certain value, the rectangular area extracted by the vehicle extraction unit 2 is updated as a template. Then, the upper left coordinates and lower right coordinates of the search area are updated simultaneously. At this time, the updated template is stored in the template information storage unit 63.

  The road condition detection unit 7 uses the detection results obtained through the vehicle detection unit 2 and the vehicle tracking unit 5 to cause events such as traffic congestion on the road, number of passing vehicles, overspeed, stop, low speed, avoidance, and reverse running. Is detected. At this time, the event is detected based on vehicle movement data obtained by the vehicle tracking unit 5.

  Here, the process of constructing the classifier will be described.

  The discriminator construction unit 3 constructs a discriminator for selection by the discriminator selection unit 24. The classifier construction unit 3 includes a learning data holding unit 31, a teaching data creation unit 32, a second higher-order feature quantity calculation unit 33, a learning unit 34, and a classifier construction processing unit 35.

  The teaching data creation unit 32 manually creates a rectangular vehicle region for the plurality of learning image data held by the learning data holding unit 31.

  The second higher-order feature quantity calculation unit 33 converts the rectangular vehicle area created by the teaching data creation unit 32 into a CoHOG (Co-occurrence Histograms of Gradients) feature quantity or a HOG (Histograms of Gradients) feature quantity, etc. Based on this, a high-order feature amount of the local region is calculated. Note that the calculated number of higher-order feature quantities in the local area matches the number of vehicle areas created by the teaching data creation unit 31.

  The learning unit 34 calculates learning data such as vehicle type information from the high-order feature quantity of the local region calculated by the second high-order feature quantity calculation unit 33.

  The discriminator construction processing unit 35 classifies the learning data calculated by the learning unit 34 into a plurality of categories such as a normal car and a large car, and holds them in the plurality of discriminators 271 to 27N for each category. Here, for the construction of the discriminators 271 to 27N, for example, a non-rule-based method for constructing discrimination parameters from a large amount of data by machine learning such as subspace method, support vector machine, K-neighbor discriminator, Bayesian classification Is used. In addition, you may implement | achieve this embodiment by methods other than the above.

  FIG. 2 is a diagram illustrating a vehicle region extraction result in the vehicle region extraction unit 25.

  Although FIG. 2 has been described using three vehicles, there is no upper limit to the number of vehicles, and the vehicle areas 201 to 203 are extracted for all vehicles on the road.

  FIG. 3 is a diagram showing a template image created by the template creation unit 4.

  The template creation unit 4 can create a template image without depending on the installation location of the camera, the size of the road surface, and the like. Further, the template image size 301 may be the rectangular size 201 at the time of vehicle detection, or the height and width may be set smaller by a preset numerical value than at the time of vehicle detection for the purpose of deleting the background and the like. Also good. At this time, the vehicle extraction area 201 and the template image 301, the vehicle extraction area 202 and the template image 302, and the vehicle extraction area 203 and the template image 303 are associated with each other. In FIG. 3, the description has been made using three vehicles, but the template creation unit 4 creates a template for all the vehicles on the road without any upper limit.

  FIG. 4 is a diagram illustrating a raster scanning process in the matching processing unit 52.

  As illustrated in FIG. 4, a raster scan 403 is performed using the template image 402 read by the template information reading unit 51, the upper left coordinates 401 and the lower right coordinates 404 of the search area, and the template image 402 performs summation of luminance differences. And the vehicle is tracked from the calculated sum of the brightness differences.

  FIG. 5 is a diagram illustrating a vehicle tracking process in the matching processing unit 52.

  501 to 506 shown in FIG. 5 respectively show template images 501 to 503 for the previous frame and matching result images 504 to 506 for the current frame.

  In FIG. 5, although three vehicles are used for explanation, the vehicle tracking unit 5 has no upper limit on the number of vehicles, and detects and tracks vehicles for all vehicles on the road.

  As described above, in the present embodiment, the detection process of the vehicle detection unit 2 and the detection process of the vehicle tracking unit 5 are used together, so that even if the detection process of one side fails in detection, the detection process of the other side is covered. it can.

  Thereby, a vehicle can be extracted and tracked stably, and a vehicle detection accuracy can be improved.

  Therefore, a road condition monitoring apparatus and a road condition monitoring method that can stably extract and track a vehicle, such as a curve, in which the traveling direction of the vehicle changes and can improve the vehicle detection accuracy. It becomes possible to provide.

  Here, the discriminators 271 to 27N of the vehicle detection unit 2 learn the detection results when the vehicle detection unit 2 fails to detect the vehicle and the vehicle tracking unit 5 succeeds in detecting and tracking the vehicle. By doing so, the subsequent detection accuracy can be improved.

  FIG. 6 is a diagram showing a plurality of candidate area extractions.

  As shown in FIG. 6, the vehicle tracking unit 5 extracts a plurality of candidate regions 603 to 604 during the matching process in the matching processing unit 62 at time t. Furthermore, the vehicle tracking unit 5 selects the tracking result 603 from the candidate areas according to the trajectories 601 to 602 of the vehicle area detected in the past.

  Thereby, since the vehicle matching to the adjacent lane does not fail, the vehicle tracking accuracy of the vehicle is improved.

  In FIG. 6, the time interval for calculating the trajectory and the number of candidate areas are not limited to the numerical values in the figure and can be freely determined.

  In the classifier construction unit 3 shown in the present embodiment, the learning unit 34 learns vehicle orientation information in advance, and the classifier construction processing unit 35 constructs a classifier. The discriminator selection unit 24 selects the discriminator according to the angle when the appearance of the vehicle in the image detected by the vehicle detection unit 2 such as a curve changes by a certain angle or more. At the same time, the template update determination unit 62 updates the rectangular area extracted by the vehicle area extraction unit 25 as the template 502, and also updates the upper left coordinates 501 and lower right coordinates 504 of the search area.

  As a result, the vehicle can be extracted and tracked stably even in an image such as a curve where the traveling direction of the vehicle changes, and the vehicle detection accuracy can be improved.

  As mentioned above, although embodiment of this invention was described, this embodiment is shown as an example and is not intending limiting the range of invention. This embodiment can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the spirit of the invention. This embodiment and its modifications are included in the scope of the present invention and the gist thereof, and are also included in the invention described in the claims and the equivalent scope thereof.

  DESCRIPTION OF SYMBOLS 1 ... Image acquisition part, 2 ... Vehicle detection part, 21 ... Search window setting part, 22 ... 1st high-order feature-value calculation part, 23 ... Likelihood calculation part, 24 ... Discriminator selection part, 25 ... Vehicle area extraction 261-26N ... dictionary, 271-27N ... discriminator, 3 ... discriminator construction unit, 31 ... learning data holding unit, 32 ... teaching data creation unit, 33 ... second higher-order feature quantity calculation unit, 34 ... Learning unit, 35 ... Classifier construction processing unit, 4 ... Template creation unit, 5 ... Vehicle tracking unit, 51 ..., 52 ..., 6 ... Template update unit, 61 ... Overlap rate calculation unit, 62 ... Template update determination unit, 63 ... Template information storage unit, 7 ... Road condition detection unit.

Claims (11)

Photographing means for photographing the surveillance area;
Obtaining means for obtaining an image of the monitoring area from the photographing means;
First detection means for detecting a vehicle in the monitoring area based on the frequency of feature quantities in a local region in the acquired image;
A creation unit for position information and characteristics information of the vehicle of the upper monitor area which is image or we detect acquired by the acquisition means calculates, to create a template for identifying the vehicle from the calculation results,
Second detecting means for detecting the same vehicle by sequentially comparing and collating the templates created by the creating means with respect to the image obtained by the obtaining means;
Wherein by comparison and collation between said first detection result of the vehicle in which the based on the feature amount of the frequency of a local region obtained by the detection means, and the image and the template of the monitoring area obtained by the second detection means vehicle A road condition monitoring device, comprising: road condition detection means for detecting a road condition based on the detection result of. The road condition monitoring apparatus according to claim 1, wherein the first detection means holds a plurality of discriminators classified into a plurality of categories. When the first detection means fails to detect the vehicle and the second detection means succeeds in detecting and tracking the vehicle, a plurality of discriminators in the first detection means are used by the second detection means. The road condition monitoring apparatus according to claim 2, wherein the detection result is learned. The first detection means, subspace method, support vector machines, K near identifier, and wherein the detection on the basis of the classifier constructed in accordance with at least one machine learning of the Bayesian classification The road condition monitoring device according to claim 2 .   The road condition monitoring apparatus according to claim 1, wherein the second detection unit sets the periphery as a search area based on position information at the time of template creation. The second detection means, wherein the sum of the template and the luminance difference of the search area, or a road condition monitoring apparatus according to claim 5, characterized in that the similarity is calculated. The second detection means, the candidate region more extraction, depending on the locus of previously detected vehicle area, the road condition monitoring apparatus according to claim 2, wherein the selecting the tracking result from the candidate region. The road condition monitoring apparatus according to claim 7 , further comprising an updating unit that updates a template for each vehicle tracked by the second detection unit. The update unit includes: a feature a rectangular region obtained in said first detection means, by overlapping ratio of the rectangular area obtained by the second detection means, to determine whether to update the template The road condition monitoring apparatus according to claim 8 . It said updating means, when said appearance of the first vehicle of the detected in the image by the detection means has changed a predetermined angle or more, according to claim 8, wherein the determining whether to update the pre-Symbol Template Road condition monitoring device. Take a picture of the surveillance area to obtain an image of the surveillance area ,
Based on the frequency of the feature amount in the local area in the acquired image, detect the vehicle that appears in the monitoring area ,
From the acquired image, the detected position of the vehicle, and based on the orientation information of the vehicle to create a template for identifying the vehicle,
It detects vehicle by sequentially comparing and collating the created template to the acquired image,
The road condition is detected based on the detection result based on the detection result of the vehicle based on the frequency of the feature amount in the local region and the detection result of the vehicle based on the comparison check between the image of the monitoring area and the template A road condition monitoring method characterized by this.

Images