JP2014056441A - Road state grasping apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a road state grasping apparatus capable of accurately detecting an abnormality in an image from an imaging means.SOLUTION: A road state grasping apparatus pertaining to the embodiment images an image of a road on which a vehicle travels and determines a state of the road based on the imaged image. In this case, the road state grasping apparatus includes: a calculation means for calculating a unit time mean brightness from a plurality of images with different imaging time obtained from the imaging means, a comparison means for comparing the unit time mean brightness calculated by this calculation means and a pre-registered unit time mean brightness and deriving a coincidence of the two; and an abnormality detection means for detecting the abnormality in the image from the imaging means by comparing the coincidence derived by this comparison means with a pre-registered threshold.

Description

  Embodiments described herein relate generally to a road condition grasping apparatus that determines a road condition.

  On a toll road such as an expressway, an image on the road is captured with a camera installed on the roadside, and a vehicle traveling on the road is detected from the captured image by image processing or the like, and the position of the detected vehicle, There is known a road condition grasping device that determines a current road condition (congested, stopped vehicle, low-speed vehicle, etc.) from information such as speed.

  In such a road situation grasping device, if there is an abnormality in the image obtained from the camera, the road situation cannot be determined correctly. Therefore, various techniques for analyzing images and detecting image abnormalities based on the analysis results have been proposed. However, it is difficult to detect image abnormalities by mere image analysis because road conditions change greatly over time. .

For example, since the brightness of an image captured in the daytime and an image captured in the night are different, it is not easy to detect an abnormality of the image only by image analysis.
Further, for example, under the condition that there is no illumination at night and no vehicle is present, there is a possibility that an abnormality is always determined when a camera is installed in a completely dark place.

JP 2000-207563 A

  The problem to be solved by the present invention is to provide a road condition grasping device capable of accurately detecting an abnormality of an image from an imaging means.

  The road condition grasping device according to the embodiment is a road condition grasping device that picks up an image on a road on which a vehicle travels by an image pickup unit and determines the road condition based on the picked-up image. The calculation means for calculating the unit time average luminance from a plurality of images obtained at different imaging times obtained by comparing the unit time average luminance calculated by this calculation means with the unit time average luminance registered in advance, and the two match Comparing means for obtaining a degree, and an abnormality detecting means for detecting an abnormality of the image from the imaging means by comparing the degree of coincidence obtained by the comparing means with a threshold value registered in advance.

The schematic diagram which shows roughly the structure of the road condition grasping | ascertainment apparatus which concerns on embodiment. 6 is a flowchart for describing image abnormality detection processing according to the embodiment. 6 is a flowchart for describing image abnormality detection processing according to the embodiment. 6 is a flowchart for describing image abnormality detection processing according to the embodiment. The figure explaining the specific example of the calculation process of the unit time average brightness | luminance which concerns on embodiment. The figure explaining the specific example of the calculation process of the coincidence threshold value in consideration of the weather conditions according to the embodiment. The figure explaining the specific example of the coincidence degree calculation process of the unit time average brightness which concerns on embodiment.

Hereinafter, a road condition grasping device according to an embodiment will be described with reference to the drawings.
FIG. 1 schematically shows a configuration of a road condition grasping apparatus according to an embodiment. In FIG. 1, a road 11 is a toll road such as an expressway (hereinafter sometimes simply referred to as a road), and a vehicle 12 travels in the direction of an arrow shown in the figure.

  A video camera (hereinafter simply abbreviated as a camera) 13 as an image pickup means is installed at a predetermined portion of the road 11. The camera 13 captures images in a predetermined area including the vehicle 12 passing through a predetermined part of the road 11 at a constant interval (for example, every 33 ms) and outputs the images as frame images. To the image conversion unit 14.

  The image conversion unit 14 is provided in the vicinity of the camera 13, for example, and converts the frame image output from the camera 13 into a predetermined image signal (for example, an optical signal, a LAN packet signal, etc.) and outputs the image signal.

  The image signal output from the image conversion unit 14 is sent to, for example, the processing device 15 installed at a location away from the camera 13. The processing device 15 detects a vehicle by performing predetermined image processing on an image picked up by the camera 13, determines a road condition such as a traffic event and an abnormal vehicle on the road 11 from the vehicle detection result, and determines the number of passing vehicles. The image restoration unit 16 restores the input image signal to the original image, the image processing unit 17 executes predetermined image processing on the restored image, and various data used for the image processing. It includes an initial data holding unit 18 that holds, a luminance calculation unit 19 that calculates average luminance at regular intervals, and a determination processing unit 20 that determines a final road condition from the image processing result.

  Here, the road conditions determined by the determination processing unit 20 are, for example, four states of “traffic jam”, “stopped vehicle”, “low speed vehicle”, and “normal state”. “Congestion” is a state where the road 11 is congested with the vehicle 12, “Stopped vehicle” is a state where the vehicle 12 is stopped on the road 11, and “Low speed vehicle” is predetermined on the road 11. It is assumed that the vehicle 12 traveling at a speed equal to or less than the predetermined value is present. When the road condition is not the above three states, the “normal state” is set.

  The image processing unit 17 is connected to a weather database (weather DB) 21 via a communication line. The meteorological database 21 is a database provided by the Japan Meteorological Agency and the like, for example, the meteorological data of the area where the camera 13 is installed is registered for each time of each date and time (for example, see FIG. 6).

  The determination processing unit 20 is connected to the host device 22 via a communication line. The host device 22 receives the determination result and the like from the determination processing unit 20.

  The road condition grasping device captures the road 11 with a camera 13 installed on a toll road, etc., performs predetermined image processing on the obtained image, and finally the road condition (congested, stopped vehicle, low-speed vehicle). And the number of passing vehicles is measured.

  Depending on the installation positions of the camera 13 and the processing device 15, the image output from the camera 13 may not be input to the processing device 15 as it is (due to a transmission distance problem or the like). In such a case, the frame image output from the camera 13 is converted into a predetermined image signal (for example, an optical signal, a LAN packet signal, etc.) by the image conversion unit 14, and the converted image signal is transmitted to the processing device 15. A method is adopted in which the processing device 15 receives the image signal from the image conversion unit 14 and the image restoration unit 16 restores the original image.

  In such a situation, when the image restoration unit 16 that restores the original image in the processing device 15 is operating normally, the image signal is normally transmitted between the image conversion unit 14 and the processing device 15. There is a possibility that the image abnormality cannot be detected by the processing device 15.

  In addition, depending on the installation position of the camera 13, the brightness of the captured image is not uniform due to factors such as darkness at night because there is no night illumination, always dim if in a tunnel, or sudden fluctuations in weather, If the image abnormality determination is performed without considering the installation position of the camera 13 and the weather condition, there is a possibility of erroneous determination.

  The road condition grasping device described in the present embodiment can accurately detect an image abnormality based on images from the camera 12 under various installation conditions. Hereinafter, the image abnormality detection process will be described in detail with reference to the flowcharts shown in FIGS.

  First, when the present apparatus is activated, the processing device 15 acquires time data from the host device 22 (step S1), and sets the acquired time data in a clock (for example, a real time clock) in the image processing unit 17 ( Step S2).

  The camera 13 installed on the road 11 captures an image in a predetermined area including the vehicle 12 passing through a predetermined part of the road 11 at a constant interval (for example, every 33 ms) and outputs it as a frame image. The output frame images are sequentially sent to the image conversion unit 14 (step S3).

  The image conversion unit 14 converts the image from the camera 13 into a predetermined image signal and transmits it to the processing device 15 (step S4). At this time, the format of the converted signal is not limited. For example, when an image signal is transmitted to the processing device 15 via a LAN line, it is converted into a LAN packet signal. When an image signal is transmitted to the processing device 15 via an optical line, it is converted into an optical signal.

  When receiving the image signal transmitted from the image conversion unit 14, the processing device 15 restores the original image in the image restoration unit 16 and sends it to the image processing unit 17 (step S5).

  The image processing unit 17 confirms whether an image is input (step S6). If no image is input as a result of the confirmation, it is determined that the image restoration unit is abnormal (that is, no image is received from the image restoration unit 16), and an abnormality notification to that effect is transmitted to the host device 22 (step S7). .

  If the result of confirmation in step S6 is that an image has been input, it is confirmed whether an initial data registration command has been issued (step S8). The initial data registration command is issued, for example, from the host device 22 or an external terminal device. The processing device 15 shifts to the initial data registration mode when receiving the initial data registration command.

  When shifting to the initial data registration mode, the image processing unit 17 acquires a threshold coefficient for each weather condition (sunny, cloudy, rainy) from the weather database 21, registers it in the initial data holding unit 18 and holds it ( Step S9).

  Next, the image processing unit 17 performs predetermined image processing on the image obtained from the image restoration unit 16, and determines the presence or absence of a vehicle (step S10). For example, a technique disclosed in Japanese Patent Application Laid-Open No. 2010-44502 can be applied to the vehicle detection process.

  If no vehicle is present as a result of the determination in step S10, the luminance calculation unit 19 calculates the average luminance of the entire image based on the image, and holds (stores) the calculated average luminance inside (step S11). . If the result of determination in step S10 is that there is a vehicle, processing returns to step S3 and the same operation is repeated.

  Note that the average brightness is calculated only when there is no vehicle, because when there is a vehicle in the image, the brightness tends to increase due to the influence of the lighting of the vehicle, etc. Is excluded from the calculation target of average luminance, and an image without a vehicle is set as a calculation target of average luminance.

This is because the average luminance is calculated only from the background image in which no vehicle exists.

  Each time the image processing unit 17 receives an image from the image restoration unit 16, the image processing unit 17 determines the presence or absence of a vehicle in the image, and when there is no vehicle in the image, the luminance calculation unit 19 performs the entire image based on the image. The average luminance is calculated, and the calculated average luminance is held (stored).

  By repeating this and when a certain time has elapsed (step S12), the luminance calculation unit 19 calculates the unit time average luminance using the held average luminance (step S13), and the obtained unit time average luminance is the initial data. Register (store) in the holding unit 18 (step S14).

  When the predetermined time does not elapse in step S12 or when the process of step S14 ends, the process returns to step S3 and the same operation as described above is repeated.

  It is assumed that the predetermined time is an arbitrary value and is stored in the initial data holding unit 18. In addition, the predetermined time can be changed by an external terminal or the like.

  Steps S10 to S14 are performed until no initial data registration command is issued.

Here, a specific example of the unit time average luminance calculation process in the luminance calculation unit 19 will be described with reference to FIG.
For example, as shown in FIG. 5, the luminance calculation unit 19 follows a calculation period of unit time average luminance of 60 minutes (fixed time), and a plurality of images in which a vehicle does not exist among a plurality of images obtained in 60 minutes. The average brightness of the images, that is, the unit time average brightness is calculated. Furthermore, the luminance calculation unit 19 registers the unit time average luminance in the initial data holding unit 18 for each time zone (0 o'clock, 1 o'clock,..., 23:00 o'clock). Thereby, as shown in FIG. 5, the initial data holding unit 18 holds (stores) a database including the unit time average luminance for each time zone.

When the initial data registration command is not issued (step S8), the processing device 15 shifts to the operation mode.
When the mode is shifted to the operation mode, the image processing unit 17 first searches the weather database 21 for weather data at the calculation time of the unit time average luminance as shown in a specific example of the processing in FIG. 6 (step S15).
Next, the image processing unit 17 searches for the corresponding threshold coefficient from the searched weather data from the threshold coefficient for each weather condition registered in step S <b> 9, and the matching threshold value registered in advance in the initial data holding unit 18. The initial threshold value is multiplied by the searched threshold coefficient and registered in the initial data holding unit 18 as a new matching degree threshold value (step S16).

  The specific example of FIG. 6 is, for example, that the threshold coefficient for each registered weather condition is “sunny = 1”, “cloudy = 0.8”, “rain = 0.5”, and the initial value of the registered coincidence threshold Is “80”, when the weather condition is “clear”, the new coincidence threshold is “80 (= 80 × 1)”, and when the weather condition is “cloudy”, the new coincidence threshold is “64”. (= 80 × 0.8) ”, when the weather condition is“ rain ”, it indicates that the new coincidence threshold is“ 40 (= 80 × 0.5) ”.

  Next, the image processing unit 17 performs predetermined image processing on the image obtained from the image restoration unit 16, and determines the presence or absence of a vehicle (step S17). For example, a technique disclosed in Japanese Patent Application Laid-Open No. 2010-44502 can be applied to the vehicle detection process.

  If the result of determination in step S17 is that there is no vehicle, the luminance calculation unit 19 calculates the average luminance of the entire image based on the image, and holds (stores) the calculated average luminance inside (step S18). . If the result of determination in step S17 is that there is a vehicle, processing returns to step S3 and the same operation is repeated.

  Note that the average brightness is calculated only when there is no vehicle, because when there is a vehicle in the image, the brightness tends to increase due to the influence of the lighting of the vehicle, etc. Is excluded from the calculation target of average luminance, and an image without a vehicle is set as a calculation target of average luminance.

This is because the average luminance is calculated only from the background image in which no vehicle exists.

  Each time the image processing unit 17 receives an image from the image restoration unit 16, the image processing unit 17 determines the presence or absence of a vehicle in the image, and when there is no vehicle in the image, the luminance calculation unit 19 performs the entire image based on the image. The average luminance is calculated, and the calculated average luminance is held (stored).

  This process is repeated, and when a certain time has elapsed (step S19), the luminance calculation unit 19 calculates unit time average luminance using the held average luminance (step S20). If the predetermined time does not elapse in step S19, the process returns to step S3 and the same operation is repeated.

  It is assumed that the predetermined time is an arbitrary value and is stored in the initial data holding unit 18. In addition, the predetermined time can be changed by an external terminal or the like.

  Further, as described above, the image processing unit 17 may have a clock that operates based on the time data from the host device 22 and associate time information with the received image. Thereby, time information is associated with each image, and the unit time average luminance calculated from the images at the time included in the first time zone can be set as the unit time average luminance in the first time zone.

  Next, the determination processing unit 20 searches the initial data holding unit 18 based on the calculation time of the unit time average luminance obtained in step S20, and the corresponding part of the unit time average luminance registered in step S14 (same time) A portion linked with a band or the like is searched (step S21).

  Next, the determination processing unit 20 searches for the unit time average luminance (hereinafter referred to as “registered unit time average luminance”) retrieved from the initial data holding unit 18 in step S21 and the unit time average luminance (hereinafter, referred to as “registered unit time average luminance”). This is used as a calculated unit time average luminance), and the degree of coincidence between them is calculated (step S22).

Here, a specific example of the unit time average luminance matching degree calculation processing in the determination processing unit 20 will be described with reference to FIG.
For example, as illustrated in FIG. 7, the determination processing unit 20 compares the registered unit time average luminance at the 0:00 level with the calculated unit time average luminance at the 0:00 level, and calculates the degree of coincidence. Similarly, the determination processing unit 9 compares the registration unit time average luminance at the 1 o'clock range with the calculated unit time average luminance at the 1 o'clock range, and calculates the degree of coincidence between them.

There are various known examples of the method for calculating the degree of coincidence. For example, it is defined by the following equation.
Consistency = (1- (| calculated unit time average brightness−registration unit time average brightness) / registration unit time
Average brightness)) * 100
Next, the determination processing unit 20 detects an abnormality of the input image to the image restoration unit 16 based on the degree of coincidence calculated in step S22. For example, the determination processing unit 20 compares the coincidence degree calculated in step S22 with the coincidence degree threshold value registered in the initial data holding unit 18 in step S16 (step S23), and the calculated coincidence degree is the coincidence degree threshold value. In the following case, it is determined that some abnormality has occurred in the input image to the image restoration unit 16, and an abnormality (image abnormality) is notified to the host device 22 (step S24).

  As a result of the comparison in step S23, when the calculated coincidence is higher than the coincidence threshold, the determination processing unit 20 determines that the input image is normal, and performs a road situation determination process based on the input image. Then, the determination result is notified to the host device 22 (step S25).

Here, an example of road condition determination processing performed by the determination processing unit 20 will be briefly described.
First, the image processing unit 17 detects the vehicle 12 traveling on the road 11 by performing predetermined image processing for each input frame image.
Specifically, first, a background image in which the vehicle 12 does not exist is stored in advance in a memory (not shown) as initial data, and the coordinate value of the image is converted into an absolute distance (actual distance) from the camera 13. These conversion tables are also stored in the memory.

When an image is input in this state, a luminance difference image is generated by obtaining a difference in luminance value of each pixel between the input image and a background image previously stored in the memory. Next, the vehicle area (vehicle 12) is detected by performing binarization processing on the generated difference image using parameters registered in advance. Next, the detected position of the vehicle 12 (region), the vehicle area, and the traveling speed are obtained by converting the detected coordinate value of the vehicle region into an absolute distance using a conversion table previously stored in a memory.
The details of the vehicle detection process are described in, for example, Japanese Patent Application No. 2008-207034, so please refer to it.

Next, the determination processing unit 20 determines at least four conditions of the traffic 11 as a situation of the road 11 based on the vehicle information detected as described above, that is, a traffic jam, a stopped vehicle, a low speed vehicle, and a normal state.
More specifically, for example, whether or not the road 11 is congested is determined based on the position, vehicle area, and traveling speed of the vehicle 12 acquired as described above, and based on the acquired traveling speed. The stop vehicle and the low-speed vehicle are determined.
Note that details of the road condition determination processing are described in, for example, Japanese Patent Application No. 2008-207034.

According to the road condition grasping device of at least one embodiment described above, the following effects can be expected.
Even if the image from the camera is not directly input to the image processing unit and the image conversion unit and the image restoration unit are interposed between the camera and the image processing unit, it is determined whether there is an abnormality in the image from the camera. It can be detected accurately.
In addition, since the abnormality determination of the image is not performed only by the brightness of the image in consideration of the camera installation position, it is possible to accurately detect the image abnormality without fear of erroneous determination of the image abnormality. For example, an image abnormality can be accurately detected even when there is no illumination around the camera installation position or when the camera installation position is in a tunnel.

  In addition, because image conditions are not determined based on only the brightness of the image in consideration of weather conditions, it is possible to accurately detect image anomalies, for example, without the risk of misidentifying image anomalies during sudden weather changes. It becomes.

  Although several embodiments of the present invention have been described, these embodiments are presented by way of example and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the scope of the invention. These embodiments and their modifications are included in the scope and gist of the invention, and are also included in the invention described in the claims and the equivalents thereof.

  DESCRIPTION OF SYMBOLS 11 ... Road, 12 ... Vehicle, 13 ... Camera (imaging means), 14 ... Image conversion part, 15 ... Processing apparatus, 16 ... Image restoration part, 17 ... Image processing part, 18 ... Initial data holding part, 19 ... Luminance calculation Part, 20 ... determination processing part, 21 ... weather database, 22 ... host device.

Claims (4)

A road situation grasping device that takes an image on a road on which a vehicle travels with an imaging means and determines the road situation based on the taken image,
Calculation means for calculating unit time average luminance from a plurality of images having different imaging times obtained from the imaging means;
A comparison means for comparing the unit time average brightness calculated by the calculation means with a unit time average brightness registered in advance and obtaining a degree of coincidence between the two;
An abnormality detecting means for detecting an abnormality of the image from the imaging means by comparing the degree of coincidence obtained by the comparing means and a threshold value registered in advance;
A road condition grasping device. An image on a road on which the vehicle is traveling is captured by an imaging unit, the captured image is converted into a predetermined image signal, and the converted image signal is received at a place away from the imaging unit to be an original image. A road condition grasping device for restoring and determining the situation of the road based on the restored image,
Calculating means for calculating a unit time average brightness from the restored images having different imaging times;
A comparison means for comparing the unit time average brightness calculated by the calculation means with a unit time average brightness registered in advance and obtaining a degree of coincidence between the two;
An abnormality detecting means for detecting an abnormality of the image from the imaging means by comparing the degree of coincidence obtained by the comparing means and a threshold value registered in advance;
A road condition grasping device.   The road condition grasping device according to claim 1 or 2, wherein the threshold is set in consideration of a weather condition of an installation position of the imaging unit. The calculation means calculates a unit time average luminance of the predetermined time period from the plurality of images of the predetermined time period,
The road condition grasping according to claim 1 or 2, wherein the comparison means compares the unit time average luminance calculated by the calculation means in the predetermined time zone with the unit time average luminance registered in advance in the predetermined time zone. apparatus.

Images