JP4973565B2 - Road condition grasping system and road condition grasping method - Google Patents

Description

  The present invention relates to a moving object detection / situation grasping method using moving images, and in particular, a vehicle is stably detected in real time from a moving image captured on a road for 24 hours, and the vehicle is detected with high reliability. The present invention relates to a technology for automatically grasping a driving situation.

  Development of a device for grasping road conditions (the presence of vehicles and their running conditions) using visible images has been underway. It is not always necessary to detect and track individual vehicles as long as the traffic flow is captured. However, in order to develop an ITS that allows roads and vehicles to communicate individually and provide more advanced services, it is necessary to It is necessary to grasp the running situation of the vehicle.

  For this purpose, a technique (hereinafter referred to as a vehicle detection technique) that can accurately measure the existence of a vehicle and its position / speed using a moving image from the road is very important.

  When visible images are used, performance that can meet operational requirements is obtained in places where lighting is stable, such as in tunnels, but in the outdoors (lights), “low illumination at night” and “shadows from the sun”. It is difficult to obtain the expected vehicle detection performance due to the influences of “swaying and shaking”, “west sun”, “lights of oncoming vehicles”, “mirroring in the rain”.

  On the other hand, in recent years, the development of a device for grasping the rational situation using infrared images has been promoted, and it is possible to detect vehicles with extremely high reliability (vehicle detection rate, position, speed measurement accuracy) compared to visible images outdoors. Has been demonstrated in actual road tests.

  However, it has also been found that the infrared image has a problem in detection performance for a vehicle having a small temperature difference between the vehicle bodies such as a vehicle and a scooter in rainy weather. Therefore, development of a road condition grasping method capable of grasping the road condition stably for 24 hours under various environmental conditions is expected.

As a related technology, each input image obtained by a visible camera and an infrared camera and a background image captured in advance and stored in a background image storage unit are subjected to difference processing by a background difference processing unit to create a background difference image. An object in which an object is detected by an object detection unit by combining two background difference images that are ternarized by a ternary processing unit is disclosed (for example, see Patent Document 1).
JP 2001-357388 A

  However, there are some problems in the road condition grasping device using moving images such as the related technology described above.

  The first problem is that when a vehicle is detected using a visible image, the vehicle detection accuracy is greatly affected by the sunlight conditions.

  The reason for this is that the visible image has a low illuminance at night, shadows and fluctuations during the day, disturbance light from the western sun during dusk, and the sudden change in illuminance that occurs when the sun is hidden in the clouds. This is because image processing cannot follow this.

  A second problem is that when a vehicle is detected using a visible image, the vehicle detection accuracy is lowered when it rains.

  The reason for this is that the road surface gets wet due to rain and becomes a mirror surface, so that light from the headlights of other vehicles, street lights, etc. is reflected on the road surface and disturbs the visible image as disturbance light.

  The third problem is that when a vehicle is detected using an infrared image, the vehicle detection accuracy decreases when it rains.

  The reason is that the temperature difference between the vehicle and the road is reduced by being covered with water, and the contrast difference between the vehicle and the road is reduced.

  The present invention has been made to solve the above-described problems, and provides a road condition grasping system and a road condition grasping method for measuring the traveling condition of a traveling vehicle in real time and with high reliability for 24 hours. For the purpose.

A first road condition grasping system according to the present invention includes a visible camera and an infrared camera, a gain control unit that generates a gain that is a weighted value of a ratio adapted to environmental conditions, and the visible camera and the infrared camera. Using a gain multiplying unit that multiplies the image by the gain generated by the gain control unit, an image combining unit that adds a visible image and an infrared image multiplied by the gain, and a combined image generated by the image combining unit An image processing unit that detects a traveling vehicle and grasps the traveling state thereof, and the gain, which is a weighting value of a ratio adapted to the environmental condition, is obtained when the daylight condition is a problem in the daytime, when it is cloudy. It is characterized in that the mix ratio of the infrared image is set to a high value at time and night .

The second road condition grasping system according to the present invention includes a visible camera and an infrared camera, a gain control unit that generates a gain as a weight value, and the gain control unit on an image obtained by the visible camera and the infrared camera. A gain multiplier that multiplies the gain generated in step (a), an image composition unit that adds a visible image to the infrared image multiplied by the gain, and an infrared image using the composite image generated by the image composition unit An image processing unit that detects a traveling vehicle by processing and grasps the traveling state thereof, and the gain that is a weighting value of a ratio adapted to the environmental condition is a clear daytime when the sunshine condition is a problem, It is characterized in that the mix ratio of infrared images is set to a high value during cloudy weather and at night .

The third road condition grasping system according to the present invention includes a visible camera and an infrared camera, a gain control unit that generates a gain that is a weight value, and the gain control unit on an image obtained by the visible camera and the infrared camera. A gain multiplying unit that multiplies the gain generated in step (a), an image composition unit that adds an infrared image to the visible image multiplied by the gain, and a visible image process using the composite image generated by the image composition unit. The gain, which is a weighting value of the ratio adapted to the environmental condition, is obtained when the daylight condition is a problem in the daytime, when the daytime is fine. It is characterized in that the mix ratio of the infrared image is set to a high value at time and night .

The first road condition grasping method of the present invention includes a step of capturing an image with a visible camera and an infrared camera, a gain control step of generating a gain that is a weighted value of a ratio adapted to environmental conditions, A gain multiplication step of multiplying an image generated by the visible camera and the infrared camera by the gain generated in the gain control step, an image synthesis step of adding the visible image and the infrared image multiplied by the gain, and the image synthesis step Image processing step for detecting a traveling vehicle and grasping the traveling state using the composite image generated in step (2), and the gain, which is a weighting value of the ratio adapted to the environmental condition, has a problem with the sunshine condition. It is characterized in that the mix ratio of infrared images is set to a high value during sunny daytime, cloudy daytime, and nighttime .

Further, the second road condition grasping method of the present invention includes a step of capturing an image with a visible camera and an infrared camera, a gain control step of generating a gain that is a weighted value of a ratio adapted to environmental conditions, A gain multiplying step of multiplying the image generated by the gain control step to an image obtained by the visible camera and the infrared camera, an image combining step of adding a visible image to the infrared image multiplied by the gain, and the image An image processing step of detecting a traveling vehicle by infrared image processing using the composite image generated in the composite step and grasping the travel state thereof, and the gain that is a weight value of the ratio adapted to the environmental condition the fine weather of daytime sunshine condition is a problem, cloudy weather and at night characterized in that the high value mix rate of the infrared image to .

The third road condition grasping method of the present invention includes a step of capturing an image with a visible camera and an infrared camera, a gain control step of generating a gain that is a weighted value of a ratio adapted to environmental conditions, A gain multiplication step of multiplying an image generated by the visible camera and the infrared camera by the gain generated in the gain control step, an image composition step of adding an infrared image to the visible image multiplied by the gain, and the image An image processing step of detecting a traveling vehicle by visual image processing using the composite image generated in the composite step and grasping the traveling state thereof, and the gain that is a weighting value of the ratio adapted to the environmental condition is , fine weather of daytime sunshine condition is a problem, cloudy weather and at night to, characterized in that the high value mix rate of the infrared image .

  According to the present invention, it is possible to measure the traveling state of a traveling vehicle in real time and with high reliability for 24 hours.

  Hereinafter, a first embodiment of the present invention will be described in detail with reference to the drawings. The road condition grasping system in the present embodiment shown in FIG. 1 first performs vehicle filtering on a visible image acquired by the visible camera 111 to reduce noise and perform filtering on the filtered image. A visible pre-processing unit 101 that generates an image (moving region extraction image) obtained by extracting only a temporally changing region from a visible image using an inter-frame difference method, a background difference method, and the like, and the moving region extraction image is inverted in black and white A black-and-white reversing unit 103, a visible gain calculation unit 104 that performs specific weighting on the black-and-white reversal image, and a filter that enhances the vehicle and reduces noise with respect to the infrared image acquired by the infrared camera 112 This is a region that has been temporally changed from the infrared image using the inter-frame difference method or the background difference method for the filtered image. An infrared pre-processing unit 102 that generates an image (moving region extraction image) extracted, an infrared gain calculation unit 105 that performs specific weighting on the moving region extraction image, and a moving region are extracted and each has a specific weighting The image composition unit 106 that synthesizes the visible image and the infrared image, the composite image obtained by the image composition unit, information from the image processing unit in the subsequent stage, and the visible image, red using the time and weather information A gain control unit 107 that determines the weighting value of each of the outer images and an image processing unit 108 that detects a traveling vehicle using the composite image and grasps the traveling state thereof.

  Next, the overall operation of the present embodiment will be described in detail with reference to FIG. Here, it is assumed that a visible image is added to an infrared image.

  First, as a premise, the visible image acquired by the visible camera unit 111 is an image of the reflection intensity distribution of light. An image that matches the human visual image is obtained. On the other hand, the infrared image acquired by the infrared camera unit 112 is an image of the radiation distribution of heat (infrared rays), and the darker the color, the easier it is to warm, and the darker the more the infrared is at the same temperature. Is emitted (high emissivity), and thus is imaged as high luminance (white). Actually, when the image of the infrared camera is reversed in black and white, an image close to a visible camera can be obtained.

  The visible image acquired by the visible camera 111 and the infrared image acquired by the infrared camera 112 (S201) are respectively displayed in the visible pre-processing unit 101 and the infrared pre-processing unit 102 by utilizing the characteristics of the vehicle. After applying the filtering process for emphasizing or reducing noise (S202), only a region that has changed in time from the acquired image is obtained by means such as a difference from the previous frame or a difference from the background image acquired in advance. An extracted image (moving region extracted image) is generated (S203).

  As described above, the black-and-white reversing unit 103 makes use of the fact that an image having characteristics close to an infrared image can be obtained by reversing the visible image in black and white, and that the measurement environment conditions that each is good at are different. The extracted image is reversed in black and white (S204).

  The gain control unit 107 determines visible and infrared gains using the composite image, the image processing result using the composite image, and measurement environment conditions (time, weather, etc.) (S205). The visible gain multiplication unit 104 multiplies the visible image by a visible gain set according to the measurement environment condition (S206). The infrared gain multiplication unit 105 multiplies the infrared gain set for the infrared moving region extraction image (S207), and the image synthesis unit 106 adds the two images multiplied by the specific gain. Is generated (S208).

  The image processing unit 108 detects the traveling vehicle and grasps the traveling state by applying an image processing method for the related infrared image to the composite image (S209).

  In step S204, the black and white reversing unit 103 reverses the infrared moving region extracted image in black and white, and in step S209, the image processing unit 108 performs image processing on the visible image related to the composite image. By applying the above, it is possible to realize a pattern in which an infrared image is added to a visible image by detecting a traveling vehicle and grasping the traveling state thereof.

  According to the present embodiment described above, the vehicle detection accuracy is less affected by the sunlight conditions by the image processing using the composite image. The reason for this is that by increasing the mix ratio of infrared images during daytime sunny days, cloudy days, and nighttime when sunshine conditions are a problem, the illuminance decreases at night, shadows and fluctuations, ambient light due to the western sun, This is because the influence of a sudden change in illumination is reduced.

  In addition, vehicle detection accuracy during rainy weather is unlikely to decrease due to image processing using the composite image. The reason is that the contrast between the vehicle and the road surface, which has been reduced only by the infrared image in rainy weather, is improved by reversing the visible image in black and white and mixing it with the infrared image. At this time, the headlights and street light reflected from the streets, which had become high-intensity disturbance noise due to the mirroring of the road surface, become low-intensity due to black-and-white reversal. It is.

  Furthermore, if a composite image is used, road condition grasping utilizing the characteristics of a visible image and an infrared image can be realized with one existing image processing apparatus. The reason is that both visible and infrared images are mixed at an appropriate rate adapted to the environmental conditions.

  In the present embodiment, as shown in FIG. 1, a visible camera and an infrared camera image having the same field of view are used. The luminance image of the visible camera is an image of the reflection intensity distribution of light.Colors with strong reflection intensity are displayed with high luminance (white), and those with weak color and low luminance (black) are displayed. An image that matches the human visual image is obtained. On the other hand, an infrared camera images the radiation distribution of heat (infrared rays), and the darker the color, the easier it is to warm up, and the darker color radiates more infrared rays at the same temperature (the emissivity is higher). For this reason, the darker the color, the more infrared rays are received and the image tends to be imaged with high brightness (white). Actually, when the infrared camera image is reversed in black and white, an image close to the luminance image of the visible camera is obtained. Because of these characteristics, the measurement environment conditions that the road condition grasping apparatus using the visual camera image is good and the road environment using the infrared camera image are different from the measurement environment conditions that the apparatus is not good at.

  Therefore, in the present embodiment, the visible (infrared) image is reversed in black and white to generate a composite image that is mixed with the infrared (visible) image at a ratio optimal for the measurement environment condition, and the composite image is used. By performing image processing, it takes advantage of the features of infrared camera images that are strong in illuminance changes, clear / cloudy, and nighttime, and visible camera images that are strong in daytime cloudy and rainy days, and interpolates each weak point to provide daytime and nighttime characteristics. A road condition grasping system having a vehicle detection performance excellent in environmental resistance is provided. In this embodiment, the visible (infrared) video is reversed in black and white and mixed with the infrared (visible) video, the mix ratio is dynamically controlled according to the measurement environment conditions, and the image processing unit is already installed. It has the feature that it can utilize what was developed for infrared (visible).

  Each of the above-described embodiments is a preferred embodiment of the present invention, and various modifications can be made without departing from the scope of the present invention. For example, you may perform the process which implement | achieves the function of each apparatus by making each apparatus read and run the program for implement | achieving the function of a road condition grasping | ascertainment system. Further, the program is transmitted to another computer system by a transmission wave via a computer-readable recording medium such as a CD-ROM or a magneto-optical disk, or via a transmission medium such as the Internet or a telephone line. Also good. In addition, it is also within the scope of the present invention that the functions of each device are realized by other devices collectively or the functions are distributed by additional devices.

  According to the present invention, road conditions (detection of traveling vehicles and their traveling conditions), specifically, traffic volume measurement, traffic congestion detection, traffic jam end detection, stop / low speed vehicle detection, escape vehicle, accident vehicle, accident situation Can be applied to a purpose of stably grasping for 24 hours with real time and high reliability. It can also be used for purposes such as grasping the situation of pedestrians at intersections and detecting intruders and suspicious objects in buildings.

It is a block diagram which shows the system configuration | structure which concerns on embodiment of this invention. It is a flowchart which shows the operation | movement of the whole system which concerns on embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 111 Visible camera part 112 Infrared camera part 101 Visible image pre-processing part 102 Infrared image pre-processing part 103 Black-and-white inversion part 104 Visible gain multiplication part 105 Infrared gain multiplication part 106 Image composition part 107 Gain control part 108 Image processing part

Claims (8)

A visible camera and an infrared camera;
A gain control unit that generates a gain that is a weighted value of a ratio adapted to environmental conditions;
A gain multiplier that multiplies the image generated by the gain controller by an image obtained by the visible camera and the infrared camera;
An image composition unit for adding a visible image and an infrared image multiplied by the gain;
An image processing unit that detects a traveling vehicle using the composite image generated by the image composition unit and grasps the traveling state thereof, and
The gain, which is a weighted value of a ratio adapted to the environmental conditions, is characterized in that the infrared image mix ratio is set to a high value during sunny daytime, cloudy daytime and nighttime when the sunshine condition is a problem. Road condition grasping system. A visible camera and an infrared camera;
A gain control unit that generates a gain that is a weighted value;
A gain multiplier that multiplies the image generated by the gain controller by an image obtained by the visible camera and the infrared camera;
An image composition unit that adds a visible image to an infrared image multiplied by the gain;
An image processing unit that detects a traveling vehicle by infrared image processing using the synthesized image generated by the image synthesizing unit and grasps the traveling state thereof, and
The gain, which is a weighted value of a ratio adapted to the environmental conditions, is characterized in that the infrared image mix ratio is set to a high value during sunny daytime, cloudy daytime and nighttime when the sunshine condition is a problem. Road condition grasping system. A visible camera and an infrared camera;
A gain control unit that generates a gain that is a weighted value;
A gain multiplier that multiplies the image generated by the gain controller by an image obtained by the visible camera and the infrared camera;
An image composition unit for adding an infrared image to a visible image multiplied by the gain;
An image processing unit that detects a traveling vehicle by visible image processing using the composite image generated by the image composition unit and grasps the traveling state thereof, and
The gain, which is a weighted value of a ratio adapted to the environmental conditions, is characterized in that the infrared image mix ratio is set to a high value during sunny daytime, cloudy daytime and nighttime when the sunshine condition is a problem. Road condition grasping system.   The gain multiplication is provided with a visible image preprocessing unit and an infrared image preprocessing unit that perform predetermined filtering on an image obtained by the visible camera and the infrared camera to generate an image (moving region extracted image) obtained by extracting a moving region. 4. The road condition grasping system according to claim 1, wherein the unit multiplies the moving region extraction image by a gain generated by the gain control unit. 5. Capturing an image with a visible camera and an infrared camera;
A gain control step for generating a gain that is a weighted value of a proportion adapted to the environmental conditions;
A gain multiplication step of multiplying an image generated by the visible camera and the infrared camera by the gain generated in the gain control step;
An image synthesis step of adding a visible image and an infrared image multiplied by the gain;
An image processing step for detecting a traveling vehicle and grasping its traveling state using the composite image generated in the image composition step, and
The gain, which is a weighted value of a ratio adapted to the environmental conditions, is characterized in that the infrared image mix ratio is set to a high value during sunny daytime, cloudy daytime and nighttime when the sunshine condition is a problem. Road situation grasp method. Capturing an image with a visible camera and an infrared camera;
A gain control step for generating a gain that is a weighted value of a proportion adapted to the environmental conditions;
A gain multiplication step of multiplying an image generated by the visible camera and the infrared camera by the gain generated in the gain control step;
An image synthesis step of adding a visible image to the infrared image multiplied by the gain;
An image processing step of detecting a traveling vehicle by infrared image processing using the synthesized image generated in the image synthesizing step and grasping the traveling state thereof, and
The gain, which is a weighted value of a ratio adapted to the environmental conditions, is characterized in that the infrared image mix ratio is set to a high value during sunny daytime, cloudy daytime and nighttime when the sunshine condition is a problem. Road situation grasp method. Capturing an image with a visible camera and an infrared camera;
A gain control step for generating a gain that is a weighted value of a proportion adapted to the environmental conditions;
A gain multiplication step of multiplying an image generated by the visible camera and the infrared camera by the gain generated in the gain control step;
An image synthesis step of adding an infrared image to the visible image multiplied by the gain;
An image processing step of detecting a traveling vehicle by visual image processing using the synthesized image generated in the image synthesizing step and grasping the traveling state thereof, and
The gain, which is a weighted value of a ratio adapted to the environmental conditions, is characterized in that the infrared image mix ratio is set to a high value during sunny daytime, cloudy daytime and nighttime when the sunshine condition is a problem. Road situation grasp method. A visible image preprocessing step and an infrared image preprocessing step for generating an image (moving region extraction image) obtained by performing predetermined filtering on an image obtained by the visible camera and the infrared camera and extracting a moving region; The road condition grasping method according to any one of claims 5 to 7 , wherein in the multiplication step, the moving region extraction image is multiplied by the gain generated in the gain control step.

Images