JP2015153312A - vehicle driving support device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a vehicle driving support device which is mountable on a vehicle, whose cost, processing burden and processing time can be reduced and which can detect a point of time when a traffic light changes from red to green.SOLUTION: A vehicle driving support device 1 includes; a camera 10 which captures images of a scene ahead of a host vehicle and outputs images having luminance values corresponding to respective pixels; an image storage unit 12 for storing first and second images captured at a predetermined time interval; a differential image generation unit 20 which generates a differential image by calculating difference between the first and second images; an area/feature extraction unit 30 which extracts a particular area where the luminance value changes from the differential image and extracts a feature quantity corresponding to the particular area; and a traffic light change determination unit 40 which determines the presence of a traffic light that has changed from red to green based on the extracted feature quantity.

Description

  The present invention relates to a vehicle driving support device that performs a predetermined notification when a vehicle changes from a red signal to a green signal when stopping at an intersection or the like.

  Conventionally, a traffic light recognition device (see, for example, Patent Document 1) that recognizes a lighting color based on the color and circularity of a traffic light, or by turning on a traffic light by storing a luminance value when the stationary camera is turned off, 2. Description of the Related Art A signal identification device (for example, see Patent Document 2) that determines whether the light is turned off is known. By using these, it is possible to recognize the timing when the traffic light has changed from red to blue after stopping at an intersection with the traffic light.

JP 2009-244946 A JP 2000-353292 A

  By the way, in the traffic signal recognition apparatus disclosed in Patent Document 1 described above, since the lighting color is identified based on the luminance information, a color camera and an image memory for three colors of RGB are required, and the component cost increases. There was a problem that the burden of processing became heavy.

  Further, in the signal identification device disclosed in Patent Document 2 described above, it is necessary to take in a luminance value for one cycle of the traffic light (about 150 seconds) from an image obtained by photographing the entire lighting portion of the traffic light, and the vehicle traveling on the road Assuming the case where an image taken with a camera mounted on the camera is used, there is a problem that processing cannot be performed in a short time using a signal waiting time.

  The present invention was created in view of the above points, and its purpose is that it can be mounted on a vehicle and can reduce cost, reduce processing load, and shorten processing time. An object of the present invention is to provide a vehicle driving support device capable of detecting the time of change to a green light.

  In order to solve the above-described problem, a vehicle driving support apparatus according to the present invention captures an image of the front of the host vehicle and outputs an image having a luminance value corresponding to each pixel, and a predetermined time interval by the imaging unit. Image storage means for storing the first and second images picked up in step, and differential image generation for reading out the first and second images from the image storage means and calculating a difference between these images to generate a difference image And a region / feature extraction unit for extracting a specific area in which the luminance value changes from the difference image generated by the difference image generating unit, and extracting a feature amount corresponding to the specific region, and a region / feature Signal change determination means for determining whether there is a traffic light whose lighting state has changed from red to blue based on the feature amount extracted by the extraction means.

  Since it is possible to detect that the lighting state of the traffic light has changed from red to blue using an image having a luminance value instead of a color image, an inexpensive camera equipped with a control unit having a small number of image sensors and low processing capability is provided. In addition to being able to be used, it is possible to reduce the processing load and the processing time when processing is performed using the obtained image.

  In addition, it is desirable to further include notification means for performing a predetermined notification to the vehicle driver when it is determined by the signal change determination means that there is a traffic light whose lighting state has changed from red to blue. As a result, the driver who does not notice that the traffic light has changed from red to blue can be notified of the change, and the vehicle can be started with a sufficient margin.

  Moreover, it is desirable that the specific region described above has a circular shape. Thereby, when a lot of areas where the luminance value has changed are included, the specific area corresponding to the signal lamp can be narrowed down.

  The feature amount extracted by the region / feature extraction means described above preferably includes the direction in which the size, position, and luminance of the specific region change. This makes it possible to know the presence of signal lights that have changed from light-off to light-on, as well as their relative positions and sizes, and to identify traffic lights that have changed from red to blue. Is possible.

  Further, the signal change determination means described above has the same size of the two specific areas based on the feature quantity, the two specific areas are arranged side by side, and the luminance of the right or upper area changes. Determining that there is a traffic light whose lighting state has changed from red to blue when the direction in which the brightness changes and the direction in which the brightness of the left or lower specific area changes indicates an increase in brightness. desirable. This makes it possible to accurately determine whether the red signal lamp included in one traffic light is turned off and the green signal lamp is turned on.

  Each of the first and second images described above is preferably generated by averaging the luminance values of a plurality of images obtained by continuous imaging. As a result, even when a signal lamp using an LED (light emitting diode) as a light source blinks at a predetermined cycle, it is possible to reliably image the signal lamp in a lit state regardless of the blink timing.

  Further, the vehicle further includes a stop determination unit that determines whether or not the host vehicle is stopped, and when the stop determination unit determines that the host vehicle is stopped, a difference image generation unit, a region / feature extraction unit, It is desirable that the operation by the signal change determination means is performed. By performing the switching determination of the traffic light from red to blue only while the vehicle is stopped, the processing burden during traveling can be further reduced.

It is a figure which shows the structure of the vehicle driving assistance device of one Embodiment. It is a flowchart which shows the operation | movement procedure which detects and notifies when the traffic signal ahead of a vehicle changes from red to blue. It is a figure which shows the specific example of the imaged image and a difference image. It is explanatory drawing of the averaging of an image.

  Hereinafter, a vehicle driving support apparatus according to an embodiment to which the present invention is applied will be described with reference to the drawings. FIG. 1 is a diagram illustrating a configuration of a vehicle driving support device 1 according to an embodiment. The vehicle driving support device 1 is mounted on a vehicle and detects a point in time when the traffic light changes from a red light to a green light while the vehicle is stopped at a certain position of the traffic light, and performs a predetermined notification.

  As shown in FIG. 1, the vehicle driving support device 1 of this embodiment includes a camera 10, an image storage unit 12, a difference image generation unit 20, a region / feature extraction unit 30, a signal change determination unit 40, a notification unit 50, and a stop. A determination unit 60 is provided.

  The camera 10 has a predetermined range set in front of the vehicle as an imaging range, images the imaging range with a predetermined resolution, and outputs an image having a luminance value corresponding to each pixel. In this embodiment, since it is necessary to image a traffic light in front of the vehicle, an imaging range including at least a range of 4 m or more in front of the vehicle is set. Further, in the subsequent processing, a resolution that can identify the shape and brightness of the red signal light and the green signal light included in the traffic light is set. The camera 10 captures an imaging range, for example, every 1/30 seconds, and outputs an image obtained by each imaging. The image storage unit 12 stores an image output from the camera 10.

  The difference image generation unit 20 reads a set of images captured at a predetermined interval (for example, 1 second) from the image storage unit 12, calculates a difference between the set of images, and generates a difference image. This difference calculation is performed by calculating a difference in luminance value for each pixel constituting each image. Assuming that an image one second before is a “first image” and an image one second after the first image is a “second image”, the first value is determined based on the luminance value of each pixel constituting the second image. A difference image is generated by subtracting the luminance value of each pixel constituting the image.

  The region / feature extraction unit 30 extracts a circular-shaped region from the region where the luminance has changed between the first and second images from the difference image generated by the difference image generation unit 20, A feature amount corresponding to the extracted region is extracted. When the red light of the traffic light is continuously lit, the luminance value of each pixel of the first image and the luminance value of each pixel of the second image are the same, so each pixel included in the difference image The value of “0” is “0”. On the other hand, if the lighting state of the traffic light changes from red to blue between the first image capturing time point and the second image capturing time point, areas corresponding to the red signal light and the blue signal light in the difference image Has a value other than “0”. For example, as shown in FIG. 3A, the luminance value of each pixel corresponding to the red signal lamp included in the first image (D1) is R (> 0), and as shown in FIG. Assuming that the luminance value of each pixel corresponding to the green signal lamp included in the second image (D2) is + B (> 0), in the difference image (D3) shown in FIG. 3C, the value of the area corresponding to the red signal lamp Is “−R”, and the value of the area corresponding to the green light is “+ B”.

  Actually, in the first and second images, since the blinker of the vehicle included in the imaging range blinks or the brake lamp or the like is turned on or off, each pixel in the region corresponding to these in the difference image is also displayed. Has a value other than zero.

  The area / feature extraction unit 30 extracts a circular area from areas having values other than 0, and the size (diameter) and position (coordinates) of the circular area and the luminance gradient ( (Positive / negative) is extracted. In the example described above, the luminance gradient of the region having the value “−R” in the difference image is “negative”, and the luminance gradient of the region having the value “+ B” is “positive”. The luminance gradient is the direction in which the luminance changes, and the luminance value decreases when the luminance of each pixel of the second image is considered based on the first image (within the first image). “Negative” when the red signal light turned on is turned off in the second image), on the contrary, when the luminance value increases (the blue signal light turned off in the first image is the second light) It is “Positive” when it is lit in the image.

  The signal change determination unit 40 determines the presence / absence of a traffic light whose lighting state has changed from red to blue based on the feature amount corresponding to the circular region extracted by the region / feature extraction unit 30. Specifically, when there are two or more circular regions that are the target of feature extraction, select two regions included in them as a set, and whether the following relationship is satisfied for all combinations: To check.

(1) The size of the two areas must be the same. (2) The two areas should be aligned side by side or top and bottom. (3) The luminance gradient of the right or upper area is negative (red signal light), The luminance gradient of the area on the side is positive (green signal light). When there are two areas that satisfy all the relationships (1) to (3), the signal change determination unit 40 turns on one traffic light. It is determined that the state has changed from red to blue.

  The notification unit 50 performs a predetermined notification when the signal change determination unit 40 determines that the lighting state of the traffic light has changed from red to blue. For example, as a specific example of notification, a predetermined notification sound is output, a sound such as “changed to a green light” is output, a predetermined image is displayed on a display device (not shown), a predetermined in a meter panel It is conceivable to turn on (or blink) the display part.

  The stop determination unit 60 determines whether or not the host vehicle is stopped. For example, this determination is performed based on a vehicle speed pulse output at a certain distance from a vehicle speed sensor (not shown), or a signal indicating a vehicle running state input from a vehicle control device (not shown). The case where it carries out based on, etc. can be considered.

  The above-described camera 10 is an imaging unit, the image storage unit 12 is an image storage unit, the difference image generation unit 20 is a difference image generation unit, the area / feature extraction unit 30 is an area / feature extraction unit, and the signal change determination unit 40. Corresponds to the signal change determination unit, the notification unit 50 corresponds to the notification unit, and the stop determination unit 60 corresponds to the stop determination unit.

  The vehicle driving support apparatus 1 of this embodiment has such a configuration, and the operation thereof will be described next. FIG. 2 is a flowchart showing an operational procedure for detecting and notifying when the traffic light in front of the vehicle changes from red to blue.

  The stop determination unit 60 determines whether or not the host vehicle is stopped (step 100). If the host vehicle is traveling, a negative determination is made, and this determination is repeated. Further, when the host vehicle is stopped, an affirmative determination is made in the determination of step 100.

  Next, when imaging by the camera 10 is performed (step 102), the image storage unit 12 acquires and stores an image output from the camera 10 (step 104). Further, the difference image generation unit 20 reads out the image (first image) one second before together with this image (second image) from the image storage unit 12 (step 106), and calculates the difference between these two images. Then, a difference image is generated (step 108).

  Next, the region / feature extraction unit 30 extracts a circular region (specific region) in which the luminance included in the difference image is changed and a feature amount (size, coordinates, luminance gradient) corresponding to the specific region. (Step 110).

  Next, the signal change determination unit 40 determines whether or not the number of extracted circular regions (specific regions) is 2 or more (step 112). If there is one or no circular area, a negative determination is made, and the process returns to step 100 and the operations after the stop determination are repeated. If two or more circular regions are included, an affirmative determination is made in step 112.

  Next, the signal change determination unit 40 sets two areas as one set, and the characteristics when the lighting state of the traffic light changes from red to blue for all the combinations (the relations (1) to (3) described above). Is determined (step 114). If there is something having this characteristic, an affirmative determination is made, and the signal change determination unit 40 detects (determines) that the lighting state of the traffic light has changed from red to blue (step 116). Further, when there is no combination having a feature when the lighting state of the traffic light changes from red to blue, a negative determination is made in the determination in step 114, and the operation after the stop determination is repeated by returning to step 100. Next, the notification unit 50 notifies the driver that the lighting state of the traffic light has changed from red to blue (step 118).

  Thus, in the vehicle driving support device 1 of the present embodiment, it is possible to detect that the lighting state of the traffic light has changed from red to blue using an image having a luminance value instead of a color image. Therefore, it is possible to use an inexpensive camera 10 having a control unit with a low processing capability and a reduction in processing load and processing time when processing is performed using the obtained image.

  In addition, when it is determined that there is a traffic light whose lighting state has changed from red to blue, a predetermined notification is given to the driver of the vehicle so that the driver does not notice that the traffic light has changed from red to blue. The person can be notified of the change, and the vehicle can be started with a sufficient margin.

  In addition, by limiting the shape of the region (specific region) where the luminance value has changed to only a circular shape, the specific region corresponding to the signal lamp can be narrowed down when there are many regions where the luminance value has changed. .

  In addition, by including the size, position, and direction in which the luminance changes (luminance gradient) in the feature value extracted together with the specific area where the luminance value has changed, this is the opposite of a signal lamp that has been switched from on to off. It is possible to know the presence of signal lights that have been switched from off to on, and their relative positions and sizes, and to identify traffic lights that have changed from red to blue.

  In addition, based on the extracted feature amount, the sizes of the two specific areas match, and these two specific areas are arranged side by side or vertically, and the direction in which the luminance of the right or upper area changes changes in luminance. When the direction in which the luminance of the left or lower specific area changes indicates an increase in luminance, it is determined that there is a traffic light whose lighting state has changed from red to blue. This makes it possible to accurately determine whether the red signal lamp included in one traffic light is turned off and the green signal lamp is turned on.

  In addition, the processing load during traveling can be further reduced by determining whether the traffic light is switched from red to blue only while the vehicle is stopped.

  In addition, this invention is not limited to the said embodiment, A various deformation | transformation implementation is possible within the range of the summary of this invention. For example, in the above-described embodiment, the first and second images stored in the image storage unit 12 are read and these difference images are generated. Each of the first and second images is captured continuously. The brightness values of a plurality of images obtained by the above may be averaged.

  FIG. 4 is an explanatory diagram of image averaging. For example, images are captured at a rate of 30 images per second (interval of 33 ms), and the luminance values of the pixels of 8 consecutive images are averaged to generate the first or second image. . In the example shown in FIG. 4, the numbers 1, 2, 3,..., 31, 32, 33,. For example, if the first image (A1) is generated by averaging eight images 1 to 8, the eight images 31 to 38 captured one second later are averaged to obtain the second image. Image (A2) is generated. A series of operations shown in FIG. 2 is performed using the first image (A1) and the second image (A2) generated in this way. Similarly, the 8 images 2 to 9 are averaged to generate the first image (B1), and the 8 images 32 to 39 captured 1 second later are averaged to obtain the second image. (B2) is generated. A series of operations shown in FIG. 2 is performed using the first image (B1) and the second image (B2) generated in this way. Further, the eight images 3 to 10 are averaged to generate the first image (C1), and the eight images 33 to 40 captured one second later are averaged to obtain the second image (C2). ) Is generated. A series of operations shown in FIG. 2 is performed using the first image (C1) and the second image (C2) generated in this way.

  As described above, in the example shown in FIG. 4, the series of operations shown in FIG. 2 is repeated at intervals of 33 ms. As a result, even when a signal lamp using an LED (light emitting diode) as a light source blinks at a predetermined cycle, it is possible to reliably image the signal lamp in a lit state regardless of the blink timing.

  In the above-described embodiment, since the difference image is generated by calculating the difference between the first image and the second image, the luminance value is given to each pixel of the difference image. Instead of this, information obtained by binarizing the luminance value may be used. As a result, the data amount of the difference image can be reduced and the processing load using the difference image can be reduced.

  Further, before using the luminance value, each pixel luminance value of the difference image may be smoothed using a Gaussian filter or the like. This makes it possible to reliably separate each pixel whose luminance has changed from other pixels by binarization.

  As described above, according to the present invention, it is possible to detect that the lighting state of the traffic light has changed from red to blue using an image having a luminance value instead of a color image. It is possible to use an inexpensive camera provided with a unit, and to reduce the processing load and processing time when processing is performed using the obtained image.

DESCRIPTION OF SYMBOLS 1 Vehicle driving assistance device 10 Camera 12 Image storage part 20 Difference image generation part 30 Area | region / feature extraction part 40 Signal change determination part 50 Notification part 60 Stop determination part

Claims (7)

Imaging means for imaging the front of the host vehicle and outputting an image having a luminance value corresponding to each pixel;
Image storage means for storing first and second images imaged at predetermined time intervals by the imaging means;
Differential image generation means for reading the first and second images from the image storage means and calculating a difference between these images to generate a difference image;
A region / feature extraction unit that extracts a specific region in which a luminance value changes from the difference image generated by the difference image generation unit, and extracts a feature amount corresponding to the specific region;
Based on the feature amount extracted by the region / feature extraction unit, a signal change determination unit that determines whether there is a traffic light whose lighting state has changed from red to blue; and
A vehicle driving support device comprising: In claim 1,
Vehicle driving assistance, further comprising notification means for making a predetermined notification to the vehicle driver when it is determined by the signal change determination means that there is a traffic light whose lighting state has changed from red to blue. apparatus. In claim 1 or 2,
The vehicle driving support device according to claim 1, wherein the specific region has a circular shape. In any one of Claims 1-3,
The vehicle driving support apparatus, wherein the feature amount extracted by the region / feature extraction unit includes a direction in which the size, position, and luminance of the specific region change. In claim 4,
The signal change determining means is configured such that the two specific areas have the same size based on the feature amount, the two specific areas are arranged side by side, and the luminance of the right or upper area is When the direction of change indicates a decrease in luminance and the direction of change in the luminance of the specific area on the left or lower side indicates an increase in luminance, it is determined that there is a traffic light whose lighting state has changed from red to blue. A vehicle driving support device. Any one of claims 1 to 5
Each of the said 1st and 2nd image is produced | generated by averaging the luminance value of the several image obtained by the continuous imaging, The vehicle driving assistance apparatus characterized by the above-mentioned. In any one of Claims 1-6,
The vehicle further comprises stop determination means for determining whether or not the host vehicle is stopped,
When the stop determination means determines that the host vehicle is stopped, the vehicle is operated by the difference image generation means, the region / feature extraction means, and the signal change determination means. Support device.

Images