JP2008276291A - Method and apparatus for presenting sign recognition information - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a method for presenting sign recognition information that allows a driver to easily deal with a warning that he or she receives while driving at high speed. <P>SOLUTION: The method for presenting sign recognition information is for recognizing any road sign from a picked up image in front of a vehicle, and presenting the recognition result to the driver. A plurality of steps for the process of recognizing a sign are set according to the distance to the sign. Warnings based on the results of the plurality of steps of the process of recognizing the sign are presented to the driver. At a distance far enough for the driver to recognize the shape of the sign, a warning sound matching the result of recognizing the shape of the sign is presented to arouse the driver's attention by warning him or her at a stage prior to recognizing the content of the sign. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention belongs to the technical field of a method and an apparatus for presenting sign recognition information by image recognition.

Conventionally, an image is input to the image input unit by a color camera, and the color signal of the image input by the image processing calculation unit is converted into hue / saturation / lightness information, and a specific color of the sign is input from the input image. Extract the image into an extracted image, compare the shape of the extracted image with the underlying shape, recognize the sign, identify the sign with a relatively small amount of data and in a short time, and create an image captured by a color camera. The sign is easily identified based on this, and an alarm is issued or the vehicle is automatically controlled in response to a driver's carelessness or an oversight of a traffic sign due to random driving (see, for example, Patent Document 1). .
Japanese Patent Laid-Open No. 9-185703 (page 2-4, all figures)

  However, in the past, in order to effectively process color information, it is necessary to approach the target to some extent. This is conspicuous at low illumination such as at night. However, when the vehicle is traveling at high speed, it is slow to present the warning after the vehicle has come to a distance where the color information can be obtained, and it is difficult for the driver to respond with sufficient margin.

  The present invention has been made paying attention to the above-mentioned problems, and the object of the present invention is to provide sign recognition information that can be dealt with by a driver with a margin even when driving at high speed. It is to provide a presentation method and apparatus.

  In order to achieve the above object, in the present invention, in a method for presenting sign recognition information that recognizes a road sign from a captured image in front of a vehicle and presents a recognition result to a driver, the process of recognizing the sign is performed up to the sign. A plurality of stages are set in accordance with the distance of the warning, and a warning based on each of the recognition processing results of the plurality of stages is presented to the driver.

  Therefore, in the present invention, even when driving at a high speed, the driver receives a warning and can handle it with a margin.

  Hereinafter, an embodiment for realizing a method and an apparatus for presenting sign recognition information according to the present invention will be described based on Example 1 corresponding to the invention according to claims 1, 2, 3, 4, and 5.

First, the configuration will be described.
FIG. 1 is a block diagram of an apparatus that implements the method for presenting sign recognition information according to the first embodiment.
The sign recognition information presenting apparatus mainly includes an image processing unit 1, a control unit 2, a camera 3, a display unit 4, a warning sound output unit 5, and a vehicle speed 6.
The image processing unit 1 includes a sign position detection processing unit 11, a sign shape identification processing unit 12, and a sign content identification processing unit 13, and performs processing for recognizing a road sign by image processing captured by the camera 3.

The sign position detection processing unit 11 is a part that performs processing for detecting the presence of a sign from an image captured by the camera 3.
The sign shape identification processing unit 12 performs processing for identifying any of the sign shapes, for example, a round shape, a triangular shape, an inverted triangular shape, or a square (diamond) shape, from an image in which the presence of the sign is detected. It is.
The sign content identification processing unit 13 is a part that performs processing for identifying the contents of the sign by pattern matching.

The control unit 2 outputs a control command to the camera 3 and the image processing unit 1 based on the state of the image processing unit 1 and the state of the vehicle speed 6, and according to information indicating the state of the image processing unit 1 and the image processing result. The control command is output to the display unit 4 and the warning sound output unit 5.
The camera 3 captures an image in front of the vehicle. In the first embodiment, switching between color and monochrome is performed under the control of the control unit 2. Also, switching between color and infrared camera may be used.

The display unit 4 performs display according to the sign recognition state under the control of the control unit 2. These are equivalent to or equivalent to warnings, and the contents of signs. This will be described in detail later.
The warning sound output unit 5 outputs a warning sound and a warning announcement to the driver under the control of the control unit 2.
The vehicle speed 6 is vehicle speed information input from other vehicle devices or sensors.

The operation will be described.
[Signal recognition information presentation process]
FIG. 2 is a flowchart showing the flow of the label recognition information presentation process executed by the image processing unit 1 and the control unit 2 of the label recognition information presentation apparatus according to the first embodiment. Each step will be described below.
Note that the following processing is performed only when the sign recognition is difficult at night or during high-speed driving. In other cases, normal sign recognition / result display processing is performed.

  In step S <b> 1, the color imaged by the camera 3 is input to the image processing unit 1.

  In step S2, it is determined whether the input image has low illuminance and the vehicle speed is high. If the condition is satisfied, the process proceeds to step S3, and if not, the process proceeds to step S11.

  In step S3, the input image is switched to monochrome. Or switch to an infrared camera.

  In step S4, processing for detecting the presence of the sign in the input image is performed.

In step S5, it is determined whether a sign-like object has been detected. If detected, the process proceeds to step S6, and if not detected, the process returns to step S4.
If not detected in all the input images, the process ends or the process returns to step S1.

  In step S6, the fact that the marker position has been detected is highlighted on the display unit 4 (display).

  In step S7, a marker shape identification process is performed. It is determined whether the shape of the detected sign corresponds to, for example, a round shape, a triangular shape, an inverted triangular shape, or a quadrangular shape.

  In step S8, it is determined whether the shape of the sign corresponds to a triangle, an inverted triangle, or a quadrangle (diamond). If yes, the process proceeds to step S9, and if not, the process proceeds to step S13.

  In step S9, a warning sound is output. At that time, a short warning sound of about 0.2 seconds is presented to alert the driver.

  In step S10, the input image is switched to color.

  In step S11, a process for identifying the contents of the sign is performed.

In step S <b> 12, a warning announcement is output from the warning sound output unit 5.
In this step, “There is a pedestrian crossing”, “This is a temporary stop”, “Following is a junction,” etc. will be announced.

  In step S13, the sign content is displayed on the display unit 4 (display or HUD).

[Presentation of sign recognition information]
FIG. 3 is an explanatory diagram illustrating a state at a long distance in the method of presenting the label recognition information according to the first embodiment. FIG. 4 is an explanatory diagram illustrating a state at a medium distance in the method of presenting the marker recognition information according to the first embodiment. FIG. 5 is an explanatory diagram illustrating a state at a short distance in the method for presenting the label recognition information according to the first embodiment. FIG. 6 is an explanatory diagram illustrating a relationship between a warning sound, a warning announcement presentation time lag, and a driver's time lag in the method of presenting sign recognition information according to the first embodiment.

In the method for presenting sign recognition information according to the first embodiment, this function is turned on when the low illumination condition such as at night and the vehicle speed is 40 km / h or higher (step S1).
The input image is switched between a color camera and a monochrome camera (step S2).
An object that seems to be a sign is detected from the input image (step S5).
Judgment is made based on the brightness, size, aspect ratio, difference from surrounding brightness, brightness unevenness, and the like of the object. When a sign-like object is detected, the display unit 4 surrounds the mark with a red frame and performs highlighting (see step S6, FIG. 3).

As described above, in the first embodiment, when the distance to the sign is a long distance, information is presented according to the recognition of whether there is a road sign that can be recognized at the distance. This makes it possible to alert the driver from an early stage.
In this case, display is performed only on the display unit 4, and no warning sound is output. Therefore, the result cannot be seen unless the driver is aware of it. Since it is still uncertain, care should be taken not to bother it. It is not annoying because you cannot see it without consciousness.
Moreover, by performing this display, the operating state of the system can be confirmed, and the certainty of a warning to be transmitted reliably can be increased. In addition, the driver can confirm that the method for presenting the label recognition information according to the first embodiment is operating without any problem.

Next, when the sign shape approaches a detectable distance, the importance of the content is determined from the sign shape (step S7).
If the shape of the sign falls into one of a triangle shape indicating a pedestrian crossing, an inverted triangle shape indicating a stop, etc., or a square (diamond) shape generally used for warning signs, a warning sound is presented to the driver. Then, alerting is performed (step S8, see FIG. 4). In that case, it is more effective to judge the degree of urgency based on the sign shape and change the way the warning sound is played.

As described above, in the first embodiment, when the distance to the sign is a medium distance, information is presented according to the recognition of the shape of the road sign that can be recognized at the distance. This makes it possible to alert the driver from an early stage.
In addition, the warning sound processing performed for about 0.2 seconds at this stage reduces the time lag until the presentation of the sign contents performed after pattern matching of the sign contents at the next short distance stage in terms of the driver's experience. .
Furthermore, since the warning sound is not output when the importance of the sign is low, it is possible to take care not to bother the driver.

Next, when the distance of the sign color information can be obtained, the contents of the sign are identified by template matching, and a warning announcement corresponding to the sign contents is presented to the driver (see steps S10 to S13, FIG. 5).
In this way, when the distance to the sign is a short distance, information is presented according to the recognition of the content of the road sign that can be recognized at that distance. This makes it possible to alert the driver according to the contents of the sign.
Furthermore, the warning announcement and warning sound at this timing, the display on the display unit 4 and the like are for performing recognition processing when a warning sound is output when the shape of the sign is recognized just before that. It is possible to make the driver feel that the time lag for the time required for the vehicle is reduced.

As described above, in the first embodiment, the recognition process divided into stages according to the distance to the road sign and the presentation considering the recognition state and the distance to the road sign are performed in a plurality of stages, so that it can be performed at high speed or at night. Even in low-light conditions such as, the driver receives a warning and can deal with it with a margin.
Here, in the presentation considering the recognition state and the distance to the road sign, the processing time from the state in which the shape of the sign is recognized until the next pattern matching is performed to recognize the content of the sign is shown in FIG. Thus, it is considered to correspond to the distance of the vehicle to the road sign. Finally, during the processing time for recognizing the contents of the sign and presenting a warning announcement according to the contents, to alert the driver to the state in which the shape of the previous sign is recognized By presenting the warning, when the driver receives the final warning, the driver has already been alerted, so that the driver can receive the warning and cope with a margin.

Furthermore, in the presentation considering the recognition state and the distance to the road sign, the processing time from recognition of the sign shape to recognition of the sign content by performing the next pattern matching is 0.5-0. It takes about 8 seconds. In the first embodiment, by outputting a warning sound in a state in which the shape of the sign is recognized, the output time of about 0.2 seconds is excluded from the time when the driver feels the time lag in terms of experience, and the driver reduces the time lag. The feeling time is reduced to about 0.3 to 0.5 seconds (see FIG. 6).
As a result, before the warning announcement is made according to the sign content, the warning by the warning sound of the sign shape recognition result presented to the driver is prevented from being damaged by the driver feeling the time lag. By giving a sense of unity to the subsequent warning and the warning in the previous stage, it is possible to enhance the action of having a margin when the driver receives the warning.

Next, the effect will be described.
In the method for presenting label recognition information according to the first embodiment, the effects listed below can be obtained.

  (1) In the method of presenting sign recognition information that recognizes road signs from the captured image in front of the vehicle and presents the recognition result to the driver, a process for recognizing the signs is set in multiple stages according to the distance to the sign, In order to present warnings to the driver based on each of the recognition processing results of the signs made in multiple stages, the warning is given to the driver from the stage before recognizing the contents of the signs, so even during high-speed driving, The driver receives a warning and can respond with a margin.

  (2) Upon reaching the distance at which the sign shape can be recognized, the process of step S4 for recognizing the sign shape is performed, and a warning sound based on the sign shape is presented to the driver by the process of step S9. In order to present the warning based on the contents of the sign to the driver in the processes of steps S12 and S13 by performing the process of step S11 for recognizing the contents of the sign when the distance at which the contents of the sign can be recognized is reached. A warning sound according to the recognition result of the sign shape is presented at a distance that can recognize the sign shape, and a warning is given to the driver from the stage before recognizing the sign contents. Even if there is, the driver receives a warning and can cope with it with a margin.

  (3) As a step prior to the step of recognizing the shape of the sign, the process of step S4 for recognizing the presence of the sign is performed at a stage where the distance capable of recognizing the presence of the sign is reached. Since only the display on the display unit 4 in step S6 indicating that the presence of the sign has been recognized is performed, the driver can not see that the sign has been recognized unless the driver is conscious of it, so that the driver does not feel bothered. it can. Further, when the driver sees the display content on the display unit 4, the operating state of the system can be confirmed.

  (4) The warning sound output based on the shape of the sign (step S9) is performed in the interval time until the warning is started based on the result of the process for recognizing the content of the sign performed thereafter. Reduces the time lag until the warning announcement based on the recognition result of the content starts, gives the front and rear warnings a sense of unity, prevents the driver's alerting from being impaired in the previous stage, and even during high-speed driving The driver receives a warning and can respond with a margin.

  (5) In a sign recognition information presentation apparatus having an image processing unit 1 for recognizing a road sign from a captured image in front of a vehicle, a display unit 4 for presenting a recognition result to a driver, and a warning sound output unit 5, an image processing unit Reference numeral 1 denotes a label position detection processing unit 11 that recognizes the presence of the label at a stage where the distance that can recognize the presence of the label is reached, and a label that recognizes the shape of the label when the distance that can recognize the shape of the label is reached. At the stage when the distance that can recognize the content of the sign and the shape identification processing unit 12 is reached, the label content identification processing unit 13 that recognizes the content of the sign is provided, and the display unit 4 and the warning sound output unit 5 are provided at each recognition stage. Based on the recognition result, in order to present a warning to the driver at each recognition stage, display is performed on the display unit 4 according to the recognition result of the sign position at the distance where the presence of the sign is recognized, and the sign shape is recognized. At a distance that can be The warning sound output unit 5 outputs a warning sound according to the shape recognition result, recognizes the sign contents, and alerts the driver from the stage before the warning sound output unit 5 issues the warning announcement. Therefore, even when driving at high speed, the driver can receive a warning and respond with sufficient margin.

As mentioned above, although the presentation method and apparatus of the label | marker recognition information of this invention were demonstrated based on Example 1, it is not restricted to these Examples about a concrete structure, Each claim of a claim Design changes and additions are permitted without departing from the spirit of the invention.
For example, in the first embodiment, the color camera and the monochrome camera are switched. However, if the monochrome camera is an infrared camera, the effect can be further improved. Further, only a color camera may be used.
For example, the display unit may be an independent warning display or a display unit of another device. Moreover, HUD may be sufficient.
For example, the warning sound output unit may present other than the warning sound and the warning announcement as long as it is a sound or a voice.

It is a block diagram of the apparatus which implements the presentation method of the label recognition information of Example 1. It is a flowchart figure which shows the flow of the presentation process of the label | marker recognition information performed with the image process part of the presentation apparatus of the label | marker recognition information of Example 1, and a control part. It is explanatory drawing which shows the state at the time of long distance in the presentation method of the label | marker recognition information of Example 1. FIG. It is explanatory drawing which shows the state at the time of the middle distance in the presentation method of the label | marker recognition information of Example 1. FIG. It is explanatory drawing which shows the state at the time of short distance in the presentation method of the label | marker recognition information of Example 1. FIG. It is explanatory drawing which shows the relationship of the time lag which a warning sound and the warning announcement presentation time lag in the presentation method of the label recognition information of Example 1 feels.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Image processing part 11 Sign position detection processing part 12 Sign shape identification processing part 13 Sign content identification processing part 2 Control part 3 Camera 4 Display part 5 Warning sound output part 6 Vehicle speed C Vehicle S

Claims (5)

In the method of presenting sign recognition information for recognizing a road sign from a captured image in front of the vehicle and presenting a recognition result to the driver,
A process for recognizing the sign is set in a plurality of stages according to the distance to the sign, and a warning based on each of the plurality of signs recognition processing results is presented to the driver.
A method for presenting sign recognition information characterized by the above. In the presentation method of the sign recognition information according to claim 1,
At the stage where the distance that can recognize the shape of the sign is reached, a process for recognizing the shape of the sign is performed, and a warning based on the shape of the sign is presented to the driver,
Thereafter, at a stage where the distance at which the content of the sign can be recognized is reached, a process for recognizing the content of the sign is performed, and a warning based on the content of the sign is presented to the driver.
A method for presenting sign recognition information characterized by the above. In the presentation method of the sign recognition information according to claim 2,
As a step prior to the step of recognizing the shape of the sign, a process of recognizing the presence of the sign is performed at a stage where the distance at which the presence of the sign can be recognized is reached,
If the sign is present, only display indicating that the presence of the sign has been recognized.
A method for presenting sign recognition information characterized by the above. In the presentation method of the sign recognition information according to claim 2 or claim 3,
The warning based on the shape of the sign is performed in an interval time until the warning based on the result of the process for recognizing the content of the sign performed thereafter is started.
A method for presenting sign recognition information characterized by the above. In a sign recognition information presenting device having means for recognizing a road sign from a captured image in front of a vehicle and a presenting means for presenting a recognition result to a driver,
The sign recognition means includes:
A label position detecting means for recognizing the presence of the label at a stage where the distance capable of recognizing the presence of the label is reached;
A sign shape identifying means for recognizing the shape of the sign when reaching a distance where the shape of the sign can be recognized;
A sign content identifying means for recognizing the contents of the sign when reaching a distance at which the contents of the sign can be recognized;
With
The presenting means presents a warning to the driver at each recognition stage based on the recognition result at each recognition stage.
A device for presenting sign recognition information.

Images