JP5423696B2 - Sign recognition device - Google Patents

Description

  The present invention relates to a sign recognition device that is mounted on a vehicle and recognizes a road sign of a road on which the host vehicle travels.

  Conventionally, a sign recognition device that recognizes a road sign based on image data in front of the host vehicle captured by a camera is known (see, for example, Patent Document 1). In Patent Document 1, the recognition result of the road sign is presented to the driver within a recognizable range according to the distance from the host vehicle to the road sign. For example, as the vehicle approaches the road sign, the presentation method is changed such as highlighting the recognized road sign on the display, generating a warning sound, and outputting the sound of the sign content.

JP 2008-276291 A

  According to the technology disclosed in Patent Document 1, based on the recognition accuracy of the image data according to the distance to the road sign, the recognition result of what can present the road sign to the driver within the recognizable range is presented as it is. Misrecognized road signs may be presented.

  The present invention has been made to solve the above-described problem, and improves recognition accuracy of road signs by recognizing road signs as fixed signs and temporary signs based on the installed positions. An object is to provide a sign recognition device.

The inventor of the present application paid attention to the fact that even the same road sign can be distinguished from a fixed sign that is always installed at the same position and a temporary sign that is temporarily installed during construction or the like and whose installation position is not specified.
Since the fixed signs are road signs that are always installed at the same position, the information is usually registered in the map data. Therefore, for example, it is conceivable to improve the recognition accuracy by comparing the type of the fixed sign registered in the map data with the recognized type of the fixed sign.

  On the other hand, the temporary sign is a sign that is temporarily installed and is not registered in the map data. Therefore, unlike the fixed sign, the sign type registered in the map data cannot be collated with the recognized sign type.

  However, when used as a temporary sign in road signs, for example, there are installation conditions such as being installed next to other temporary signs in the traveling direction of the vehicle, but not being installed next to fixed signs. Some are set. In addition, if it can be recognized as a temporary sign, the recognized temporary sign is registered in the map data, and when the temporary sign is recognized at the same position, the recognized sign type can be compared with the registered sign type in the same way as the fixed sign. .

  Therefore, if a road sign is recognized separately from a fixed sign and a temporary sign, an appropriate method for recognizing the temporary sign with high accuracy can be obtained by installing the temporary sign or registering it in the map data. Can be considered.

Therefore, according to the first to fourth aspects of the present invention, the sign recognition device is based on image acquisition means for acquiring image data in front of the host vehicle captured by the in-vehicle camera, and image data acquired by the image acquisition means. A recognition means for recognizing a road sign, a sign position detection means for detecting a position of the road sign recognized by the recognition means, a position detected by the sign position detection means for the road sign recognized by the recognition means, and a map data storage means A determination means for determining whether the road sign recognized by the recognition means is a fixed sign that is always installed or a temporary sign that is temporarily installed from the stored map data; and the recognition means Notification control means for informing the notification means of the recognized road sign .

  Fixed signs are installed at the same position, and temporary signs are usually installed at unspecified positions different from fixed signs, so the position of the recognized road sign on the map is determined from the position of the road sign and map data. I understand. Thereby, it can be determined whether the recognized road sign is a fixed sign or a temporary sign. Then, by recognizing whether it is a fixed sign or a temporary sign, an appropriate method for judging whether the sign is recognized normally or erroneously is selected for the temporary sign. The recognition accuracy of the sign can be improved.

Furthermore, according to the invention described in 4 of claims 1, determine the constant means is a road sign is fixed label recognition means recognizes previous case road sign recognition means recognizes this time the release of the temporary label recognition When the determination means determines that the current recognition by the means is a misrecognition, and the notification control means determines that the recognition of the current road sign by the recognition means is a misrecognition, the notification means notifies the road sign recognized by the recognition means this time. Do not notify the means.

  When the recognition means normally recognizes the temporary sign, the temporary sign recognized this time is notified, and when the temporary sign is erroneously recognized, for example, the previously recognized road sign remains notified. As a result, an accurate road sign can be notified to the vehicle occupant.

According to the invention described in claim 2, when the determination means determines that the road sign recognized by the recognition means this time is a temporary sign, the temporary sign recognized by the recognition means is added to the map data stored in the map data storage means. Registration control means for registering and not registering the temporary sign recognized by the recognition means this time in the map data stored in the map data storage means when the determination means determines that the recognition of the current road sign by the recognition means is erroneous recognition Is provided.

  According to this configuration, if the temporary sign recognized this time by the recognition means is normal, it is registered in the map data. Therefore, the temporary sign acquired from the map data and the temporary sign recognized by the recognition means when passing the same position next time. By recognizing the sign, the recognition accuracy of the temporary sign is improved.

According to the invention described in claim 3 , when the road sign recognized by the recognition means exists within a predetermined range from the node in the map data based on the position detected by the sign position detection means for the road sign recognized by the recognition means. Is determined as a fixed sign, and when it is outside the predetermined range, it is determined as a temporary sign.

Thus, by using the difference in the installation position between the fixed sign and the temporary sign in the map data, it can be recognized with high accuracy whether it is a fixed sign or a temporary sign.
According to the invention described in claim 4 , the sign position detection means is based on the position of the own vehicle acquired from the own vehicle position detection means and the distance from the own vehicle to the road sign recognized by the recognition means. The position of the road sign recognized by is detected.

  For example, if the vehicle is equipped with GPS (Global Positioning System) as the vehicle position detection means, the position of the recognized road sign is accurately determined based on the vehicle position and the distance from the vehicle to the road sign. It can be detected. The distance from the host vehicle to the road sign is determined based on, for example, the recognized standard size of the road sign and the number of pixels on the image data or, if there is a distance measuring device such as a laser, from the detection result. Can do.

  The functions of the plurality of means provided in the present invention are realized by hardware resources whose functions are specified by the configuration itself, hardware resources whose functions are specified by a program, or a combination thereof. The functions of the plurality of means are not limited to those realized by hardware resources that are physically independent of each other.

1 is a block diagram showing a sign recognition system according to an embodiment. (A) is a schematic diagram showing an example in which a temporary sign is erroneously recognized, and (B) is a schematic diagram showing an example in which the recognized temporary sign is registered in map data. Explanatory drawing which shows the installation conditions of a road sign. The flowchart which shows a label | marker recognition process. The flowchart which shows a label | marker recognition process. The flowchart which shows a label | marker recognition process.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
FIG. 1 shows a sign recognition system 10 of the present embodiment. The sign recognition system 10 includes a camera 20, a sign recognition device 30, a GPS 50, a map DB 52, a display 60, an indicator 62, a speaker 64, and the like, and is mounted on a vehicle.

The camera 20 photoelectrically converts the front image of the vehicle incident from the lens 22 by the image sensor 24 to generate image data.
The sign recognition device 30 includes an image creation unit 32, a sign detection unit 34, a sign recognition unit 36, a determination unit 38, a notification control unit 40, a registration control unit 42, and the like. The image creating unit 32, the sign detecting unit 34, the sign recognizing unit 36, the determining unit 38, the notification control unit 40, and the registration control unit 42 constituting the sign recognition device 30 are each a microcomputer comprising a CPU, a RAM, a ROM, and the like. (Also referred to as a microcomputer), or a plurality may be configured by one microcomputer.

  The image creating unit 32 performs contour processing for detecting the contour (edge) of the object from the image data in front of the vehicle acquired from the image sensor 24. The sign detection unit 34 detects a road sign similar to the shape of the road sign from the contour image created by the image creation unit 32.

The sign recognition unit 36 performs image analysis on the image data detected by the sign detection unit 34 as a road sign, and recognizes the type of the road sign based on the outer shape, color, characters and figures in the sign.
The determination unit 38 determines whether the recognized road sign is a fixed sign that is always installed or a temporary sign that is temporarily installed. Determine whether or not.

  When the road sign is recognized normally, the notification control unit 40 displays the road sign recognized this time as an image on the display 60, turns on the indicator 62 corresponding to the recognized road sign, or recognizes the road sign recognized. Is output from the speaker 64 as a sound, and the road sign recognized by the driver is notified, and when it is erroneously recognized, the driver is not notified. In the present embodiment, a speed regulation sign will be described as an example of a road sign notified to the driver.

The registration control unit 42 registers the recognized temporary sign type and coordinates in the map DB 52 when the temporary sign is normally recognized, and does not register the temporary sign in the map DB 52 when the temporary sign is erroneously recognized.
(Sign recognition)
As shown in FIG. 2, the map data stored in the map DB 52 includes links and nodes representing roads. A link connects nodes indicating an intersection, a branch point, a junction, and the like. A road is constructed by connecting the links.

  Registered in the link are data such as an identification number, a link length, a latitude / longitude indicating coordinates of a start point and an end point, a road type indicating a highway, a national road, and the like, and the number of lanes. In the node, an identification number, coordinates indicating the position of the node, an identification number of a link connected to the node, a type of road sign installed within a predetermined range from the node, and the like are registered.

  The aforementioned fixed sign is placed within a predetermined range from the position indicated by the node on the map data, for example, within a range of 10 m from the node along the road. On the other hand, the temporary sign is installed outside a predetermined range with respect to the node.

  The position where the road sign is installed is known from the own vehicle position acquired from the GPS 50 and the distance from the own vehicle to the recognized road sign. Then, it can be determined whether the recognized road sign is a fixed sign or a temporary sign from the installation position of the road sign and the map data registered in the map DB 52.

  In the case of fixed signs, the weight of recognition accuracy of the type of road sign registered in the node is increased by comparing the type of road sign registered in the node with the type of recognized road sign. The accuracy when recognizing a road sign based on image data can be improved.

  On the other hand, since the temporary sign is a sign temporarily installed, it is not registered in the map DB. Therefore, it is necessary to recognize the road sign by analyzing the image data acquired from the camera 20.

  In this case, as shown in FIG. 2A, for example, when an object similar to the release marker is imaged outside a predetermined range from the node, the temporary marker is displayed when the contour processing for the image data and the pattern processing of the image data are performed. There is a risk of misrecognizing it as a release sign. The analog of the release sign shown in FIG. 2A is a rod-like shape, although the release sign and the mark are similar, and the contour is different from the actual circular release sign. However, if contour processing is performed including image data behind similar objects, there is a possibility of erroneous recognition as a circular contour.

Here, as shown in FIG. 3, the change of the restriction speed by the temporary speed restriction sign is effective regardless of whether the speed restriction sign recognized last time is a fixed sign or a temporary sign.
On the other hand, the temporary release sign can be released only when the previously recognized speed restriction sign is the temporary speed restriction sign, and when the previously recognized speed restriction sign is a fixed speed restriction sign, the temporary release sign Then, speed regulation cannot be canceled. That is, when the road sign recognized this time is a temporary release sign and the road sign recognized last time is a fixed speed regulation sign, it can be determined that the recognition as the temporary release sign this time is a false recognition.

  On the other hand, as shown in FIG. 2B, for example, a speed regulation sign can be installed as a temporary sign regardless of whether the previously recognized road sign is a fixed sign or a temporary sign. Therefore, when a speed regulation sign is recognized as a temporary sign, coordinates may be registered in the map DB 52 together with the type of the sign as a temporary speed restriction sign. As a result, the next time the vehicle passes the same road, the temporary sign can be recognized with high accuracy by comparing the temporary speed regulation sign registered in the map DB 52 with the currently recognized road sign.

(Sign recognition recognition processing)
Next, road sign recognition processing and display processing will be described based on the flowcharts of FIGS. 4 to 6, speed restriction signs and release signs will be described as examples of road signs. 4 to 6, “S” represents a step. The flowchart of FIG. 4 shows the main routine of the sign recognition notification process, which is always executed. 4 will be described in detail with reference to FIG. 5, and S410 will be described in detail with reference to FIG.

  First, in FIG. 4, the sign recognition device 30 acquires captured image data from the camera 20 (S400), performs contour detection processing on the acquired image data, and performs detection for performing contour circle detection processing. Image data is created (S402).

Then, a road sign having a circular outline is detected from the image data subjected to the outline detection process (S404).
Then, pattern recognition processing is performed on the image data detected as a road sign having a circular outline to determine whether it is a speed regulation sign or a release sign (S406).

  If the recognized road sign is a speed restriction sign or a release sign, it is determined whether it is a fixed sign or a temporary sign. If it is a temporary sign, the recognized road sign is registered in the map DB 52 (S408). .

  If the recognition result using the release sign as a temporary sign is normal, the speed restriction sign displayed on the display 60 or the indicator 62 is erased, or a voice is output from the speaker 64 that the speed restriction has been released (S410).

  On the other hand, if the recognition result using the release sign as a temporary sign is erroneous recognition, the speed restriction sign displayed on the display 60 or the indicator 62 is displayed as it is, or the speed restriction from the speaker 64 is continued. Is output as voice, and the data of the temporary sign registered in the map DB 52 is deleted (S410).

(Fixed sign / temporary sign judgment processing)
FIG. 5 is a flowchart for explaining in detail the processing in S408 of FIG.
In S420 of FIG. 5, the sign recognition device 30 determines whether the recognized road sign is a speed regulation sign or a release sign based on the determination result of S406 of FIG. When it is neither a speed regulation sign nor a release sign (S420: No), the sign recognition device 30 ends the process of the main routine of FIG. In this case, when the speed regulation sign is displayed on the display 60 or the indicator 62, it is left as it is. Also, nothing is output as sound from the speaker 64.

  In the case of either a speed restriction sign or a release sign (S420: Yes), node information around the host vehicle is acquired from the map DB 52 based on the position information of the host vehicle acquired from the GPS 50 (S422).

  The distance from the host vehicle to the recognized road sign can be calculated based on, for example, the size of the road sign determined by the standard and the number of pixels of the image data. Then, based on the calculated distance to the road sign, the position information of the host vehicle, and the position information of the node, whether the recognized road sign exists within a predetermined range from the node or outside the predetermined range. Determination is made (S424).

If the recognized road sign is within a predetermined range from the node (S424: Yes), the recognized road sign is determined to be a fixed sign that is always installed (S426).
If the recognized road sign is outside the predetermined range from the node (S424: No), the recognized road sign is determined to be a temporary sign temporarily installed during construction or the like (S428). Since the temporary sign is not registered in the map DB 52 in the initial state, the type of temporary sign and its coordinates are registered in the map DB 52 (S430).

  In S432, the sign recognition device 30 stores the current determination result of whether the recognized road sign is a speed regulation sign or a release sign, and whether it is a fixed sign or a temporary sign, and S410 in FIG. Specifically, the process proceeds to S440 in FIG. In S432, at least determination results for the previous time and the current time are stored.

(Speed regulation sign notification process)
FIG. 6 to be described next is a flowchart for explaining in detail the processing in S410 of FIG.

  In S440 of FIG. 6, the sign recognition device 30 determines whether or not the road sign recognized this time is a temporary release sign based on the determination result stored in S432 of FIG. When the road sign recognized this time is not a temporary release sign (S440: No), the sign recognition apparatus 30 determines whether the road sign recognized this time is either a fixed or temporary speed regulation sign or a fixed release sign. Judge that there is. In this case, the sign recognition device 30 determines that the road sign has been recognized normally, and the process proceeds to S446.

  When the road sign recognized this time is a temporary release sign (S440: Yes), the determination result whether the road sign recognized last time is a fixed sign or a temporary sign is acquired (S442), and the road sign recognized last time is It is determined whether or not it is a temporary sign (S444).

  When the previously recognized road sign is a temporary sign (S444: Yes), the sign recognition device 30 recognizes the temporary release sign recognized this time because the road sign recognized before the temporary release sign recognized this time is the temporary sign. Is determined to have been recognized normally, and the process proceeds to S446.

  The notification content update process of S446 is executed when the road sign recognized this time is not a temporary release sign (S440: No), that is, either a fixed or temporary speed regulation sign or a fixed release sign. Alternatively, the road sign recognized this time is a temporary release sign (S440: Yes), and the previously recognized road sign is a temporary sign (S444: Yes).

  When the road sign recognized this time is a fixed or temporary speed regulation sign, the sign recognition device 30 updates the content displayed on the display 60 or the indicator 62 or the speed regulation sign that recognizes the content to be output from the speaker 64 ( S446). In S446, the notification content is updated, but the actual notification is executed in S456.

  When the road sign recognized this time is a fixed release sign, or when the road sign recognized this time is a temporary release sign and the road sign recognized last time is a temporary sign, the sign recognition device 30 displays the display 60 or the indicator 62. The content to be displayed is updated by erasing the content, or the content to be output from the speaker 64 is updated so that the speed restriction is released (S446).

  The notification content updated in S446 gives the driver a sense of incongruity when the actual notification timing varies, so a predetermined notification timing is calculated (S448). The predetermined notification timing is, for example, when the host vehicle reaches the recognized road sign. This notification timing is determined by calculating the time required for the host vehicle to reach the recognized road sign from the distance from the host vehicle to the recognized road sign and the traveling speed of the host vehicle. After executing the process of S448, the process proceeds to S456.

  In the determination of S444, if the road sign recognized last time is not a temporary sign but a fixed sign (S444: No), the sign recognition device 30 does not set a temporary release sign next to the fixed sign, so this temporary release It is determined that the recognition of the sign is a misrecognition (S450).

  Then, the sign recognition device 30 determines that the previous content is not updated without updating the content displayed on the display 60 or the indicator 62, or that no sound is output from the speaker 64 (S452). Then, in S430 of FIG. 5, the information of the temporary release sign that is erroneously recognized this time is registered in the map DB 52, so that information is deleted (S454), and the process proceeds to S456.

  In S456, the sign recognition device 30 notifies the currently valid road sign through the display 60, the indicator 62, or the speaker 64 according to the notification contents set in S446 or S452, and ends the processing of the main routine of FIG.

  In the above embodiment, the camera 20 corresponds to the in-vehicle camera of the present invention, the sign recognition device 30 corresponds to the sign recognition device of the present invention, the GPS 50 corresponds to the vehicle position detection means of the present invention, the display 60, the indicator At least one of 62 and the speaker 64 corresponds to the notification means of the present invention, and the map DB 52 corresponds to the map data storage means of the present invention. The image creating unit 32 of the sign recognition device 30 corresponds to the image acquisition unit of the present invention, the sign detection unit 34 and the sign recognition unit 36 correspond to the recognition unit of the present invention, and the determination unit 38 is the determination unit of the present invention. The notification control unit 40 corresponds to the notification control unit registration control unit of the present invention, and the registration control unit 42 corresponds to the registration control unit of the present invention.

  4 corresponds to the function executed by the image acquisition means of the present invention, and the processing of S402 to S406 of FIG. 4 corresponds to the function executed by the recognition means of the present invention. The process of S408 of FIG. The processing corresponds to the functions executed by the vehicle position detection means, the sign position detection means, the determination means and the registration control means of the present invention, and the processing of S410 in FIG. 4 is the determination means, notification control means and registration control means of the present invention. Corresponds to the function to be executed.

  5 corresponds to the function executed by the determination means of the present invention, and the process of S424 of FIG. 5 is executed by the vehicle position detection means and the sign position detection means of the present invention. 5 corresponds to a function, and the process of S430 in FIG. 5 corresponds to a function executed by the registration control means of the present invention.

  6 corresponds to the function executed by the determination means of the present invention, and the processes of S446, S448, S452, and S456 of FIG. 6 are functions executed by the notification control means of the present invention. 6 corresponds to the function executed by the registration control means of the present invention.

  According to the embodiment described above, by determining whether the road sign recognized from the image data is a fixed sign or a temporary sign, the temporary sign recognized this time is a release sign, and the road sign recognized last time is In the case of a fixed sign, it is determined that the temporary release sign recognized this time is erroneous recognition based on the fact that the temporary release sign can be installed only when the previously recognized road sign is a temporary sign. Thereby, the recognition accuracy of the temporary release sign is improved.

  Further, when the temporary release sign is recognized normally this time, the type of the temporary sign is registered together with the coordinates in the map DB 52 in the same manner as when the temporary speed restriction sign is recognized. As a result, the information on the temporary sign can be acquired from the map DB 52 the next time it passes through the same position. Therefore, the type of the temporary sign recognized is compared with the type of the temporary sign acquired from the map DB 52. It can be recognized with accuracy.

[Other Embodiments]
In the above embodiment, when the notification control means adjusts the notification state of the road sign based on the determination result of whether the temporary release sign is normally recognized or erroneously recognized, and the temporary release sign is recognized normally. Registered the recognized temporary road sign in the map DB 52 in the same manner as when the temporary speed regulation sign was recognized.

On the other hand, the recognized temporary sign may be registered in the map DB 52 without determining whether the temporary release sign is normally recognized or erroneously recognized, regardless of whether or not the temporary sign is a release sign.
In the said embodiment, the speed regulation sign was illustrated as a road sign alert | reported by alerting | reporting means, such as the display 60. FIG. Other road signs may be notified by the notification means.

  In the above embodiment, the image acquisition means, recognition means, sign position detection means, determination means, notification control means, and registration control means are realized by the sign recognition device 30 whose function is specified by the control program. On the other hand, at least some of the functions of the plurality of means may be realized by hardware whose functions are specified by the circuit configuration itself.

  As described above, the present invention is not limited to the above-described embodiment, and can be applied to various embodiments without departing from the gist thereof.

20: Camera (vehicle-mounted camera), 30: Sign recognition device (image acquisition means, recognition means, sign position detection means, determination means, notification control means, registration control means), 50: GPS (own vehicle position detection means), 52 : Map DB (map data storage means), 60: Display (notification means), 62: Indicator (notification means), 64: Speaker (notification means)

Claims (4)

In a sign recognition device that is mounted on a vehicle and recognizes a road sign of a road on which the vehicle travels,
Image acquisition means for acquiring image data ahead of the host vehicle imaged by the in-vehicle camera;
Recognition means for recognizing a road sign based on image data acquired by the image acquisition means;
Sign position detection means for detecting the position of the road sign recognized by the recognition means;
A fixed sign in which the road sign recognized by the recognition means is always installed from the position detected by the sign position detection means for the road sign recognized by the recognition means and the map data stored in the map data storage means. Or a determination means for determining whether it is a temporary sign temporarily installed,
Notification control means for informing the notification means of the road sign recognized by the recognition means;
With
When the road sign recognized by the recognition means last time is the fixed sign and the road sign recognized this time by the recognition means is a temporary release sign, the determination means recognizes the current road sign incorrectly. Determine that it is recognition,
When the determination unit determines that the recognition of the current road sign by the recognition unit is erroneous recognition, the notification control unit does not cause the notification unit to notify the road sign recognized by the recognition unit;
A sign recognition device characterized by that. When the determination means determines that the road sign recognized this time by the recognition means is the temporary sign, the temporary sign recognized by the recognition means is registered in the map data stored in the map data storage means, Registration control not to register the temporary sign recognized by the recognition unit this time in the map data stored in the map data storage unit when the determination unit determines that the recognition of the current road sign by the recognition unit is erroneous recognition The sign recognition apparatus according to claim 1 , further comprising means. When the road sign recognized by the recognition means is within a predetermined range from the node in the map data based on the position detected by the sign position detection means for the road sign recognized by the recognition means. The sign recognition apparatus according to claim 1 or 2 , wherein the sign recognition apparatus determines that the sign is a fixed sign, and if the sign is outside the predetermined range, the sign is determined to be the temporary sign. The sign position detecting means is based on the position of the own vehicle acquired from the own vehicle position detecting means and the distance from the own vehicle to the road sign recognized by the recognizing means. The sign recognition device according to any one of claims 1 to 3, wherein

Images