US10037692B2 - Traffic sign recognition system - Google Patents

Traffic sign recognition system Download PDF

Info

Publication number
US10037692B2
US10037692B2 US15/354,065 US201615354065A US10037692B2 US 10037692 B2 US10037692 B2 US 10037692B2 US 201615354065 A US201615354065 A US 201615354065A US 10037692 B2 US10037692 B2 US 10037692B2
Authority
US
United States
Prior art keywords
traffic sign
detected
road surface
driver
surface condition
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
US15/354,065
Other versions
US20170154523A1 (en
Inventor
Takayuki Moritani
Yasutaka Otsubo
Tsuyoshi Arinaga
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Mazda Motor Corp
Original Assignee
Mazda Motor Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Mazda Motor Corp filed Critical Mazda Motor Corp
Assigned to MAZDA MOTOR CORPORATION reassignment MAZDA MOTOR CORPORATION ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: ARINAGA, TSUYOSHI, Moritani, Takayuki, Otsubo, Yasutaka
Publication of US20170154523A1 publication Critical patent/US20170154523A1/en
Application granted granted Critical
Publication of US10037692B2 publication Critical patent/US10037692B2/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/07—Controlling traffic signals
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R1/00—Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
    • B60W40/02—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to ambient conditions
    • B60W40/06—Road conditions
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
    • B60W50/08—Interaction between the driver and the control system
    • B60W50/14—Means for informing the driver, warning the driver or prompting a driver intervention
    • G06K9/00818—
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
    • G06V20/582—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads of traffic signs
    • G—PHYSICS
    • G08—SIGNALLING
    • G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B21/00—Alarms responsive to a single specified undesired or abnormal condition and not otherwise provided for
    • G08B21/02—Alarms for ensuring the safety of persons
    • G08B21/06—Alarms for ensuring the safety of persons indicating a condition of sleep, e.g. anti-dozing alarms
    • G—PHYSICS
    • G08—SIGNALLING
    • G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B25/00—Alarm systems in which the location of the alarm condition is signalled to a central station, e.g. fire or police telegraphic systems
    • G08B25/01—Alarm systems in which the location of the alarm condition is signalled to a central station, e.g. fire or police telegraphic systems characterised by the transmission medium
    • G08B25/08—Alarm systems in which the location of the alarm condition is signalled to a central station, e.g. fire or police telegraphic systems characterised by the transmission medium using communication transmission lines
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/09—Arrangements for giving variable traffic instructions
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/09—Arrangements for giving variable traffic instructions
    • G08G1/0962—Arrangements for giving variable traffic instructions having an indicator mounted inside the vehicle, e.g. giving voice messages
    • G08G1/09623—Systems involving the acquisition of information from passive traffic signs by means mounted on the vehicle

Definitions

  • the present invention relates to a traffic sign recognition system.
  • JP2014-120111A discloses an art in which a detector for detecting a line of sight of a driver driving a vehicle is provided and if a viewing direction (line of sight) of the driver is different from a direction of a traffic sign ahead of the vehicle, it is considered that the traffic sign is not viewed by the driver and caution information is displayed.
  • warning signs mainly illustrated in yellow.
  • Some of various kinds of warning signs which indicate road situations ahead of vehicles indicate a slippery road condition, existence of a sharp curve, a road intersection or branching, a narrowing road width (e.g., merging of lanes), road under construction, a high possibility of rocks falling, etc.
  • the present invention is made in view of the above issues and aims to provide a traffic sign recognition system, which is capable of enhancing the understanding of a traffic sign by a driver when a slippery road condition.
  • a traffic sign recognition system includes a display unit for displaying, for a driver of a vehicle, caution information regarding a traffic sign ahead of the vehicle, and a processor configured to execute a viewing direction detecting module for detecting a viewing direction of the driver, a traffic sign detecting module for detecting the traffic sign, and a road surface condition detecting module for detecting a slippery road surface condition.
  • the processor is configured to control the display unit to display the caution information regarding the detected traffic sign when the detected viewing direction of the driver is different from a direction of the detected traffic sign, and controls the display unit to refrain from displaying the caution information when the detected viewing direction of the driver matches the direction of the detected traffic sign.
  • the processor controls the display unit to display the caution information regardless of the viewing direction of the driver.
  • the caution information regarding the traffic sign is displayed to bring it to the attention of the driver.
  • the line of sight of the driver is not oriented toward the detected traffic sign
  • the caution information regarding the traffic sign is displayed to bring it to the attention of the driver.
  • the slippery road surface condition is detected, since the vehicle easily becomes unstable due to steering operation, acceleration, deceleration, etc., the caution information regarding the traffic sign is displayed even when the line of sight is oriented toward the traffic sign.
  • the attention of the driver can be raised more reliably.
  • the traffic sign of which the caution information is displayed regardless of the viewing direction of the driver may be a warning sign.
  • the attention of the driver to the information regarding the traffic sign of which the driver should be cautious can be raised more reliably.
  • the warning sign may indicate a road condition ahead of the vehicle.
  • the road condition ahead of the vehicle requires an attention and also the road surface is slippery, an even higher attention is desired. In such a state, it is extremely preferable in firmly reminding the driver to drive carefully by reliably bringing it to the attention of the driver.
  • the warning sign may indicate at least one of a slippery road surface, existence of a curve, a road intersection, a narrowing road width, caution for falling rocks, and road under construction.
  • the road surface condition detecting module may detect the slippery road surface condition when an outdoor air temperature detected by an outdoor air temperature sensor is below a given temperature.
  • the slippery road surface condition can easily be detected based on the outdoor air temperature.
  • the road surface condition detecting module may detect the slippery road surface condition when a road surface ⁇ detected by a road surface ⁇ sensor is below a given value.
  • the slippery road surface condition can be detected reliably based on the road surface ⁇ .
  • the road surface condition detecting module may detect the slippery road surface condition when a rainy weather is detected by a rain detector.
  • a wiper provided to the vehicle can be utilized to easily detect the slippery road surface condition.
  • FIG. 1 is a block diagram illustrating an example of a control system according to the present invention.
  • FIG. 2 is a view illustrating an example of a warning sign indicating a road condition ahead.
  • FIG. 3 is a view illustrating another example of a warning sign indicating a road condition ahead.
  • FIG. 4 is a flowchart illustrating an example of a control according to the present invention.
  • the reference character “U” is a controller (control unit) configured by using a microcomputer, including a processor 20 .
  • the controller U receives signals from various sensors or devices 1 to 6 .
  • the reference character “ 1 ” is a temperature sensor for detecting an outdoor air temperature.
  • the reference character “ 2 ” is a ⁇ -sensor for detecting a road surface friction coefficient ⁇ (referred to herein simply as “road surface ⁇ ”). Note that the road surface ⁇ may be detected by a suitable conventionally known method, such as a calculation based on a vehicle wheel rotational speed.
  • the reference character “ 3 ” is a wiper switch (for rain and snow detection).
  • the reference character “ 4 ” is an exterior vehicle camera for imaging a view forward of a vehicle concerned (hereinafter, simply referred to as “the vehicle”), and it detects existence of a traffic sign ahead of the vehicle (including traffic lights installed at an intersection), especially a warning sign indicating a road condition ahead of the vehicle.
  • the reference character “ 5 ” is a geographical data acquiring system (navigation system) for procuring information regarding the existence of traffic signs and a road condition ahead of the vehicle.
  • the reference character “ 6 ” is an in-cabin camera for detecting a line of sight of the driver of the vehicle in order to detect whether the traffic sign ahead of the vehicle is viewed.
  • the controller U controls a display unit 11 via a processor 20 of the controller U.
  • the display unit 11 displays caution information regarding the traffic sign.
  • This display unit 11 may be configured by a display provided in front of a steering wheel, a head-up display, and/or a display for navigation system.
  • the controller U includes a sign information memory 21 for storing image data of various traffic signs and meanings thereof in association with each other (creating a database).
  • FIGS. 2 and 3 are views illustrating examples of the warning sign as the traffic sign, respectively indicating a road condition ahead. Specifically, the warning sign of FIG. 2 indicates a slippery road condition ahead, and the warning sign of FIG. 3 indicates a curvy road ahead.
  • Q indicates each process.
  • the signals from the various sensors or the devices 1 to 6 are received.
  • the traffic sign stored in the sign information memory 21 is detected from view(s) imaged by the exterior vehicle camera 4 (meaning of the detected traffic sign is determined and the caution information corresponding thereto is extracted), and also a direction of the detected traffic sign (a direction toward the location thereof) is determined.
  • the in-cabin camera 6 detects the line of sight of the driver.
  • Q 4 whether the traffic sign is detected is determined. If the determination result in Q 4 is negative, the control returns to the start of the control. If the determination result in Q 4 is positive, in Q 5 , whether the line of sight of the driver (the direction detected by the in-cabin camera 6 ) is oriented toward the detected traffic sign (the direction detected by the exterior vehicle camera 4 ) is determined. If the determination result of Q 5 is negative, in Q 6 , the display unit 11 displays the caution information regarding the detected traffic sign.
  • the display contents of the caution information is, for example, letters “Caution. Slippery road ahead.” Further if the detected traffic sign is the same as the sign illustrated in FIG. 3 , the display contents of the caution information are, for example, letters “Caution. Curvy road ahead.” Note that caution information which is displayed corresponding to traffic signs is stored in the sign information memory 21 of the controller U, in association with the individual traffic signs.
  • the caution information regarding the traffic sign may be assumed to be not displayed on the display unit 11 . However, in the slippery road condition, the caution information is displayed on the display unit 11 . Specifically, if the determination result of one of Q 8 to Q 10 is positive, the road condition is considered as slippery and the control proceeds to Q 6 .
  • Q 8 whether the wiper is turned on is determined and if the wiper is determined to be turned on, the control proceeds to Q 6 (the caution information is displayed). Further in Q 9 , whether the road surface ⁇ is low, which is below a given value, is determined. If the determination result of Q 9 is positive (low ⁇ ), the control proceeds to Q 6 (the caution information is displayed). Further in Q 10 , whether the outdoor air temperature is low, which is a given temperature (e.g., 2° C.) or below (whether a possibility of the road surface to freeze is high). If the determination result of Q 10 is positive, the control proceeds to Q 6 (the caution information is displayed).
  • a given temperature e.g., 2° C.
  • the caution information regarding the detected traffic sign is not displayed.
  • the detected traffic sign indicates that the road condition ahead of the vehicle requires a high attention and also the road surface is slippery, for more reliable ensured safety
  • the caution information regarding the detected traffic sign is displayed on the display unit 11 . This will raise an attention in case the driver views the detected traffic sign but does not sufficiently understands the meaning of the traffic sign, and it is extremely preferable in ensuring safety.
  • the road surface is not slippery (e.g., dry paved road)
  • the caution information is not displayed as long as the viewing direction of the driver is toward the traffic sign, which is preferable in avoiding annoying the driver by frequently displaying the caution information.
  • the slippery road surface condition may be detected by a suitable conventionally known method, other than the method described in this embodiment.
  • the warning sign indicating a road condition ahead of the vehicle includes, in addition to the sign described in this embodiment, various other kinds of signs, for example, a sign indicating a curving direction of the road (including rightward and leftward turning directions), a sign indicating a narrowing road width (including merging of lanes), a sign indicating that the road is under construction, a sign indicating a high possibility of rocks falling, a sign indicating existence of an intersection (e.g., three-way junction or four-way junction), a sign indicating existence of a railroad crossing, a sign indicating existence of a branch section of the road, and a sign indicating a possibility of a wild animal entering the road.
  • a sign indicating a curving direction of the road including rightward and leftward turning directions
  • a sign indicating a narrowing road width including merging of lanes
  • a sign indicating that the road is under construction
  • the traffic sign indicating a road condition ahead of the vehicle may be detected based on geographical data.
  • the direction of the traffic sign may be detected by using, other than the camera, a radar.
  • Each one or a combination of two or more of the group of processes of the flowchart indicates a function of the controller U, and by adding “module” to the name/phrase indicating the function, it is possible to consider them as software components of the controller.
  • the controller U may comprise a specific traffic sign detecting module 22 executable by the processor 20 of the controller U to detect the specific traffic sign ahead of the vehicle via the exterior vehicle camera 4 ; a viewing direction detecting module 23 executable by the processor 20 to detect whether the specific traffic sign is viewed by the driver of the vehicle via the in-cabin camera 6 ; and a road surface condition detecting module 24 executable by the processor 20 to detect a road surface ⁇ via the road surface ⁇ -sensor 2 .
  • the purpose of the present invention is not limited to the explicitly described purpose, and also implicitly includes providing what is expressed as practically preferable or advantageous.
  • the present invention is preferable in safety securing by enhancing grasp of a traffic sign by a driver when a slippery road condition.

Abstract

A traffic sign recognition system includes a display for displaying caution information regarding a traffic sign ahead of a vehicle, and a processor configured to execute a viewing direction detecting module for detecting a viewing direction of a driver, a traffic sign detecting module for detecting the traffic sign, and a road surface condition detecting module for detecting a slippery road surface condition. The processor controls the display to display the caution information regarding the detected traffic sign when the detected viewing direction of the driver is different from a direction of the detected traffic sign, and controlling the display to refrain from displaying the caution information when the detected viewing direction of the driver matches the direction of the detected traffic sign. When the slippery road surface condition is detected, the processor controls the display to display the caution information regardless of the viewing direction of the driver.

Description

BACKGROUND
The present invention relates to a traffic sign recognition system.
For safe driving etc., it is undesirable to be unaware of a traffic sign installed on a road. JP2014-120111A discloses an art in which a detector for detecting a line of sight of a driver driving a vehicle is provided and if a viewing direction (line of sight) of the driver is different from a direction of a traffic sign ahead of the vehicle, it is considered that the traffic sign is not viewed by the driver and caution information is displayed.
Among a variety of types of traffic signs, there are warning signs mainly illustrated in yellow. Some of various kinds of warning signs which indicate road situations ahead of vehicles indicate a slippery road condition, existence of a sharp curve, a road intersection or branching, a narrowing road width (e.g., merging of lanes), road under construction, a high possibility of rocks falling, etc.
With the art described in JP2014-120111A, when the viewing direction of the driver is toward the traffic sign, the traffic sign is considered as viewed and caution information regarding the traffic sign is not displayed. However, even if the viewing direction of the driver is toward the traffic sign, it does not mean that the driver sufficiently understands the contents of the traffic sign. If the contents of the traffic sign are for bringing high attention to a road condition ahead of the vehicle, it is desirable to lower a vehicle speed or refrain from sharp acceleration. Especially if a road surface condition is slippery, since a behavior of the vehicle may become unstable due to acceleration or change in steering, it is desirable to enhance the understanding of the traffic sign by the driver.
SUMMARY
The present invention is made in view of the above issues and aims to provide a traffic sign recognition system, which is capable of enhancing the understanding of a traffic sign by a driver when a slippery road condition.
According to one aspect of the present invention, a traffic sign recognition system is provided. The system includes a display unit for displaying, for a driver of a vehicle, caution information regarding a traffic sign ahead of the vehicle, and a processor configured to execute a viewing direction detecting module for detecting a viewing direction of the driver, a traffic sign detecting module for detecting the traffic sign, and a road surface condition detecting module for detecting a slippery road surface condition. The processor is configured to control the display unit to display the caution information regarding the detected traffic sign when the detected viewing direction of the driver is different from a direction of the detected traffic sign, and controls the display unit to refrain from displaying the caution information when the detected viewing direction of the driver matches the direction of the detected traffic sign. When the slippery road surface condition is detected, the processor controls the display unit to display the caution information regardless of the viewing direction of the driver.
According to the above configuration, when the line of sight of the driver (the viewing direction) is not oriented toward the detected traffic sign, the caution information regarding the traffic sign is displayed to bring it to the attention of the driver. Whereas, when the line of sight is oriented toward the traffic sign, it is considered that the traffic sign is viewed by the driver and the caution information is not displayed unnecessarily. Additionally, when the slippery road surface condition is detected, since the vehicle easily becomes unstable due to steering operation, acceleration, deceleration, etc., the caution information regarding the traffic sign is displayed even when the line of sight is oriented toward the traffic sign. Thus, the attention of the driver can be raised more reliably.
The traffic sign of which the caution information is displayed regardless of the viewing direction of the driver may be a warning sign. Thus, the attention of the driver to the information regarding the traffic sign of which the driver should be cautious can be raised more reliably.
The warning sign may indicate a road condition ahead of the vehicle. When the road condition ahead of the vehicle requires an attention and also the road surface is slippery, an even higher attention is desired. In such a state, it is extremely preferable in firmly reminding the driver to drive carefully by reliably bringing it to the attention of the driver.
The warning sign may indicate at least one of a slippery road surface, existence of a curve, a road intersection, a narrowing road width, caution for falling rocks, and road under construction. Thus, the attention of the driver to the information regarding traffic signs indicating a condition which requires a high attention to travel can be raised more reliably.
The road surface condition detecting module may detect the slippery road surface condition when an outdoor air temperature detected by an outdoor air temperature sensor is below a given temperature. Thus, the slippery road surface condition can easily be detected based on the outdoor air temperature.
The road surface condition detecting module may detect the slippery road surface condition when a road surface μ detected by a road surface μ sensor is below a given value. Thus, the slippery road surface condition can be detected reliably based on the road surface μ.
The road surface condition detecting module may detect the slippery road surface condition when a rainy weather is detected by a rain detector. Here, a wiper provided to the vehicle can be utilized to easily detect the slippery road surface condition.
BRIEF DESCRIPTION OF THE DRAWINGS
FIG. 1 is a block diagram illustrating an example of a control system according to the present invention.
FIG. 2 is a view illustrating an example of a warning sign indicating a road condition ahead.
FIG. 3 is a view illustrating another example of a warning sign indicating a road condition ahead.
FIG. 4 is a flowchart illustrating an example of a control according to the present invention.
 DETAILED DESCRIPTION OF EMBODIMENT
In FIG. 1, the reference character “U” is a controller (control unit) configured by using a microcomputer, including a processor 20. The controller U receives signals from various sensors or devices 1 to 6. Specifically, the reference character “1” is a temperature sensor for detecting an outdoor air temperature. The reference character “2” is a μ-sensor for detecting a road surface friction coefficient μ (referred to herein simply as “road surface μ”). Note that the road surface μ may be detected by a suitable conventionally known method, such as a calculation based on a vehicle wheel rotational speed. The reference character “3” is a wiper switch (for rain and snow detection). The reference character “4” is an exterior vehicle camera for imaging a view forward of a vehicle concerned (hereinafter, simply referred to as “the vehicle”), and it detects existence of a traffic sign ahead of the vehicle (including traffic lights installed at an intersection), especially a warning sign indicating a road condition ahead of the vehicle. The reference character “5” is a geographical data acquiring system (navigation system) for procuring information regarding the existence of traffic signs and a road condition ahead of the vehicle. The reference character “6” is an in-cabin camera for detecting a line of sight of the driver of the vehicle in order to detect whether the traffic sign ahead of the vehicle is viewed.
The controller U controls a display unit 11 via a processor 20 of the controller U. The display unit 11 displays caution information regarding the traffic sign. This display unit 11 may be configured by a display provided in front of a steering wheel, a head-up display, and/or a display for navigation system. The controller U includes a sign information memory 21 for storing image data of various traffic signs and meanings thereof in association with each other (creating a database).
FIGS. 2 and 3 are views illustrating examples of the warning sign as the traffic sign, respectively indicating a road condition ahead. Specifically, the warning sign of FIG. 2 indicates a slippery road condition ahead, and the warning sign of FIG. 3 indicates a curvy road ahead.
Next, an example of the control according to the present invention is described with reference to FIG. 4. Note that in the following description, “Q” indicates each process. First in Q1, the signals from the various sensors or the devices 1 to 6 are received. In Q2, the traffic sign stored in the sign information memory 21 is detected from view(s) imaged by the exterior vehicle camera 4 (meaning of the detected traffic sign is determined and the caution information corresponding thereto is extracted), and also a direction of the detected traffic sign (a direction toward the location thereof) is determined. Then in Q3, the in-cabin camera 6 detects the line of sight of the driver.
Then in Q4, whether the traffic sign is detected is determined. If the determination result in Q4 is negative, the control returns to the start of the control. If the determination result in Q4 is positive, in Q5, whether the line of sight of the driver (the direction detected by the in-cabin camera 6) is oriented toward the detected traffic sign (the direction detected by the exterior vehicle camera 4) is determined. If the determination result of Q5 is negative, in Q6, the display unit 11 displays the caution information regarding the detected traffic sign.
In Q6, if the detected traffic sign is the same as the sign illustrated in FIG. 2, the display contents of the caution information is, for example, letters “Caution. Slippery road ahead.” Further if the detected traffic sign is the same as the sign illustrated in FIG. 3, the display contents of the caution information are, for example, letters “Caution. Curvy road ahead.” Note that caution information which is displayed corresponding to traffic signs is stored in the sign information memory 21 of the controller U, in association with the individual traffic signs.
If the determination result in Q5 is positive, in Q7, whether the detected traffic sign is a warning sign indicating a road condition ahead of the vehicle, such as one of the signs illustrated in FIGS. 2 and 3, is determined. If the determination result in Q7 is negative, the control proceeds to Q11 where the caution information regarding the detected traffic sign is not displayed on the display unit 11.
If the determination result of Q7 is positive, the caution information regarding the traffic sign may be assumed to be not displayed on the display unit 11. However, in the slippery road condition, the caution information is displayed on the display unit 11. Specifically, if the determination result of one of Q8 to Q10 is positive, the road condition is considered as slippery and the control proceeds to Q6.
In Q8, whether the wiper is turned on is determined and if the wiper is determined to be turned on, the control proceeds to Q6 (the caution information is displayed). Further in Q9, whether the road surface μ is low, which is below a given value, is determined. If the determination result of Q9 is positive (low μ), the control proceeds to Q6 (the caution information is displayed). Further in Q10, whether the outdoor air temperature is low, which is a given temperature (e.g., 2° C.) or below (whether a possibility of the road surface to freeze is high). If the determination result of Q10 is positive, the control proceeds to Q6 (the caution information is displayed).
If the determination result of Q10 is negative, the road surface is considered as not slippery, and in Q11, the caution information regarding the detected traffic sign is not displayed.
As described above, when the line of sight of the driver is oriented toward the detected traffic sign, the traffic sign is considered as viewed, the caution information regarding the detected traffic sign is not displayed. However, when the detected traffic sign indicates that the road condition ahead of the vehicle requires a high attention and also the road surface is slippery, for more reliable ensured safety, the caution information regarding the detected traffic sign is displayed on the display unit 11. This will raise an attention in case the driver views the detected traffic sign but does not sufficiently understands the meaning of the traffic sign, and it is extremely preferable in ensuring safety. Obviously if the road surface is not slippery (e.g., dry paved road), the caution information is not displayed as long as the viewing direction of the driver is toward the traffic sign, which is preferable in avoiding annoying the driver by frequently displaying the caution information.
Although the embodiment is described above, the present invention is not limited to this, and may suitably be modified within the scope of the claims. The slippery road surface condition may be detected by a suitable conventionally known method, other than the method described in this embodiment. The warning sign indicating a road condition ahead of the vehicle includes, in addition to the sign described in this embodiment, various other kinds of signs, for example, a sign indicating a curving direction of the road (including rightward and leftward turning directions), a sign indicating a narrowing road width (including merging of lanes), a sign indicating that the road is under construction, a sign indicating a high possibility of rocks falling, a sign indicating existence of an intersection (e.g., three-way junction or four-way junction), a sign indicating existence of a railroad crossing, a sign indicating existence of a branch section of the road, and a sign indicating a possibility of a wild animal entering the road. The traffic sign indicating a road condition ahead of the vehicle (warning sign) may be detected based on geographical data. The direction of the traffic sign may be detected by using, other than the camera, a radar. Each one or a combination of two or more of the group of processes of the flowchart indicates a function of the controller U, and by adding “module” to the name/phrase indicating the function, it is possible to consider them as software components of the controller. For example, the controller U may comprise a specific traffic sign detecting module 22 executable by the processor 20 of the controller U to detect the specific traffic sign ahead of the vehicle via the exterior vehicle camera 4; a viewing direction detecting module 23 executable by the processor 20 to detect whether the specific traffic sign is viewed by the driver of the vehicle via the in-cabin camera 6; and a road surface condition detecting module 24 executable by the processor 20 to detect a road surface μ via the road surface μ-sensor 2. The purpose of the present invention is not limited to the explicitly described purpose, and also implicitly includes providing what is expressed as practically preferable or advantageous.
The present invention is preferable in safety securing by enhancing grasp of a traffic sign by a driver when a slippery road condition.
It should be understood that the embodiments herein are illustrative and not restrictive, since the scope of the invention is defined by the appended claims rather than by the description preceding them, and all changes that fall within metes and bounds of the claims, or equivalence of such metes and bounds thereof, are therefore intended to be embraced by the claims.
LIST OF REFERENCE CHARACTERS
  • U Controller
  • 1 Sensor (For Outdoor Air Temperature Detection)
  • 2 Sensor (For Road Surface μ Detection)
  • 3 Wiper Switch (For Rain and Snow Detection)
  • 4 Exterior Vehicle Camera (For Traffic Sign Detection)
  • 5 Geographical Data Acquiring System (Navigation System)
  • 6 In-cabin Camera (For Driver's Line of Sight Detection)
  • 11 Display Unit
  • 21 Sign Information Memory

Claims (7)

What is claimed is:
1. A traffic sign recognition system, comprising:
a display unit for displaying, for a driver of a vehicle, caution information regarding a traffic sign ahead of the vehicle; and
a processor configured to execute:
a viewing direction detecting module for detecting a viewing direction of the driver;
a traffic sign detecting module for detecting the traffic sign;
a road surface condition detecting module for detecting a slippery road surface condition; wherein
the processor is configured to control the display unit to display the caution information regarding the detected traffic sign when the detected viewing direction of the driver is different from a direction of the detected traffic sign, and control the display unit to refrain from displaying the caution information when the detected viewing direction of the driver matches with the direction of the detected traffic sign,
wherein when the slippery road surface condition is detected, the processor controls the display unit to display the caution information even when the viewing direction of the driver matches with the direction of the detected traffic sign.
2. The system of claim 1, wherein the traffic sign of which the caution information is displayed regardless of the viewing direction of the driver is a warning sign.
3. The system of claim 2, wherein the warning sign indicates a road condition ahead of the vehicle.
4. The system of claim 3, wherein the warning sign indicates at least one of a slippery road surface, existence of a curve, a road intersection, a narrowing road width, caution for falling rocks, and road under construction.
5. The system of claim 1, wherein the road surface condition detecting module detects the slippery road surface condition when an outdoor air temperature detected by an outdoor air temperature detector is below a given temperature.
6. The system of claim 1, wherein the road surface condition detecting module detects the slippery road surface condition when a road surface μ detected by a road surface μ detector is below a given value.
7. The system of claim 1, wherein the road surface condition detecting module detects the slippery road surface condition when rainy weather is detected by a rain detector.
US15/354,065 2015-11-26 2016-11-17 Traffic sign recognition system Active US10037692B2 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2015-230222 2015-11-26
JP2015230222A JP6418139B2 (en) 2015-11-26 2015-11-26 Sign recognition system

Publications (2)

Publication Number Publication Date
US20170154523A1 US20170154523A1 (en) 2017-06-01
US10037692B2 true US10037692B2 (en) 2018-07-31

Family

ID=58693282

Family Applications (1)

Application Number Title Priority Date Filing Date
US15/354,065 Active US10037692B2 (en) 2015-11-26 2016-11-17 Traffic sign recognition system

Country Status (4)

Country Link
US (1) US10037692B2 (en)
JP (1) JP6418139B2 (en)
CN (1) CN107054221B (en)
DE (1) DE102016013810A1 (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110211403A (en) * 2019-06-19 2019-09-06 广州市高速公路有限公司营运分公司 A kind of road information sharing method and system
CN110310483A (en) * 2019-07-01 2019-10-08 淮阴师范学院 A kind of LED light source signal machine testing warning device and detection method

Families Citing this family (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9975480B2 (en) 2016-04-12 2018-05-22 Denso International America, Inc. Methods and systems for blind spot monitoring with adaptive alert zone
US9931981B2 (en) 2016-04-12 2018-04-03 Denso International America, Inc. Methods and systems for blind spot monitoring with rotatable blind spot sensor
US9947226B2 (en) 2016-04-12 2018-04-17 Denso International America, Inc. Methods and systems for blind spot monitoring with dynamic detection range
US9994151B2 (en) * 2016-04-12 2018-06-12 Denso International America, Inc. Methods and systems for blind spot monitoring with adaptive alert zone
DE102017204256A1 (en) * 2017-03-14 2018-09-20 Bayerische Motoren Werke Aktiengesellschaft Method and device for reminding a driver to start on a light-emitting device with variable output function
CN108583425B (en) * 2018-05-02 2023-01-24 宁波大红鹰学院 Control method for automatically moving warning board
EP3690404B1 (en) * 2019-01-30 2023-02-01 Zenuity AB Method and system for indicating traffic information in a vehicle
CN110154893B (en) * 2019-05-29 2020-12-11 辽宁工业大学 Automobile safe driving early warning method based on driver characteristics
CN110154894B (en) * 2019-05-29 2020-12-11 辽宁工业大学 Automobile safe driving early warning method based on road surface condition
DE102020122977A1 (en) 2020-09-02 2022-03-03 Bayerische Motoren Werke Aktiengesellschaft Driver assistance system and driver assistance method for a vehicle
CN112669631B (en) * 2020-12-15 2022-08-05 重庆电子工程职业学院 Intelligent traffic monitoring and alarming system
CN114093135A (en) * 2021-11-26 2022-02-25 中国电建集团成都勘测设计研究院有限公司 Dangerous rock mass group instability early warning system

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120154591A1 (en) * 2009-09-01 2012-06-21 Magna Mirrors Of America, Inc. Imaging and display system for vehicle
JP2014120111A (en) 2012-12-19 2014-06-30 Aisin Aw Co Ltd Travel support system, travel support method, and computer program
US20140200759A1 (en) * 2011-01-26 2014-07-17 Magna Electronics Inc. Rear vision system with trailer angle detection
US20160180182A1 (en) * 2014-12-18 2016-06-23 Magna Electronics Inc. Vehicle vision system with 3d registration for distance estimation
US20170096144A1 (en) * 2015-10-05 2017-04-06 Ford Global Technologies, Llc System and Method for Inspecting Road Surfaces

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP3226153B2 (en) * 1996-03-18 2001-11-05 シャープ株式会社 Multimedia data display device
JP4941234B2 (en) * 2007-10-31 2012-05-30 株式会社エクォス・リサーチ Road sign display device
JP2015052548A (en) * 2013-09-09 2015-03-19 富士重工業株式会社 Vehicle exterior environment recognition device
JP6323074B2 (en) * 2014-03-06 2018-05-16 株式会社デンソー Display object information display device

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120154591A1 (en) * 2009-09-01 2012-06-21 Magna Mirrors Of America, Inc. Imaging and display system for vehicle
US20140200759A1 (en) * 2011-01-26 2014-07-17 Magna Electronics Inc. Rear vision system with trailer angle detection
JP2014120111A (en) 2012-12-19 2014-06-30 Aisin Aw Co Ltd Travel support system, travel support method, and computer program
US20160180182A1 (en) * 2014-12-18 2016-06-23 Magna Electronics Inc. Vehicle vision system with 3d registration for distance estimation
US20170096144A1 (en) * 2015-10-05 2017-04-06 Ford Global Technologies, Llc System and Method for Inspecting Road Surfaces

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110211403A (en) * 2019-06-19 2019-09-06 广州市高速公路有限公司营运分公司 A kind of road information sharing method and system
CN110310483A (en) * 2019-07-01 2019-10-08 淮阴师范学院 A kind of LED light source signal machine testing warning device and detection method

Also Published As

Publication number Publication date
US20170154523A1 (en) 2017-06-01
CN107054221A (en) 2017-08-18
CN107054221B (en) 2019-08-30
DE102016013810A1 (en) 2017-06-01
JP2017097683A (en) 2017-06-01
JP6418139B2 (en) 2018-11-07

Similar Documents

Publication Publication Date Title
US10037692B2 (en) Traffic sign recognition system
US9218535B2 (en) Arrangement and method for recognizing road signs
KR102610743B1 (en) Apparatus for controlling lane change of vehicle, system having the same and method thereof
CN108263382B (en) Cooperative adaptive cruise control system based on driving pattern of target vehicle
US10915100B2 (en) Control system for vehicle
US11328606B2 (en) Hazardous vehicle prediction device, hazardous vehicle warning system, and hazardous vehicle prediction method
US9865165B2 (en) Traffic sign recognition system
US9798323B2 (en) Crowd-sourced transfer-of-control policy for automated vehicles
US9767686B2 (en) Method and control and detection device for determining the plausibility of a wrong-way travel of a motor vehicle
US11097730B2 (en) Implicit activation and control of driver assistance systems
US20130057397A1 (en) Method of operating a vehicle safety system
US9315191B2 (en) Driving assistance device
US9786172B2 (en) Warning guidance system, method, and program that provide information to vehicle navigation systems
US9824284B2 (en) Traffic sign recognition system
US20150232026A1 (en) Method for Assisting a Driver at a Traffic Light for a Vehicle
US20170154227A1 (en) Traffic sign recognition system
EP3207494B1 (en) Systems and methods for traffic sign validation
US20200406753A1 (en) Display control device, display device, and display control method
US20180068559A1 (en) Method and system for enabling a vehicle to discard non-pertinent road sign information
CN113272877B (en) Control system for vehicle
JP2019061352A (en) Vehicle control device
JPWO2019065564A1 (en) Automatic operation control device and method
US9950711B2 (en) Device for a vehicle
JP6625480B2 (en) Display system
WO2022057721A1 (en) Method for operating vehicle hazardous parking warning system

Legal Events

Date Code Title Description
AS Assignment

Owner name: MAZDA MOTOR CORPORATION, JAPAN

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:MORITANI, TAKAYUKI;OTSUBO, YASUTAKA;ARINAGA, TSUYOSHI;SIGNING DATES FROM 20161101 TO 20161107;REEL/FRAME:040356/0298

STCF Information on status: patent grant

Free format text: PATENTED CASE

MAFP Maintenance fee payment

Free format text: PAYMENT OF MAINTENANCE FEE, 4TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1551); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

Year of fee payment: 4