US10800455B2 - Vehicle turn signal detection - Google Patents

Vehicle turn signal detection Download PDF

Info

Publication number
US10800455B2
US10800455B2 US14/973,454 US201514973454A US10800455B2 US 10800455 B2 US10800455 B2 US 10800455B2 US 201514973454 A US201514973454 A US 201514973454A US 10800455 B2 US10800455 B2 US 10800455B2
Authority
US
United States
Prior art keywords
vehicle
turn signal
signal indicator
image
lane
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
US14/973,454
Other languages
English (en)
Other versions
US20170174261A1 (en
Inventor
Ashley Elizabeth Micks
Harpreetsingh Banvait
Jinesh J Jain
Brielle Reiff
Sneha Kadetotad
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Ford Global Technologies LLC
Original Assignee
Ford Global Technologies LLC
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ford Global Technologies LLC filed Critical Ford Global Technologies LLC
Assigned to FORD GLOBAL TECHNOLOGIES, LLC reassignment FORD GLOBAL TECHNOLOGIES, LLC ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: BANVAIT, HARPREETSINGH, JAIN, JINESH J, KADETOTAD, SNEHA, Reiff, Brielle, MICKS, ASHLEY ELIZABETH
Priority to US14/973,454 priority Critical patent/US10800455B2/en
Priority to RU2016147944A priority patent/RU2016147944A/ru
Priority to DE102016123878.3A priority patent/DE102016123878A1/de
Priority to CN201611138148.7A priority patent/CN106891888B/zh
Priority to GB1621096.5A priority patent/GB2547082A/en
Priority to MX2016016811A priority patent/MX2016016811A/es
Publication of US20170174261A1 publication Critical patent/US20170174261A1/en
Publication of US10800455B2 publication Critical patent/US10800455B2/en
Application granted granted Critical
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B62—LAND VEHICLES FOR TRAVELLING OTHERWISE THAN ON RAILS
    • B62D—MOTOR VEHICLES; TRAILERS
    • B62D15/00—Steering not otherwise provided for
    • B62D15/02—Steering position indicators ; Steering position determination; Steering aids
    • B62D15/025—Active steering aids, e.g. helping the driver by actively influencing the steering system after environment evaluation
    • B62D15/0265—Automatic obstacle avoidance by steering
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W30/00—Purposes of road vehicle drive control systems not related to the control of a particular sub-unit, e.g. of systems using conjoint control of vehicle sub-units
    • B60W30/08—Active safety systems predicting or avoiding probable or impending collision or attempting to minimise its consequences
    • B60W30/095—Predicting travel path or likelihood of collision
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W30/00—Purposes of road vehicle drive control systems not related to the control of a particular sub-unit, e.g. of systems using conjoint control of vehicle sub-units
    • B60W30/08—Active safety systems predicting or avoiding probable or impending collision or attempting to minimise its consequences
    • B60W30/09—Taking automatic action to avoid collision, e.g. braking and steering
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B62—LAND VEHICLES FOR TRAVELLING OTHERWISE THAN ON RAILS
    • B62D—MOTOR VEHICLES; TRAILERS
    • B62D15/00—Steering not otherwise provided for
    • B62D15/02—Steering position indicators ; Steering position determination; Steering aids
    • B62D15/029—Steering assistants using warnings or proposing actions to the driver without influencing the steering system
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60Q—ARRANGEMENT OF SIGNALLING OR LIGHTING DEVICES, THE MOUNTING OR SUPPORTING THEREOF OR CIRCUITS THEREFOR, FOR VEHICLES IN GENERAL
    • B60Q9/00—Arrangement or adaptation of signal devices not provided for in one of main groups B60Q1/00 - B60Q7/00, e.g. haptic signalling
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R1/00—Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W30/00—Purposes of road vehicle drive control systems not related to the control of a particular sub-unit, e.g. of systems using conjoint control of vehicle sub-units
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W30/00—Purposes of road vehicle drive control systems not related to the control of a particular sub-unit, e.g. of systems using conjoint control of vehicle sub-units
    • B60W30/08—Active safety systems predicting or avoiding probable or impending collision or attempting to minimise its consequences
    • B60W30/095—Predicting travel path or likelihood of collision
    • B60W30/0956—Predicting travel path or likelihood of collision the prediction being responsive to traffic or environmental parameters
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
    • B60W50/0097—Predicting future conditions
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
    • B60W50/0098—Details of control systems ensuring comfort, safety or stability not otherwise provided for
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
    • B60W50/08—Interaction between the driver and the control system
    • B60W50/14—Means for informing the driver, warning the driver or prompting a driver intervention
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W60/00—Drive control systems specially adapted for autonomous road vehicles
    • B60W60/001—Planning or execution of driving tasks
    • B60W60/0027—Planning or execution of driving tasks using trajectory prediction for other traffic participants
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B62—LAND VEHICLES FOR TRAVELLING OTHERWISE THAN ON RAILS
    • B62D—MOTOR VEHICLES; TRAILERS
    • B62D15/00—Steering not otherwise provided for
    • B62D15/02—Steering position indicators ; Steering position determination; Steering aids
    • B62D15/025—Active steering aids, e.g. helping the driver by actively influencing the steering system after environment evaluation
    • B62D15/0255—Automatic changing of lane, e.g. for passing another vehicle
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B62—LAND VEHICLES FOR TRAVELLING OTHERWISE THAN ON RAILS
    • B62D—MOTOR VEHICLES; TRAILERS
    • B62D6/00—Arrangements for automatically controlling steering depending on driving conditions sensed and responded to, e.g. control circuits
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02—Control of position or course in two dimensions
    • G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0276—Control of position or course in two dimensions specially adapted to land vehicles using signals provided by a source external to the vehicle
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
    • G06V20/584—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads of vehicle lights or traffic lights
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/01—Detecting movement of traffic to be counted or controlled
    • G08G1/017—Detecting movement of traffic to be counted or controlled identifying vehicles
    • G08G1/0175—Detecting movement of traffic to be counted or controlled identifying vehicles by photographing vehicles, e.g. when violating traffic rules
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/01—Detecting movement of traffic to be counted or controlled
    • G08G1/04—Detecting movement of traffic to be counted or controlled using optical or ultrasonic detectors
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/16—Anti-collision systems
    • G08G1/166—Anti-collision systems for active traffic, e.g. moving vehicles, pedestrians, bikes
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
    • B60W2050/0001—Details of the control system
    • B60W2050/0043—Signal treatments, identification of variables or parameters, parameter estimation or state estimation
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
    • B60W50/08—Interaction between the driver and the control system
    • B60W50/14—Means for informing the driver, warning the driver or prompting a driver intervention
    • B60W2050/143—Alarm means
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
    • B60W50/08—Interaction between the driver and the control system
    • B60W50/14—Means for informing the driver, warning the driver or prompting a driver intervention
    • B60W2050/146—Display means
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2420/00—Indexing codes relating to the type of sensors based on the principle of their operation
    • B60W2420/40—Photo, light or radio wave sensitive means, e.g. infrared sensors
    • B60W2420/403—Image sensing, e.g. optical camera
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2554/00—Input parameters relating to objects
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2554/00—Input parameters relating to objects
    • B60W2554/80—Spatial relation or speed relative to objects

Definitions

  • the disclosure relates generally to methods, systems, and apparatuses for automated driving or for assisting a driver, and more particularly relates to methods, systems, and apparatuses for detecting a vehicle's turn signal status for collision avoidance during lane-switching maneuvers or otherwise.
  • Automobiles provide a significant portion of transportation for commercial, government, and private entities.
  • Autonomous vehicles and driving assistance systems are currently being developed and deployed to provide safety, reduce an amount of user input required, or even eliminate user involvement entirely.
  • some driving assistance systems such as crash avoidance systems, may monitor driving, positions, and a velocity of the vehicle and other objects while a human is driving. When the system detects that a crash or impact is imminent the crash avoidance system may intervene and apply a brake, steer the vehicle, or perform other avoidance or safety maneuvers.
  • autonomous vehicles may drive and navigate a vehicle with little or no user input.
  • FIG. 1 is a schematic block diagram illustrating an implementation of a vehicle control system that includes an automated driving/assistance system
  • FIG. 2 is a schematic diagram illustrating a top view of vehicles on a multi-lane road
  • FIG. 3 illustrates a perspective view of a vehicle on a roadway
  • FIG. 4 illustrates a perspective view of a vehicle on a roadway
  • FIG. 5 is a schematic block diagram illustrating driving maneuver decision making, according to one implementation
  • FIG. 6 is a schematic block diagram illustrating example components of a driver intent component, according to one implementation.
  • FIG. 7 is a schematic block diagram illustrating a method for driver decision making, according to one implementation.
  • Applicants have recognized that, at least during any transition period in which both autonomous and manually-driven vehicles will be on the road, it is important that autonomous vehicles accurately predict the actions of human drivers. Detection of a turn signal of a vehicle used by another driver adds certainty to the future actions of that driver in maneuvering the vehicle. This can be useful in the case of highway lane-switching in which two vehicles, such as an autonomous vehicle and a non-autonomous vehicle, may attempt to merge into the same lane, or change lanes near each other. By detecting the turn signal of the other vehicle, the autonomous vehicle can detect if the other vehicle is attempting to enter its intended path and therefore avoid a possible collision. This detection could also be beneficial for a driver assisted vehicle that is performing a lane change or otherwise. Upon recognizing its driver's intent to change lanes (e.g., based on activation of a turn signal on an assisted vehicle), the assisted vehicle could notify its driver if another vehicle is predicted to move into the same lane.
  • the assisted vehicle Upon recognizing its driver'
  • a system uses the status of an external vehicle's turn signals in order to predict the motion of that vehicle for decision-making in highway lane-change maneuvers.
  • the assisted vehicle can be equipped with 360 degree sensing system including, but not limited to, a camera, LIDAR, radar, and/or other range-finding or imaging sensors.
  • Computer vision and sensor fusion algorithms employing deep neural networks may be trained to recognize the turn signal regions of interest on the vehicle.
  • the neural networks may also be used to identify whether the turn signal is on, possibly through comparison of the vehicle's visible turn signals, which may not be active all at the same time.
  • the system may be configured to perform the following: locate nearby vehicles that are moving in adjacent lanes; locate and determine bounding boxes for visible turn signals on adjacent vehicles; send image data within turn signal bounding box to computer vision algorithm to recognize which signals are in use; and input turn signal status into lane-changing decision matrix so that the system, or another system, can make a lane change or driving maneuver decision. Even under conditions where another vehicle has the turn signal indicators on by mistake, or a driver has just forgotten to turn them off, this information may be useful as an indication to pay extra attention to that vehicle and track it closely for potential simultaneous merges until the turn signal indicators are switched off or the other vehicle is out of zone of possible risk.
  • autonomous vehicle may be a vehicle that acts or operates completely independent of a human driver; or may be a vehicle that acts or operates independent of a human driver in some instances while in other instances a human driver may be able to operate the vehicle; or may be a vehicle that is predominantly operated by a human driver, but with the assistance of an automated driving/assistance system.
  • Implementations of the systems, devices, and methods disclosed herein may comprise or utilize a special purpose or general-purpose computer including computer hardware, such as, for example, one or more processors and system memory, as discussed in greater detail below. Implementations within the scope of the present disclosure may also include physical and other computer-readable media for carrying or storing computer-executable instructions and/or data structures. Such computer-readable media can be any available media that can be accessed by a general purpose or special purpose computer system. Computer-readable media that store computer-executable instructions are computer storage media (devices). Computer-readable media that carry computer-executable instructions are transmission media. Thus, by way of example, and not limitation, implementations of the disclosure can comprise at least two distinctly different kinds of computer-readable media: computer storage media (devices) and transmission media.
  • Computer storage media includes RAM, ROM, EEPROM, CD-ROM, solid state drives (“SSDs”) (e.g., based on RAM), Flash memory, phase-change memory (“PCM”), other types of memory, other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store desired program code means in the form of computer-executable instructions or data structures and which can be accessed by a general purpose or special purpose computer.
  • SSDs solid state drives
  • PCM phase-change memory
  • An implementation of the devices, systems, and methods disclosed herein may communicate over a computer network.
  • a “network” is defined as one or more data links that enable the transport of electronic data between computer systems and/or modules and/or other electronic devices.
  • Transmissions media can include a network and/or data links, which can be used to carry desired program code means in the form of computer-executable instructions or data structures and which can be accessed by a general purpose or special purpose computer. Combinations of the above should also be included within the scope of computer-readable media.
  • Computer-executable instructions comprise, for example, instructions and data which, when executed at a processor, cause a general purpose computer, special purpose computer, or special purpose processing device to perform a certain function or group of functions.
  • the computer executable instructions may be, for example, binaries, intermediate format instructions such as assembly language, or even source code.
  • the disclosure may be practiced in network computing environments with many types of computer system configurations, including, an in-dash computer, personal computers, desktop computers, laptop computers, message processors, hand-held devices, multi-processor systems, microprocessor-based or programmable consumer electronics, network PCs, minicomputers, mainframe computers, mobile telephones, PDAs, tablets, pagers, routers, switches, various storage devices, and the like.
  • the disclosure may also be practiced in distributed system environments where local and remote computer systems, which are linked (either by hardwired data links, wireless data links, or by a combination of hardwired and wireless data links) through a network, both perform tasks.
  • program modules may be located in both local and remote memory storage devices.
  • ASICs application specific integrated circuits
  • FIG. 1 illustrates a vehicle control system 100 that includes an automated driving/assistance system 102 .
  • the automated driving/assistance system 102 may be used to automate, assist, or control operation of a vehicle, such as a car, truck, van, bus, large truck, emergency vehicles or any other automobile for transporting people or goods, or to provide assistance to a human driver.
  • the automated driving/assistance system 102 may control one or more of braking, steering, acceleration, lights, alerts, driver notifications, radio, or any other auxiliary systems of the vehicle.
  • the automated driving/assistance system 102 may not be able to provide any control of the driving (e.g., steering, acceleration, or braking), but may provide notifications and alerts to assist a human driver in driving safely.
  • the automated driving/assistance system 102 includes a driver intent component 104 , which may predict a future movement of other vehicles based on one or more of turn signal indicators and vehicle movement.
  • the driver intent component 104 may estimate an intention of the driver of a different vehicle (e.g., a vehicle that does not include the vehicle control system 100 ) based on a turn signal state of the different vehicle.
  • the vehicle control system 100 also includes one or more sensor systems/devices for detecting a presence of nearby objects or determining a location of a parent vehicle (e.g., a vehicle that includes the vehicle control system 100 ) or nearby objects.
  • the vehicle control system 100 may include one or more radar systems 106 , one or more LIDAR systems 108 , one or more camera systems 110 , a global positioning system (GPS) 112 , and/or one or more ultrasound systems 114 .
  • the vehicle control system 100 may include a data store 116 for storing relevant or useful data for navigation and safety such as map data, driving history or other data.
  • the vehicle control system 100 may also include a transceiver 118 for wireless communication with a mobile or wireless network, other vehicles, infrastructure, or any other communication system.
  • the vehicle control system 100 may include vehicle control actuators 120 to control various aspects of the driving of the vehicle such as electric motors, switches or other actuators, to control braking, acceleration, steering or the like.
  • the vehicle control system 100 may also include one or more displays 122 , speakers 124 , or other devices so that notifications to a human driver or passenger may be provided.
  • the display 122 may include a heads-up display, a dashboard display or indicator, a display screen, or any other visual indicator, which may be seen by a driver or passenger of a vehicle.
  • the speakers 124 may include one or more speakers of a sound system of a vehicle or may include a speaker dedicated to driver notification.
  • FIG. 1 is given by way of example only. Other embodiments may include fewer or additional components without departing from the scope of the disclosure. Additionally, illustrated components may be combined or included within other components without limitation.
  • the driver intent component 104 may be separate from the automated driving/assistance system 102 and the data store 116 may be included as part of the automated driving/assistance system 102 and/or part of the driver intent component 104 .
  • the radar system 106 may include any radar system known in the art.
  • a radar system 106 operates by transmitting radio signals and detecting reflections off objects.
  • the radar may be used to detect physical objects, such as other vehicles, parking barriers or parking chocks, landscapes (such as trees, cliffs, rocks, hills, or the like), road edges, signs, buildings, or other objects.
  • the radar system 106 may use the reflected radio waves to determine a size, shape, distance, surface texture, or other information about a physical object or material. For example, the radar system 106 may sweep an area to obtain data about objects within a specific range and viewing angle of the radar system 106 .
  • the radar system 106 is configured to generate perception information from a region near the vehicle, such as one or more regions nearby or surrounding the vehicle. For example, the radar system 106 may obtain data about regions of the ground or vertical area immediately neighboring or near the vehicle.
  • the radar system 106 may include one of many widely available commercially available radar systems.
  • the radar system 106 may provide perception data including a two dimensional or three-dimensional map or model to the automated driving/assistance system 102 for reference or processing.
  • the LIDAR system 108 may include any LIDAR system in the art.
  • the LIDAR system 108 operates by emitting visible wavelength or infrared wavelength lasers and detecting reflections of the laser light off objects.
  • the lasers may be used to detect physical objects, such as other vehicles, parking barriers or parking chocks, landscapes (such as trees, cliffs, rocks, hills, or the like), road edges, signs, buildings, or other objects.
  • the LIDAR system 108 may use the reflected laser light to determine a size, shape, distance, surface texture, or other information about a physical object or material. For example, the LIDAR system 108 may sweep an area to obtain data or objects within a specific range and viewing angle of the LIDAR system 108 .
  • the LIDAR system 108 may obtain data about regions of the ground or vertical area immediately neighboring or near the vehicle.
  • the LIDAR system 108 may include one of many widely available commercially available LIDAR systems.
  • the LIDAR system 108 may provide perception data including a two dimensional or three-dimensional model or map of detected objects or surfaces.
  • the camera system 110 may include one or more cameras, such as visible wavelength cameras or infrared cameras.
  • the camera system 110 may provide a video feed or periodic images, which can be processed for object detection, road identification and positioning, or other detection or positioning.
  • the camera system 110 may include two or more cameras, which may be used to provide ranging (e.g., detecting a distance) for objects within view.
  • image processing may be used on captured camera images or video to detect vehicles, turn signals, drivers, gestures, and/or body language of a driver.
  • the camera system 110 may include cameras that obtain images for two or more directions around the vehicle.
  • the GPS system 112 is one embodiment of a positioning system that may provide a geographical location of the vehicle based on satellite or radio tower signals. GPS systems 112 are well known and widely available in the art. Although GPS systems 112 can provide very accurate positioning information, GPS systems 112 generally provide little or no information about distances between the vehicle and other objects. Rather, they simply provide a location, which can then be compared with other data, such as maps, to determine distances to other objects, roads, or locations of interest.
  • the ultrasound system 114 may be used to detect objects or distances between a vehicle and objects using ultrasonic waves.
  • the ultrasound system 114 may emit ultrasonic waves from a location on or near a bumper or side panel location of a vehicle.
  • the ultrasonic waves which can travel short distances through air, may reflect off other objects and be detected by the ultrasound system 114 .
  • the ultrasound system 114 may be able to detect accurate distances between a bumper or side panel and any other objects. Due to its shorter range, ultrasound systems 114 may be more useful to detect objects during parking or to detect imminent collisions during driving.
  • the radar system(s) 106 , the LIDAR system(s) 108 , the camera system(s) 110 , and the ultrasound system(s) 114 may detect environmental attributers or obstacles near a vehicle.
  • the systems 106 - 110 and 114 may detect other vehicles, pedestrians, people, animals, a number of lanes, lane width, shoulder width, road surface curvature, road direction curvature, rumble strips, lane markings, presence of intersections, road signs, bridges, overpasses, barriers, medians, curbs, or any other details about a road.
  • the systems 106 - 110 and 114 may detect environmental attributes that include information about structures, objects, or surfaces near the road, such as the presence of drive ways, parking lots, parking lot exits/entrances, sidewalks, walkways, trees, fences, buildings, parked vehicles (on or near the road), gates, signs, parking strips, or any other structures or objects.
  • the data store 116 stores map data, driving history, and other data, which may include other navigational data, settings, or operating instructions for the automated driving/assistance system 102 .
  • the map data may include location data, such as GPS location data, for roads, parking lots, parking stalls, or other places where a vehicle may be driven or parked.
  • the location data for roads may include location data for specific lanes, such as lane direction, merging lanes, highway or freeway lanes, exit lanes, or any other lane or division of a road.
  • the location data may also include locations for one or more parking stall in a parking lot or for parking stalls along a road.
  • the map data includes location data about one or more structures or objects on or near the roads or parking locations.
  • the map data may include data regarding GPS sign location, bridge location, building or other structure location, or the like.
  • the map data may include precise location data with accuracy within a few meters or within sub meter accuracy.
  • the map data may also include location data for paths, dirt roads, or other roads or paths, which may be driven by a land vehicle.
  • the transceiver 118 is configured to receive signals from one or more other data or signal sources.
  • the transceiver 118 may include one or more radios configured to communicate according to a variety of communication standards and/or using a variety of different frequencies.
  • the transceiver 118 may receive signals from other vehicles. Receiving signals from another vehicle is referenced herein as vehicle-to-vehicle (V2V) communication.
  • V2V vehicle-to-vehicle
  • the transceiver 118 may also be used to transmit information to other vehicles to potentially assist them in locating vehicles or objects.
  • the transceiver 118 may receive information from other vehicles about their locations, previous locations or states, other traffic, accidents, road conditions, the locations of parking barriers or parking chocks, or any other details that may assist the vehicle and/or automated driving/assistance system 102 in driving accurately or safely.
  • the transceiver 118 may receive updated models or algorithms for use by a driver intent component 104 in detecting vehicle movement, turn signals, or body language of a driver of another vehicle.
  • the transceiver 118 may receive signals from other signal sources that are at fixed locations.
  • Infrastructure transceivers may be located at a specific geographic location and may transmit its specific geographic location with a time stamp.
  • the automated driving/assistance system 102 may be able to determine a distance from the infrastructure transceivers based on the time stamp and then determine its location based on the location of the infrastructure transceivers.
  • receiving or sending location data from devices or towers at fixed locations is referenced herein as vehicle-to-infrastructure (V2X) communication.
  • V2X communication may also be used to provide information about locations of other vehicles, their previous states, or the like.
  • V2X communications may include information about how long a vehicle has been stopped or waiting at an intersection, highway on-ramp signal, or the like.
  • the term V2X communication may also encompass V2V communication.
  • the automated driving/assistance system 102 is configured to control driving or navigation of a parent vehicle.
  • the automated driving/assistance system 102 may control the vehicle control actuators 120 to drive a path on a road, parking lot, through an intersection, driveway or other location.
  • the automated driving/assistance system 102 may determine a path and speed to drive based on information or perception data provided by any of the components 106 - 118 .
  • the automated driving/assistance system 102 may determine when to change lanes, merge, or when to leave space for another vehicle to change lanes, or the like.
  • the driver intent component 104 is configured to determine an intent of a driver of a nearby vehicle and/or predict a future movement, and timing for the movement, of a vehicle under control of a human driver. For example, the driver intent component 104 is configured to determine whether or not to change lanes or to determine the timing to change lanes.
  • FIG. 2 is a schematic top view of a roadway 200 with a vehicle 202 traveling on the roadway.
  • the vehicle 202 is traveling in a first lane 208 of the roadway 200 .
  • the roadway also includes a second lane 210 and a third lane 212 that have a same direction of travel as the first lane 208 .
  • Additional vehicles 204 and 206 are also traveling along the roadway 200 in the third lane 212 .
  • the vehicle 202 may include the system 100 of FIG. 1 .
  • one or more sensors such as the camera system 110 , may gather perception data of the road 200 and regions around the vehicle 202 .
  • a viewing area of the camera system 110 , the LIDAR system 108 , the radar system 106 or other system may extend in any direction or all directions around the vehicle 202 .
  • the vehicle 202 or a driver intent component 104 of the vehicle 202 , may receive perception data from the sensors and detect the presence of other vehicles, objects, surfaces, or the like within a viewing range of the vehicle 202 .
  • the vehicle 202 may detect and identify additional vehicles 204 and 206 as nearby vehicles.
  • the vehicle 202 may detect a presence of the additional vehicles 204 and 206 in other lanes of the roadway 200 .
  • the vehicle 202 may capture one or more images of the additional vehicles and identify a sub-portion of the captured images that corresponds to locations of turn signals on the other vehicles that are visible in the images.
  • the vehicle 202 may process the captured images using a neural network to identify regions that are likely to correspond to a location of turn signals.
  • the vehicle 202 may analyze or process the sub-portion of the perception data (or image) to determine a state of a turn signal. Based on the state of the turn signal, the vehicle 202 may notify a driver to delay changing lanes into the second lane 210 or may cause an autonomous driving system to delay changing lanes into the second lane 210 .
  • the vehicle control system 100 of FIG. 1 may help warn or prevent the vehicle 202 from merging into a lane at the same time as another vehicle, or may warn a driver or cause the vehicle 202 to slow down or speed up to allow another vehicle to merge into the vehicle's 202 current lane, such as 208 .
  • Notifications and determinations that vehicles are changing lanes may help reduce accidents between moving vehicles traveling in a same or similar direction along a roadway.
  • the vehicle 202 may detect when the additional vehicle 204 has a left turn signal on and thus the vehicle 202 should wait or avoid changing lanes into the second lane 210 .
  • the vehicle 202 may detect the left turn signal of vehicle 204 and inform the driver (or an automated driving system) that the vehicle 202 should slow down to let the additional vehicle 204 to enter into the first lane 208 in front of the vehicle 202 .
  • the vehicle 202 may detect that the additional vehicle 206 is changing from the third lane 212 into the second lane 210 . If the additional vehicle 206 is within the second lane 210 with a left turn signal on, then the vehicle 202 may speed up to allow the additional vehicle 206 to enter into the first lane 208 behind the vehicle 202 .
  • the vehicle control system 100 may determine one or more details about movement of the additional vehicles 204 and 206 . For example, the vehicle control system 100 may detect a relative speed, direction, or other movement of the additional vehicles 204 and 206 . These movement details may further inform whether a lane change, slow down, acceleration, or other driving maneuver should be performed by the vehicle 202 to avoid collision or maintain safe driving distances between vehicles.
  • FIG. 3 illustrates a perspective view of an image 300 that may be captured by a camera of a vehicle control system 100 .
  • the image 300 shows a rear side of a vehicle 302 .
  • a sub-portion 304 of the image includes turn signals of the vehicle 302 .
  • the vehicle control system 100 may identify the sub-region 304 as a region to be processed for determining states of turn signals. For example, the vehicle control system 100 may generate a bounding box including the sub-region 304 and feed image content within the bounding box into a neural network that has been trained to detect turn signal states. Based on the state of the turn signal indicators, the vehicle control system 100 may determine that the vehicle 302 is about to change lanes or may leave a distance to allow the vehicle 302 to change lanes.
  • an inadvertent turn signal may indicate that the vehicle 302 movements may be unpredictable due to an inattentive human operator or a software error of an automated driving system.
  • FIG. 4 illustrates another perspective view of an image 400 that may be captured by a camera of a vehicle control system 100 .
  • the image 400 shows a front left view of the vehicle 402 .
  • a rearward or sideways facing camera on a vehicle may capture the image 400 when the vehicle 402 is behind and to the right of a parent vehicle.
  • the vehicle control system 100 may identify a first sub-region 404 and a second sub-region 406 as regions to be processed for determining states of turn signals.
  • the vehicle control system 100 may generate a bounding box, including the sub-regions 404 and 406 , and feed image content within the bounding boxes into a neural network that has been trained to detect turn signal states.
  • the second sub-region 406 includes a turn signal, specifically the left turn signal 408 of the vehicle 402 .
  • a right turn signal of the vehicle 402 is out of view and is not actually visible in the first sub-region 404 .
  • the vehicle control system 100 may detect the state of the left turn signal 408 and determine that no other turn signals are detectable. Based on the state of the left turn signal 408 , the vehicle control system 100 may be able to predict one or more actions of the vehicle 402 .
  • the vehicle control system 100 may determine that the vehicle 402 is either preparing for a switch into a lane to the left of the vehicle 402 or may determine that the vehicle 402 has hazard lights flashing and thus a driver of the vehicle may be in a hurry and space should be provided to allow the vehicle 402 room to navigate.
  • the vehicle control system 100 may determine that the vehicle 402 will likely remain in the same lane or move to a lane to the right of the vehicle. Because staying in the same lane or moving to the right may indicate that the vehicle 402 is not likely to make movements that would be threatening or unsafe to the vehicle that captured the image 400 .
  • FIG. 5 is a schematic block diagram illustrating a method 500 for deciding on a driving maneuver.
  • Perception data such as camera data, LIDAR data, radar data and ultrasound data
  • a driver intent component 104 identifies and localizes a vehicle based on the perception data at 504 .
  • the driver intent component 104 may identify a region of a viewing area or a region of an image that corresponds to a vehicle and may localize that vehicle in a same or adjacent lane.
  • the driver intent component 104 may identify a lane of the other vehicle with respect to a parent vehicle of the driver intent component 104 .
  • the adjacent lane may include an immediately adjacent lane (e.g., to a right or left of a current lane of a vehicle) or may include a lane offset by one or more lanes from the vehicle (e.g., two lanes to the right or left of the current lane).
  • the driver intent component 104 finds a region of interest at 506 in an image in the camera data that includes or likely includes a visible turn signal of a localized vehicle. For example, the driver intent component 104 may feed an image captured by a camera into a neural network that identifies regions including, or likely including, vehicle turn signals.
  • the region of interest may include a region near a front or rear bumper, a headlight, and/or a tail light.
  • the driver intent component 104 also determines a turn signal state or identifies the status of visible turn signals at 508 . For example, the driver intent component 104 may determine whether a turn signal indicator is off or blinking at 508 . Furthermore, the driver intent component 104 may determine an identity of a visible turn signal as a left front, right front, left rear, and/or right rear turn signal. Furthermore, the driver intent component 104 may determine an identity of turn signals that are not visible (e.g., left front, right front, left rear, and/or right rear turn signal).
  • the driver intent component 104 may then provide the turn signal status, and/or turn signal identity, to a decision matrix at 510 .
  • the decision matrix may include a plurality of values that may be considered in determining a future maneuver of a vehicle. For example, the matrix may be used to determine whether to change a lane, speed up, slow down, or perform any other maneuver or combination of maneuvers. In one embodiment, the matrix may be used to determine what notifications to provide to a driver of a vehicle. For example, possible notifications may include blind spot warnings, lane change instructions, or any other notifications or alerts to a human driver.
  • FIG. 6 is a schematic block diagram illustrating components of a driver intent component 104 , according to one embodiment.
  • the driver intent component 104 includes a perception data component 602 , a detection component 604 , a boundary component 606 , a turn signal component 608 , a vehicle movement component 610 , a prediction component 612 , a driving maneuver component 614 , and a notification component 616 .
  • the components 602 - 616 are given by way of illustration only and may not all be included in all embodiments. In fact, some embodiments may include only one or any combination of two or more of the components 602 - 616 . Some of the components 602 - 616 may be located outside the driver intent component 104 , such as within the automated driving/assistance system 102 or elsewhere.
  • the perception data component 602 is configured to receive sensor data from one or more sensor systems of the vehicle.
  • the perception data component 602 may receive data from the radar system 106 , the LIDAR system 108 , the camera system 110 , the GPS 112 , the ultrasound system 114 , or the like.
  • the perception data may include perception data for one or more regions near the vehicle.
  • sensors of the vehicle may provide a 360 degree view around the vehicle.
  • the camera system 110 captures an image of a vehicle.
  • the captured image of the vehicle may be proximal to a parent vehicle of the driver intent component 104 .
  • the camera system 110 captures an image of a proximal vehicle that is in a same lane or in a lane near the vehicle and traveling in the same direction as the parent vehicle.
  • the detection component 604 is configured to detect a presence of one or more nearby vehicles. In one embodiment, the detection component 604 detects the nearby vehicles based on perception data gathered by the perception data component 602 . For example, the detection component 604 may detect a moving object about the size of the vehicle, or may use object recognition on images obtained by a camera system 110 to detect the vehicle. In one embodiment, the detection component 604 may detect lane lines, or the like, to detect lanes or other physical features to localize vehicles with respect to a parent vehicle.
  • the detection component 604 is configured to determine a lane of one or more of the vehicle and the proximal vehicle. For example, the detection component 604 may determine that a parent vehicle is in a first lane and that a detected proximal vehicle is in a second lane or a third lane neighboring the first lane. In one embodiment, the detection component 604 is configured to determine whether a proximal vehicle is in an adjacent lane to a current lane of the parent vehicle. The adjacent lane may be a lane immediately adjacent a current lane or may be offset by one or more intervening lanes.
  • the detection component 604 may determine whether a detected vehicle is within a risk zone of the parent vehicle.
  • the risk zone may be a region within a threshold distance of the parent vehicle. In one embodiment, the threshold distance may vary based on a speed or current road conditions.
  • the risk zone may include an area where there is a risk of collision between a parent vehicle and a detected vehicle.
  • the detection component 604 may also determine whether the vehicle is moving in the same or similar direction as a parent vehicle. For example, the detection component 604 may determine whether the proximal vehicle is traveling in the same direction along the roadway as the parent vehicle.
  • the boundary component 606 is configured to identify a sub-region of perception data that corresponds to, or likely corresponds to, a location of a turn signal. In one embodiment, the boundary component 606 is configured to locate one or more vehicles within images or other perception data. For example, object recognition algorithms may be used to identify detected objects or obstacles as vehicles. In one embodiment, the boundary component 606 may identify a boundary of the vehicle and identify pixels or objects in that region as corresponding to the vehicle. Edge or boundary finding image processing algorithms may be used to find the edges of the vehicle.
  • the boundary component 606 is configured to identify a sub-portion of an image (or multiple images) that contains a turn signal indicator of a vehicle.
  • the sub-portion of the image may include a turn signal indicator light positioned on a front, rear, or any other location of a vehicle.
  • the sub-portion may include a region on or near a bumper, and/or a region near a headlight or tail light of the vehicle.
  • the boundary component 606 may identify the sub-portion of the image that contains the turn signal using object recognition or edge detection image processing algorithms.
  • the boundary component 606 may identify an edge or boundary of an indicator light, headlight, tail light, or the like.
  • the boundary component 606 is configured to identify the sub-portion containing the turn signal indicator by processing the image of the proximal vehicle using a neural network trained to recognize one or more turn signal regions of interest. In one embodiment, the boundary component 606 may determine a region surrounding the turn signal indicator or a region larger than the turn signal indicator so that states of the turn signal indicator may be accurately determined even if the boundary is not perfectly aligned or centered on the turn-signal indicator.
  • the turn signal component 608 is configured to determine a state of a turn signal of a proximal vehicle. For example, the turn signal component 608 may determine a state of one or more turn signals of a vehicle that is located in a same or adjacent lane near a parent vehicle. In one embodiment, the turn signal component 608 may process one or more sub-regions determined by the boundary component 606 to determine the state of the turn signal indicator. For example, the turn signal component 608 may detect whether a turn signal indicator is emitting light or blinking based on one or more images of the turn signal indicator. In one embodiment, the turn signal component 608 is configured to process the sub-portion of the image using a neural network trained to determine a state of one or more turn signal indicators.
  • the turn signal component 608 may determine whether one or more of a left turn signal indicator and a right turn indicator are on or blinking. For example, the turn signal indicator 608 may determine whether only the left turn signal indicator is flashing, only the right turn signal indicator is flashing, both the left turn signal indicator and the right turn signal indicator are flashing, or neither the left turn signal indicator nor the right turn signal indicator are flashing. In one embodiment, the turn signal component 608 may also determine whether some of the turn signal indicators of a vehicle are not visible in an image. For example, the turn signal component 608 may determine that a specific turn signal indicator has an unknown state because it is not visible, or is unknown for some other reason.
  • the vehicle movement component 610 is configured to detect one or more movements of a proximal vehicle. For example, the vehicle movement component 610 may detect the movements of the proximal vehicle based on perception data or other sensor data received by the perception data component 602 . In one embodiment, the vehicle movement component 610 may determine one or more accelerations, decelerations, turns, or the like of a proximal vehicle. In one embodiment, the vehicle movement component 610 may detect that a vehicle is traveling in a same or similar direction as a parent vehicle. For example, the vehicle movement component 610 may determine that the proximal vehicle is traveling along a roadway in a same direction, even if the vehicle is in the same or different lane as the parent vehicle.
  • the boundary component 606 , turn signal component 608 , and/or the vehicle movement component 610 may include models, neural networks, machine learned parameters, or the like to detect body language, turn signal states, and vehicle movements.
  • guided or unguided machine learning algorithms may process perception data from real-world or virtual environments to learn shapes, movements, or other image content that corresponds to body language, turn signal states, or vehicle movement.
  • the results of these machine learning algorithms may be included in models or databases for use by respective components to detect the body language, turn signal states, or vehicle movement during driving of a vehicle.
  • the prediction component 612 may infer a driver's intention or predict future motions of a nearby vehicle based determinations by the turn signal component 608 , the vehicle movement component 610 , and/or other information. For example, the prediction component 612 may predict future motion, and/or a timing for the future motion, based on a state of a turn signal indicator and/or detected vehicle movements. In one embodiment, the prediction component 612 determines the driver's intention or predicts future motions. For example, the prediction component 612 may determine a timing and a movement that the driver intends to perform. Example movements may include a turn onto a different road, waiting at an intersection, merging with traffic, changing lanes, exiting a roadway, entering a roadway, parking a vehicle, exiting a parking spot, or the like.
  • the prediction component 612 references or processes a database or model to determine a predicted movement or intended movement of another vehicle.
  • the prediction component 612 may use a neural network that has been trained to determine a future vehicle movement based on detected vehicle movements and/or the status of a turn signal.
  • the database or model also correlates an intention of a driver or a future driving maneuver based on a current driving context. For example, the same gestures may mean different things based on whether the nearby vehicle or parent vehicle is stopped at an intersection, approaching an intersection, driving down a road with one or more nearby vehicles, merging onto a roadway, exiting a roadway, entering a parking lot or parking spot, exiting a parking lot or parking spot, or the like.
  • gestures and current driving context may be used to accurately infer an intention of a driver or predict a future driving maneuver.
  • the prediction component 612 may provide the predicted driving maneuvers or driver intent to the driving maneuver component 614 or the automated driving/assistance system 102 for decision making and for maneuvers or actions to be taken by the automated driving/assistance system 102 or a parent vehicle.
  • the prediction component 612 determines the driver's intention or predicts future motions of the vehicle based on a state of a turn signal indicator as determined by the turn signal component 608 .
  • the prediction component 612 may predict a timing and direction of travel for the vehicle to change lanes, merge, or exit a roadway.
  • the prediction component 612 references or processes the data based on a neural network to determine the predicted movement or intended movement of another vehicle.
  • the prediction component 612 may include or access a database or model that correlates turn signal statuses with one or more future vehicle movements.
  • the database or model may correlate a blinker direction with the vehicle changing direction in that direction.
  • the database or model may correlate a flashing of hazard lights (e.g., both or all detectable turn signal indicators blinking) with a vehicle moving quickly down a roadway in an unpredictable direction without waiting according to normal protocols.
  • hazard lights e.g., both or all detectable turn signal indicators blinking
  • extra distance and leeway may be provided to the vehicle with flashing hazard lights (or flashing emergency lights) to move down the roadway.
  • the prediction component 612 determines the driver's intention or predicts future motions based on detected movements of the vehicle as determined by the vehicle movement component 610 .
  • the prediction component 612 may predict a timing and direction of travel for the vehicle to move through the intersection.
  • the prediction component 612 references or processes the database or model to determine the predicted movement or intended movement of another vehicle.
  • the prediction component 612 may include or access a database or model that correlates one or more detected movements with one or more future movements.
  • the database may include an acceleration, velocity, deceleration, or other movement information with a predicted further movement through an intersection.
  • the prediction component 612 may predict a lane change from the adjacent lane into a same lane as the first vehicle.
  • the prediction component 612 may predict that a proximal vehicle may change lanes in direction corresponding to a side of the vehicle on which a turn signal indicator is active or blinking.
  • the prediction component 612 may determine a driver's intention or future movement of a vehicle based on a combination of data from the turn signal component 608 , vehicle movement component 610 , or other data.
  • a neural network or graphical model may include machine learning values or correlations for one or more of turn signal information and vehicle movements.
  • the driving maneuver component 614 is configured to select a driving maneuver for a parent vehicle based on the predicted driver intent or future driving maneuver of another vehicle. For example, the driving maneuver component 614 may receive one or more predicted driving maneuvers for one or more nearby vehicles from the prediction component 612 . The driving maneuver may determine a driving path to avoid collision with the other vehicles in case they perform the predicted driving maneuvers. For example, the driving maneuver component 614 may determine whether to decelerate, accelerate, and/or turn a steering wheel of the parent vehicle. In one embodiment, the driving maneuver component 614 may determine a timing for the driving maneuver. For example, the driving maneuver component 614 may determine that a parent vehicle should wait to perform a lane change or perform a lane change at a specific time because another vehicle is likely to be near the vehicle in that lane.
  • the driving maneuver component 614 may select a driving maneuver based directly on data gathered by the turn signal component 608 , vehicle movement component 610 , and/or other components of the driver intent component 104 .
  • the driving maneuver component 614 may select a driving maneuver based on a state of a turn signal indicator and/or a location and velocity of another vehicle.
  • a selected driving maneuver may be a suggested driving maneuver that is provided to a driver or a system that makes driving or navigation maneuvers.
  • the driving maneuver component 614 may enter or include data or determinations from the turn signal component 608 , vehicle movement component 610 , and/or the prediction component 612 in a decision matrix.
  • the decision matrix may include a matrix that is processed using a neural network or a processing algorithm to determine a maneuver the parent vehicle should perform.
  • the decision matrix may include a matrix for deciding whether and/or when to perform a lane change.
  • the notification component 616 is configured to provide one or more notifications to a driver or automated driving system of a vehicle.
  • the notification component 616 may provide notifications to a driver using a display 122 or speaker 124 .
  • the notification may include an instruction to perform a maneuver or may warn that another vehicle is likely to perform a specific maneuver or be at a specific location at a predicted time.
  • the notification component 616 may notify the driver or automated driving system of a driving maneuver selected or suggested by the driving maneuver component 614 .
  • the notification component 616 may notify the driver or automated driving system of a predicted future movement of another vehicle as determined by the prediction component 612 .
  • the method 700 may be performed by an automated driving/assistance system or a driver intent component, such as the automated driving/assistance system 102 of FIG. 1 or the driver intent component 104 of FIG. 1 or 6 .
  • the method 700 begins as a boundary component 606 identifies, in an image of a proximal vehicle, a sub-portion containing a turn signal indicator of the proximal vehicle at 702 .
  • a turn signal component 608 processes the sub-portion of the image to determine a state of the turn signal indicator at 704 .
  • a notification component 616 notifies a driver or a driving maneuver component 614 causes a vehicle to perform a driving maneuver based on the state of the turn signal indicator of the proximal vehicle at 706 .
  • Example 1 is a method that includes detecting, at a first vehicle, a presence of a second vehicle in an adjacent lane. The method also includes identifying, in an image of the second vehicle, a sub-portion containing a turn signal indicator of the second vehicle. The method further includes processing the sub-portion of the image to determine a state of the turn signal indicator. The method also includes notifying a driver or performing a driving maneuver, at the first vehicle, based on the state of the turn signal indicator.
  • identifying the sub-portion containing the turn signal indicator in Example 1 includes processing the image of the second vehicle using a neural network trained to recognize one or more turn signal regions of interest. Furthermore, processing the sub-portion of the image includes processing a portion of the image comprising the one or more turn signal regions of interest.
  • processing the sub-portion of the image in any of Examples 1-2 includes processing the sub-portion using a neural network trained to determine a state of one or more turn signal indicators.
  • Example 4 the method of any of Examples 1-3 further includes determining the driving maneuver for the first vehicle based on the state of the turn signal indicator.
  • determining the driving maneuver in Example 4 includes providing the state of the turn signal indicator into a lane-changing decision matrix and processing the lane-changing decision matrix to select the driving maneuver.
  • notifying the driver in Example 4 includes notifying the driver of the determined driving maneuver.
  • Example 7 the method of any of Examples 1-6 further includes predicting future movement of the second vehicle based on a state of the turn signal indicator.
  • Example 8 predicting future movement in Example 7 includes predicting a lane change from the adjacent lane into a same lane as the first vehicle.
  • notifying the driver in Example 7 includes notifying the driver of the predicted future movement of the second vehicle.
  • Example 10 the method of any of Examples 1-9 further include determining a direction of movement of the first vehicle corresponding to the state of the turn signal indicator.
  • Example 11 is a driving control system for a vehicle that includes one or more sensors for obtaining sensor data in a region near a vehicle that include a camera.
  • the driving control system also includes a boundary component, a turn signal component, and a driving maneuver component.
  • the boundary component is configured to identify a sub-portion of an image containing a turn signal indicator of a proximal vehicle.
  • the turn signal component is configured to process the sub-portion of the image to determine a state of the turn signal indicator.
  • the driving maneuver component is configured to determine a driving maneuver for the first vehicle based on the state of the turn signal indicator.
  • Example 12 the driving maneuver in Example 12 includes a suggested driving maneuver and the driving control system further includes a notification component configured to provide the suggested driving maneuver to a human driver or an automated driving system.
  • Example 13 the boundary component in any of Examples 11-12 is configured to identify the sub-portion containing the turn signal indicator by processing the image of the proximal vehicle using a neural network trained to recognize one or more turn signal regions of interest.
  • the turn signal component is configured to process the sub-portion of the image by processing a portion of the image comprising the one or more turn signal regions of interest.
  • Example 14 the driving control system of any of Examples 11-13 further includes a detection component configured to determine that the proximal vehicle is within a risk zone with respect to the vehicle.
  • the driving control system is configured to obtain the image in response to the detection component determining that the proximal vehicle is within the risk zone.
  • Example 15 the driving control system of any of Examples 11-14 further includes a detection component configured to determine a lane of one or more of the vehicle and the proximal vehicle.
  • Example 16 is a computer readable storage media storing instructions that, when executed by one or more processors, cause the one or more processors to obtain, at a first vehicle, a plurality of images of a second vehicle on a road near the first vehicle. The instructions further cause the one or more processors to identify a sub-portion of the plurality of images containing a turn signal indicator of the second vehicle. The instructions further cause the one or more processors to process the sub-portion of the plurality of images to determine a state of the turn signal indicator. The instructions further cause the one or more processors to predict a driving maneuver for the second vehicle based on the state of the turn signal indicator.
  • Example 17 the instructions of Example 16 further cause the one or more processors to determine a driving maneuver for the first vehicle based on the predicted driving maneuver for the second vehicle.
  • Example 18 the instructions in any of Examples 16-17 further cause the one or more processors to determine whether or not to cause the first vehicle to perform a lane change based on the turn signal indicator.
  • Example 19 the instructions in any of Examples 16-18 further cause the one or more processors to determine that the second vehicle is one or more of: in a lane immediately adjacent a lane of the first vehicle; driving in a similar direction along a roadway as the first vehicle; and within a risk zone of the first vehicle, wherein the risk zone corresponds to an area where there is a risk of collision between the first vehicle and the second vehicle.
  • Example 20 the instructions in any of Examples 16-19 further cause the one or more processors to determine that the state of the turn signal indicator corresponds to a direction of movement of the first vehicle.
  • Example 21 is a system or device that includes means for implementing a method or realizing a system or apparatus in any of Examples 1-20.
  • a sensor may include computer code configured to be executed in one or more processors, and may include hardware logic/electrical circuitry controlled by the computer code.
  • processors may include hardware logic/electrical circuitry controlled by the computer code.
  • Embodiments of the disclosure have been directed to computer program products comprising such logic (e.g., in the form of software) stored on any computer useable medium.
  • Such software when executed in one or more data processing devices, causes a device to operate as described herein.

Landscapes

  • Engineering & Computer Science (AREA)
  • Mechanical Engineering (AREA)
  • Transportation (AREA)
  • Automation & Control Theory (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Combustion & Propulsion (AREA)
  • Chemical & Material Sciences (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Remote Sensing (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Aviation & Aerospace Engineering (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Biomedical Technology (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Traffic Control Systems (AREA)
  • Image Analysis (AREA)
US14/973,454 2015-12-17 2015-12-17 Vehicle turn signal detection Active US10800455B2 (en)

Priority Applications (6)

Application Number Priority Date Filing Date Title
US14/973,454 US10800455B2 (en) 2015-12-17 2015-12-17 Vehicle turn signal detection
RU2016147944A RU2016147944A (ru) 2015-12-17 2016-12-07 Выявление сигнала поворота транспортного средства
DE102016123878.3A DE102016123878A1 (de) 2015-12-17 2016-12-08 Fahrzeugblinksignaldetektion
GB1621096.5A GB2547082A (en) 2015-12-17 2016-12-12 Vehicle turn signal detection
CN201611138148.7A CN106891888B (zh) 2015-12-17 2016-12-12 车辆转向信号检测
MX2016016811A MX2016016811A (es) 2015-12-17 2016-12-16 Deteccion de señal de giro de vehículo.

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
US14/973,454 US10800455B2 (en) 2015-12-17 2015-12-17 Vehicle turn signal detection

Publications (2)

Publication Number Publication Date
US20170174261A1 US20170174261A1 (en) 2017-06-22
US10800455B2 true US10800455B2 (en) 2020-10-13

Family

ID=58222043

Family Applications (1)

Application Number Title Priority Date Filing Date
US14/973,454 Active US10800455B2 (en) 2015-12-17 2015-12-17 Vehicle turn signal detection

Country Status (6)

Country Link
US (1) US10800455B2 (de)
CN (1) CN106891888B (de)
DE (1) DE102016123878A1 (de)
GB (1) GB2547082A (de)
MX (1) MX2016016811A (de)
RU (1) RU2016147944A (de)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11299148B2 (en) 2017-08-25 2022-04-12 Honda Motor Co., Ltd. Travel control apparatus and vehicle
DE102021201062A1 (de) 2021-02-04 2022-08-04 Volkswagen Aktiengesellschaft Verfahren zum Betreiben eines Kraftfahrzeugs sowie Kraftfahrzeug
DE102022003323A1 (de) 2021-09-15 2023-04-06 Mercedes-Benz Group AG System und Verfahren für die Handhabung von Überholvorgängen

Families Citing this family (71)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2017118907A1 (en) 2016-01-05 2017-07-13 Mobileye Vision Technologies Ltd. Systems and methods for estimating future paths
US10460600B2 (en) * 2016-01-11 2019-10-29 NetraDyne, Inc. Driver behavior monitoring
US10706309B2 (en) * 2016-02-29 2020-07-07 Nec Corporation Recursive neural networks on future event prediction
CN108140321A (zh) * 2016-03-01 2018-06-08 松下电器产业株式会社 信息提示方法及信息提示装置
EP3491358A4 (de) 2016-07-31 2019-07-31 Netradyne, Inc. Bestimmung der ursachen von verkehrsereignissen und förderung eines guten fahrverhalten
DE102016120876A1 (de) * 2016-11-02 2018-05-03 HELLA GmbH & Co. KGaA Verfahren, System und Computerprogrammprodukt zur Erkennung eines möglichen Spurwechsels eines Fremdfahrzeugs, sowie Fahrzeug
EP3548845B1 (de) * 2017-01-12 2021-10-13 Mobileye Vision Technologies Ltd. Navigation auf der basis von fahrzeugaktivität
US10671076B1 (en) 2017-03-01 2020-06-02 Zoox, Inc. Trajectory prediction of third-party objects using temporal logic and tree search
US10353390B2 (en) 2017-03-01 2019-07-16 Zoox, Inc. Trajectory generation and execution architecture
US11009875B2 (en) * 2017-03-09 2021-05-18 Waymo Llc Preparing autonomous vehicles for turns
DE102017211244B3 (de) * 2017-07-03 2018-12-27 Volkswagen Aktiengesellschaft Verfahren zum Anreichern eines Sichtfeldes eines Fahrers eines Fahrzeuges mit Zusatzinformationen, Vorrichtung zur Verwendung in einem Beobachter-Fahrzeug, Vorrichtung zur Verwendung in einem Objekt sowie Kraftfahrzeug
KR101973627B1 (ko) * 2017-07-11 2019-04-29 엘지전자 주식회사 차량에 구비된 차량 제어 장치 및 차량의 제어방법
CN110997415A (zh) * 2017-08-01 2020-04-10 三星电子株式会社 控制车辆的操作的电子设备和方法
US20190072978A1 (en) * 2017-09-01 2019-03-07 GM Global Technology Operations LLC Methods and systems for generating realtime map information
US10163017B2 (en) * 2017-09-01 2018-12-25 GM Global Technology Operations LLC Systems and methods for vehicle signal light detection
US10957201B2 (en) * 2017-09-15 2021-03-23 Qualcomm Incorporated System and method for relative positioning based safe autonomous driving
US10691962B2 (en) 2017-09-22 2020-06-23 Toyota Motor Engineering & Manufacturing North America, Inc. Systems and methods for rear signal identification using machine learning
EP3687863A4 (de) 2017-09-29 2020-12-02 Netradyne, Inc. Bestimmung eines mehrfach-belichtungsereignisses
CN107719347B (zh) * 2017-09-30 2020-06-02 北京小米移动软件有限公司 移动照明方法、装置和存储介质
JP6983636B2 (ja) * 2017-11-29 2021-12-17 株式会社東芝 図面機械学習支援システム、図面機械学習システム、図面構造化システム、及び図面機械学習支援プログラム
FR3076046A1 (fr) * 2017-12-22 2019-06-28 Orange Procede de determination d'un scenario de communications et terminal associe
CN111527013B (zh) * 2017-12-27 2024-02-23 宝马股份公司 车辆变道预测
US11042163B2 (en) 2018-01-07 2021-06-22 Nvidia Corporation Guiding vehicles through vehicle maneuvers using machine learning models
DE102018200944A1 (de) * 2018-01-22 2019-07-25 Ford Global Technologies, Llc Cross Traffic Alert mit Blinkerkennung
WO2019152888A1 (en) 2018-02-02 2019-08-08 Nvidia Corporation Safety procedure analysis for obstacle avoidance in autonomous vehicle
US10955851B2 (en) 2018-02-14 2021-03-23 Zoox, Inc. Detecting blocking objects
US10525873B2 (en) * 2018-02-20 2020-01-07 Toyota Research Institute, Inc. Turn by turn activation of turn signals
EP3533667B1 (de) 2018-03-01 2020-09-16 KNORR-BREMSE Systeme für Nutzfahrzeuge GmbH Vorrichtung und verfahren zur überwachung eines fahrzeugkamerasystems
US20190275888A1 (en) * 2018-03-09 2019-09-12 Honda Motor Co., Ltd. Methods and systems for providing visual notifications in the peripheral vision of a driver
US10388157B1 (en) 2018-03-13 2019-08-20 Allstate Insurance Company Processing system having a machine learning engine for providing a customized driving assistance output
US10414395B1 (en) 2018-04-06 2019-09-17 Zoox, Inc. Feature-based prediction
US11049265B2 (en) * 2018-05-16 2021-06-29 Nec Corporation Balancing diversity and precision of generative models with complementary density estimators
JP6911810B2 (ja) * 2018-05-16 2021-07-28 トヨタ自動車株式会社 車外情報共有システム、車外情報共有装置及び車外情報共有方法
US11126873B2 (en) 2018-05-17 2021-09-21 Zoox, Inc. Vehicle lighting state determination
CN108520639B (zh) * 2018-06-15 2023-10-27 中国安全生产科学研究院 一种防止车辆碰撞的预警方法及系统
US11341356B2 (en) * 2018-06-15 2022-05-24 Uatc, Llc System and method for determining object intention through visual attributes
US11966838B2 (en) 2018-06-19 2024-04-23 Nvidia Corporation Behavior-guided path planning in autonomous machine applications
US20190389385A1 (en) * 2018-06-25 2019-12-26 Ford Global Technologies, Llc Overlay interfaces for rearview mirror displays
EP3816668A4 (de) * 2018-06-29 2021-08-04 Sony Group Corporation Fahrzeugsteuerungsvorrichtung und fahrzeugsteuerungsverfahren
CN110733496B (zh) * 2018-07-18 2024-08-23 松下知识产权经营株式会社 信息处理装置、信息处理方法以及记录介质
US11287814B2 (en) 2018-08-07 2022-03-29 GM Global Technology Operations LLC Lane change detection system and method for an autonomous vehicle
EP3626585A1 (de) 2018-09-21 2020-03-25 Visteon Global Technologies, Inc. Fahrerassistenzsystem und verfahren zur anzeige von verkehrsinformationen
DE102018218015A1 (de) * 2018-10-22 2020-04-23 Robert Bosch Gmbh Verfahren und Vorrichtung zum Ermitteln eines Zustands einer Fahrzeugleuchte eines Fahrzeugs
DE102018126825A1 (de) * 2018-10-26 2020-04-30 Bayerische Motoren Werke Aktiengesellschaft Steuerung eines Kraftfahrzeugs
US11372403B2 (en) * 2018-11-05 2022-06-28 Tusimple, Inc. Systems and methods for dynamic predictive control of autonomous vehicles
CN113614730B (zh) * 2018-11-15 2023-04-28 御眼视觉技术有限公司 多帧语义信号的cnn分类
US11294381B2 (en) 2018-11-21 2022-04-05 Toyota Motor North America, Inc. Vehicle motion adaptation systems and methods
CN109334668B (zh) * 2018-11-28 2020-06-09 奇瑞汽车股份有限公司 一种基于v2x的自动并线系统
US11520345B2 (en) 2019-02-05 2022-12-06 Nvidia Corporation Path perception diversity and redundancy in autonomous machine applications
DE102019206178A1 (de) 2019-04-30 2020-11-05 Ford Global Technologies, Llc Verfahren und Vorrichtung zum Abschätzen einer Fahrspurwechselabsicht eines Fahrzeugs
JP7505490B2 (ja) * 2019-06-04 2024-06-25 ソニーグループ株式会社 撮像システム
US11120566B2 (en) * 2019-06-28 2021-09-14 Baidu Usa Llc Determining vanishing points based on feature maps
US11423672B2 (en) * 2019-08-02 2022-08-23 Dish Network L.L.C. System and method to detect driver intent and employ safe driving actions
US11662746B2 (en) 2019-08-06 2023-05-30 Here Global B.V. System, method, and computer program product for generating maneuver data for a vehicle
JP7165109B2 (ja) * 2019-09-09 2022-11-02 本田技研工業株式会社 車両制御装置、車両制御方法、およびプログラム
US11668797B2 (en) * 2019-12-18 2023-06-06 Micron Technology, Inc. Intelligent radar electronic control units in autonomous vehicles
US11195033B2 (en) * 2020-02-27 2021-12-07 Gm Cruise Holdings Llc Multi-modal, multi-technique vehicle signal detection
DE102020202986A1 (de) * 2020-03-10 2021-09-16 Robert Bosch Gesellschaft mit beschränkter Haftung Verfahren und Vorrichtung zum Betreiben eines Motorrads
JP7115502B2 (ja) 2020-03-23 2022-08-09 トヨタ自動車株式会社 物体状態識別装置、物体状態識別方法及び物体状態識別用コンピュータプログラムならびに制御装置
JP7381388B2 (ja) * 2020-04-03 2023-11-15 トヨタ自動車株式会社 信号灯状態識別装置、信号灯状態識別方法及び信号灯状態識別用コンピュータプログラムならびに制御装置
JP7359735B2 (ja) 2020-04-06 2023-10-11 トヨタ自動車株式会社 物体状態識別装置、物体状態識別方法及び物体状態識別用コンピュータプログラムならびに制御装置
JP7388971B2 (ja) * 2020-04-06 2023-11-29 トヨタ自動車株式会社 車両制御装置、車両制御方法及び車両制御用コンピュータプログラム
US20210334645A1 (en) * 2020-04-28 2021-10-28 Nvidia Corporation Notifications determined using one or more neural networks
US12077190B2 (en) 2020-05-18 2024-09-03 Nvidia Corporation Efficient safety aware path selection and planning for autonomous machine applications
US11511751B2 (en) * 2020-05-29 2022-11-29 Toyota Research Institute, Inc. Merge situation exposure algorithms to maximize exposure time
US20220101022A1 (en) * 2020-09-29 2022-03-31 Rivian Ip Holdings, Llc Vehicle cliff and crevasse detection systems and methods
US11749105B2 (en) * 2020-10-01 2023-09-05 Magna Electronics Inc. Vehicular communication system with turn signal identification
CN114312562A (zh) * 2020-10-12 2022-04-12 怀化市联翰信息科技有限公司 一种基于gps的车辆碰撞预警系统及方法
TR2021019621A2 (tr) * 2021-12-10 2021-12-21 Hasan Bilgehan Makineci Qr kod araciliğiyla araçlar arasi haberleşme si̇stemi̇
CN114475665B (zh) * 2022-03-17 2024-07-02 北京小马睿行科技有限公司 自动驾驶车辆的控制方法、控制装置及自动驾驶系统
US20230334874A1 (en) * 2022-04-15 2023-10-19 Gm Cruise Holdings Llc Identifying vehicle blinker states

Citations (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7065245B2 (en) 2001-11-08 2006-06-20 Fuji Jukogyo Kabushiki Kaisha Image processing apparatus and the method thereof
US7565006B2 (en) * 2002-08-21 2009-07-21 Gentex Corporation Image acquisition and processing methods for automatic vehicular exterior lighting control
US20120314070A1 (en) 2011-06-09 2012-12-13 GM Global Technology Operations LLC Lane sensing enhancement through object vehicle information for lane centering/keeping
US8615357B2 (en) 2008-02-26 2013-12-24 Robert Bosch Gmbh Method for assisting a user of a vehicle, control device for a driver-assistance system of a vehicle and vehicle having such a control device
US8660734B2 (en) 2010-10-05 2014-02-25 Google Inc. System and method for predicting behaviors of detected objects
US8903588B2 (en) * 2011-08-22 2014-12-02 Honda Research Institute Europe Gmbh Method and system for predicting movement behavior of a target traffic object
US9014905B1 (en) 2014-01-28 2015-04-21 Google Inc. Cyclist hand signal detection by an autonomous vehicle
US20150192428A1 (en) 2012-11-30 2015-07-09 Google Inc. Determining and displaying auto drive lanes in an autonomous vehicle
US9091558B2 (en) 2013-12-23 2015-07-28 Automotive Research & Testing Center Autonomous driver assistance system and autonomous driving method thereof
US9159023B2 (en) * 2010-06-18 2015-10-13 Honda Motor Co., Ltd. System for predicting a driver's intention to change lanes
US9164511B1 (en) * 2013-04-17 2015-10-20 Google Inc. Use of detected objects for image processing
US20160091609A1 (en) * 2014-09-30 2016-03-31 Umm-Al-Qura University Method and system for an accurate and energy efficient vehicle lane detection
US20160214647A1 (en) * 2015-01-23 2016-07-28 Honda Research Institute Europe Gmbh Method for assisting a driver in driving an ego vehicle and corresponding driver assistance system
US20160339959A1 (en) * 2015-05-21 2016-11-24 Lg Electronics Inc. Driver Assistance Apparatus And Control Method For The Same

Family Cites Families (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7444241B2 (en) * 2005-12-09 2008-10-28 Gm Global Technology Operations, Inc. Method for detecting or predicting vehicle cut-ins
KR20100056883A (ko) * 2008-11-20 2010-05-28 현대자동차주식회사 끼어드는 차량을 인식하는 감응식 순항 제어 시스템
CN102208019B (zh) * 2011-06-03 2013-01-09 东南大学 基于车载摄像机的车辆变道检测方法
JP5518007B2 (ja) * 2011-07-11 2014-06-11 クラリオン株式会社 車両用外界認識装置及びそれを用いた車両制御システム
DE112011105430T5 (de) * 2011-07-11 2014-04-17 Toyota Jidosha Kabushiki Kaisha Fahrzeug-Notfallevakuierungsvorrichtung
CN202345598U (zh) * 2011-11-11 2012-07-25 长安大学 一种车辆换道危险预警装置
JP5964609B2 (ja) * 2012-02-23 2016-08-03 株式会社日本自動車部品総合研究所 車両用追従制御装置
KR101715014B1 (ko) * 2013-01-23 2017-03-10 주식회사 만도 주차 보조 장치 및 그 주차 보조 방법
US9926881B2 (en) * 2013-03-11 2018-03-27 Ford Global Technologies Llc Stop/start control for stop/start vehicle in turn lane
CN104698399A (zh) * 2014-08-26 2015-06-10 安徽工程大学 一种车灯故障检测系统及其方法
DE102014220199B3 (de) * 2014-10-06 2016-01-14 Continental Automotive Gmbh Verfahren für ein Kraftfahrzeug mit einer Kamera, Vorrichtung und System

Patent Citations (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7065245B2 (en) 2001-11-08 2006-06-20 Fuji Jukogyo Kabushiki Kaisha Image processing apparatus and the method thereof
US7565006B2 (en) * 2002-08-21 2009-07-21 Gentex Corporation Image acquisition and processing methods for automatic vehicular exterior lighting control
US8615357B2 (en) 2008-02-26 2013-12-24 Robert Bosch Gmbh Method for assisting a user of a vehicle, control device for a driver-assistance system of a vehicle and vehicle having such a control device
US9159023B2 (en) * 2010-06-18 2015-10-13 Honda Motor Co., Ltd. System for predicting a driver's intention to change lanes
US8660734B2 (en) 2010-10-05 2014-02-25 Google Inc. System and method for predicting behaviors of detected objects
US20120314070A1 (en) 2011-06-09 2012-12-13 GM Global Technology Operations LLC Lane sensing enhancement through object vehicle information for lane centering/keeping
US8903588B2 (en) * 2011-08-22 2014-12-02 Honda Research Institute Europe Gmbh Method and system for predicting movement behavior of a target traffic object
US20150192428A1 (en) 2012-11-30 2015-07-09 Google Inc. Determining and displaying auto drive lanes in an autonomous vehicle
US9164511B1 (en) * 2013-04-17 2015-10-20 Google Inc. Use of detected objects for image processing
US9091558B2 (en) 2013-12-23 2015-07-28 Automotive Research & Testing Center Autonomous driver assistance system and autonomous driving method thereof
US9014905B1 (en) 2014-01-28 2015-04-21 Google Inc. Cyclist hand signal detection by an autonomous vehicle
US20160091609A1 (en) * 2014-09-30 2016-03-31 Umm-Al-Qura University Method and system for an accurate and energy efficient vehicle lane detection
US20160214647A1 (en) * 2015-01-23 2016-07-28 Honda Research Institute Europe Gmbh Method for assisting a driver in driving an ego vehicle and corresponding driver assistance system
US9708004B2 (en) * 2015-01-23 2017-07-18 Honda Research Institute Europe Gmbh Method for assisting a driver in driving an ego vehicle and corresponding driver assistance system
US20160339959A1 (en) * 2015-05-21 2016-11-24 Lg Electronics Inc. Driver Assistance Apparatus And Control Method For The Same
US9944317B2 (en) * 2015-05-21 2018-04-17 Lg Electronics Inc. Driver assistance apparatus and control method for the same

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
Almagambetov, A., Casares, M., Velipasalar, S., Autonomous tracking of vehicle rear lights and detection of brakes and turn signals, IEEE.org, Jul. 11-13, 2012.
Duan-Yu Chen. Yang-Jie Peng, Li-Chih Chen, Jun-Wei Hsieh, Nighttime Turn Signal Detection by Scatter Modeling and Reflectance-Based Direction Recognition, IEEE.org, Feb. 14, 2014.

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11299148B2 (en) 2017-08-25 2022-04-12 Honda Motor Co., Ltd. Travel control apparatus and vehicle
DE102021201062A1 (de) 2021-02-04 2022-08-04 Volkswagen Aktiengesellschaft Verfahren zum Betreiben eines Kraftfahrzeugs sowie Kraftfahrzeug
WO2022167247A1 (de) 2021-02-04 2022-08-11 Volkswagen Aktiengesellschaft Verfahren zum betreiben eines kraftfahrzeugs sowie kraftfahrzeug
DE102022003323A1 (de) 2021-09-15 2023-04-06 Mercedes-Benz Group AG System und Verfahren für die Handhabung von Überholvorgängen

Also Published As

Publication number Publication date
CN106891888A (zh) 2017-06-27
CN106891888B (zh) 2022-04-29
DE102016123878A1 (de) 2017-06-22
GB201621096D0 (en) 2017-01-25
US20170174261A1 (en) 2017-06-22
MX2016016811A (es) 2018-06-15
RU2016147944A (ru) 2018-06-08
GB2547082A (en) 2017-08-09

Similar Documents

Publication Publication Date Title
US10800455B2 (en) Vehicle turn signal detection
US11126877B2 (en) Predicting vehicle movements based on driver body language
US9983591B2 (en) Autonomous driving at intersections based on perception data
US20170206426A1 (en) Pedestrian Detection With Saliency Maps
US9751506B2 (en) Algorithms for avoiding automotive crashes at left and right turn intersections
US9862364B2 (en) Collision mitigated braking for autonomous vehicles
US10643474B2 (en) Vehicle control device, vehicle control method, and recording medium
US10077007B2 (en) Sidepod stereo camera system for an autonomous vehicle
US9688273B2 (en) Methods of improving performance of automotive intersection turn assist features
US10479373B2 (en) Determining driver intention at traffic intersections for automotive crash avoidance
CN109421799B (zh) 车辆控制装置、车辆控制方法及存储介质
US20150153184A1 (en) System and method for dynamically focusing vehicle sensors
US10150412B2 (en) Drive history parking barrier alert
CN111919211A (zh) 用于改进转弯操纵中的空间和情景感知的转弯路径可视化
KR20210083462A (ko) 운전자 보조 시스템, 그를 가지는 차량 및 그 제어 방법
US11679780B2 (en) Methods and systems for monitoring vehicle motion with driver safety alerts
JP7495179B2 (ja) 運転支援装置
US20240085209A1 (en) Display control device and display control method
RU2712732C2 (ru) Способ, система и считываемые компьютером носители для идентификации устройства для приема оградительного столбика
CN115195711A (zh) 用于超越车辆的车辆行为规划
US20230051632A1 (en) Systems and methods for an autonomous vehicle
CN117058920A (zh) 领头车辆到尾随车辆距离估计系统、方法及应用

Legal Events

Date Code Title Description
AS Assignment

Owner name: FORD GLOBAL TECHNOLOGIES, LLC, MICHIGAN

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:MICKS, ASHLEY ELIZABETH;BANVAIT, HARPREETSINGH;JAIN, JINESH J;AND OTHERS;SIGNING DATES FROM 20151201 TO 20151217;REEL/FRAME:037326/0823

STCV Information on status: appeal procedure

Free format text: ON APPEAL -- AWAITING DECISION BY THE BOARD OF APPEALS

STCV Information on status: appeal procedure

Free format text: BOARD OF APPEALS DECISION RENDERED

STPP Information on status: patent application and granting procedure in general

Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION

STPP Information on status: patent application and granting procedure in general

Free format text: NON FINAL ACTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER

STPP Information on status: patent application and granting procedure in general

Free format text: NON FINAL ACTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS

STPP Information on status: patent application and granting procedure in general

Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT VERIFIED

STCF Information on status: patent grant

Free format text: PATENTED CASE

MAFP Maintenance fee payment

Free format text: PAYMENT OF MAINTENANCE FEE, 4TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1551); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

Year of fee payment: 4