EP4308430A1 - Guidage à distance pour véhicules autonomes - Google Patents

Guidage à distance pour véhicules autonomes

Info

Publication number
EP4308430A1
EP4308430A1 EP22772009.1A EP22772009A EP4308430A1 EP 4308430 A1 EP4308430 A1 EP 4308430A1 EP 22772009 A EP22772009 A EP 22772009A EP 4308430 A1 EP4308430 A1 EP 4308430A1
Authority
EP
European Patent Office
Prior art keywords
intersection
traffic signal
guidance
autonomous vehicle
remote operator
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
EP22772009.1A
Other languages
German (de)
English (en)
Inventor
Ruben Zhao
Andrew Thomas Hartnett
Edward Stephen VENATOR
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Argo AI LLC
Original Assignee
Argo AI LLC
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Argo AI LLC filed Critical Argo AI LLC
Publication of EP4308430A1 publication Critical patent/EP4308430A1/fr
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/0011—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots associated with a remote control arrangement
    • G05D1/0038—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots associated with a remote control arrangement by providing the operator with simple or augmented images from one or more cameras located onboard the vehicle, e.g. tele-operation
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W30/00—Purposes of road vehicle drive control systems not related to the control of a particular sub-unit, e.g. of systems using conjoint control of vehicle sub-units
    • B60W30/18—Propelling the vehicle
    • B60W30/18009—Propelling the vehicle related to particular drive situations
    • B60W30/18154—Approaching an intersection
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W30/00—Purposes of road vehicle drive control systems not related to the control of a particular sub-unit, e.g. of systems using conjoint control of vehicle sub-units
    • B60W30/18—Propelling the vehicle
    • B60W30/18009—Propelling the vehicle related to particular drive situations
    • B60W30/18159—Traversing an intersection
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W60/00—Drive control systems specially adapted for autonomous road vehicles
    • B60W60/001—Planning or execution of driving tasks
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W60/00—Drive control systems specially adapted for autonomous road vehicles
    • B60W60/001—Planning or execution of driving tasks
    • B60W60/0027—Planning or execution of driving tasks using trajectory prediction for other traffic participants
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/09—Arrangements for giving variable traffic instructions
    • G08G1/0962—Arrangements for giving variable traffic instructions having an indicator mounted inside the vehicle, e.g. giving voice messages
    • G08G1/09623—Systems involving the acquisition of information from passive traffic signs by means mounted on the vehicle
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2420/00—Indexing codes relating to the type of sensors based on the principle of their operation
    • B60W2420/40—Photo, light or radio wave sensitive means, e.g. infrared sensors
    • B60W2420/403—Image sensing, e.g. optical camera
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2556/00—Input parameters relating to data
    • B60W2556/45—External transmission of data to or from the vehicle

Definitions

  • An autonomous vehicle (AV) traveling on public roads must be able to stop or proceed as appropriate when encountering traffic signals on the roads (for example, at in tersections).
  • the AV may rely on computer vision to determine the active state of the traf fic signal, but computer vision sometimes fails to accurately read the state of the traffic signal. In these cases, where the state of the traffic signal is unknown, the AV may not know how to proceed at the signal.
  • FIG. 1 depicts a schematic illustration of an example system, in accordance with one or more example embodiments of the disclosure.
  • FIGs. 2A-2B depict an example use case, in accordance with one or more example embodiments of the disclosure.
  • FIGs. 3 A-3B depict an example use case, in accordance with one or more example embodiments of the disclosure.
  • FIG. 4 depicts an example flowchart, in accordance with one or more example embodiments of the disclosure.
  • FIG. 5 depicts an example flowchart, in accordance with one or more example embodiments of the disclosure.
  • FIG. 6 depicts an example method, in accordance with one or more example em bodiments of the disclosure.
  • FIG. 7 depicts an example method, in accordance with one or more example em bodiments of the disclosure.
  • FIG. 8 depicts a schematic illustration of an example computing device architec ture, in accordance with one or more example embodiments of the disclosure.
  • This disclosure generally relates to, among other things, remote guidance for au tonomous vehicles (AVs).
  • AVs au tonomous vehicles
  • the systems and methods describe herein may re late to situations where an AV is unable to determine a state of a traffic signal (the terms “traffic signal” and “traffic signal” may be used interchangeably herein) on a road. How ever, the systems and methods may also be applied to any other situation where an AV requires guidance from a remote operator station as well.
  • the state of a traffic signal may refer to a color that the traffic signal is illuminating. For example, in some geographic lo cations, the traffic signal may illuminate a red color to indicate that vehicles should stop at the intersection or a green light to indicate that vehicles may proceed through the inter section.
  • An AV traveling on a road must be able to stop or proceed as appropriate when encountering traffic signals on the roads (for example, at an intersection on the road).
  • the AV may rely on computer vision to determine the active state of the traffic signal, but computer vision sometimes fails to accurately detect the state of the traffic signal.
  • the AV may not know whether to stop at the intersection or proceed through the intersection.
  • the AV may initially treat an unknown traffic signal as if the traffic signal is in a red light state. That is, if the AV reaches the intersection, and the state of the traffic signal is still unknown to the AV, then the AV may stop before the intersection and not proceed through the intersection.
  • the AV may maintain this stopped position indefinitely until ei ther a remote operator station provides guidance to the AV or the sensors of the AV are eventually able to determine the current state of the traffic signal. Additionally, while stopped, the AV may activate its hazard indicators to signal to other vehicles that it is stopped, or may provide this indication to other vehicles on the road through any other mechanism as well (for example, vehicle-to-vehicle (V2V) communications).
  • V2V vehicle-to-vehicle
  • the AV when it is unable to determine the state of a traffic signal, it may send a request for guidance to a remote operator station.
  • the remote opera tor station may be located remotely from the AV and the intersection, but the remote op erator station may also be integrated in the AV itself or a mobile device (for example, smartphone or other type of device) of a passenger of the AV as well.
  • a remote operator may, using the remote operator station, determine the state of the unknown traffic signal and may send an appropriate guidance to the AV.
  • guidance may refer to a message that may be provided from the remote operator station to the AV.
  • the message may include an indication of an action for the AV to take.
  • One example action may be to treat the intersection as an all-way stop.
  • An other example action may be to treat the intersection as a partial-way stop.
  • a third exam ple action may be to remain halted at the intersection.
  • the AV may be navigating a route through a road network including a number of intersections.
  • the AV may have stored in local memory (or at a remote server) a pre-established map of the road network, which may specify, among other things, the presence of traffic signals at the intersections.
  • the AV may perform one or more checks. First, the AV may determine whether the upcoming traffic signal is in a field of view of its sensors (for example, cameras, LI- DAR, RADAR, etc.). Second, if the traffic signal is in the field of view, the AV may de termine whether it is able to identify the state of the traffic signal.
  • the AV may capture one or more images and/or video feeds of the intersection and may use com puter vision to determine the state of the traffic signal (however, any other sensors may be used to identify the state of the traffic signal as well). If the traffic signal is within the field of view, but not detected, the state of the signal may be classified as unknown. Like wise, the traffic signal may be in the field of view, and the traffic signal itself may be identified, but a state of the traffic signal may be unknown. When there is an upcoming unknown traffic signal along the AV’s route, the AV may then send a request for guid ance to the remote operator station over a wireless or wired Internet connection.
  • This re quest for guidance may include any information that may assist the remote operator that is managing the remote operator station in providing an instruction to the AV.
  • the information may indicate the reason for the request (for example, an unknown traffic signal), the type of the upcoming intersection (for example, traffic signal), a distance the AV’s is from the intersection, an identifier associated with the traffic signal, which can be used by the remote operator to identify the location of the traffic signal on a map (which, in some instances, may be the same map that is used by the AV to perform navigation), a current location of the AV, as well as any other information that may be used by the re mote operator station to provide guidance to the AV.
  • the remote operator station when the remote operator station receives a request for guidance from the AV, the remote operator may be presented with any of the information received in the request, as well as any other relevant information.
  • the re mote operator station may include a user interface that may be used to display one or more images and/or live video feeds from one or more cameras from the AV, a map of the location of the AV, and/or any other relevant data.
  • the interface may also present any other types of information that may be relevant to the remote operator, including infor mation not received from the AV.
  • the interface may present a visual map that may be used to track the locations of one or more AVs in communication with the remote operator station (for example, the remote operator station may be in communica tion with a fleet of AVs).
  • the remote operator may determine the true state of the traffic signal using any of this information (as well as any other information not listed above). That is, the remote operator may view the one or more images and/or video feeds and may identify that the traffic signal is illuminating a particular color. The remote operator may then provide guidance to the AV based on the color of the traffic signal. This may be accomplished by the remote operator providing an input to the remote operator station, where the remote operator station subsequently provides guidance to the AV based on the input from the remote operator.
  • the operator may wait for the light state to change before sending the guidance (or may send the guid ance for the AV to remain halted at the intersection). If the traffic signal is solid yellow, the operator may wait for the light state to change before sending the guidance (or may send guidance for the AV to remain halted at the intersection or proceed through the in tersection depending on the length of time the yellow light has been illuminated). If the traffic signal is solid green or flashing yellow, the operator may provide guidance to the AV to proceed as if the intersection is an all-way stop.
  • the operator may provide guidance to the AV to proceed as if the intersection is a partial way stop (“all-way stop” and “partial-way stop” may be described in more detail below).
  • the AV may deactivate its hazard indicators and may proceed based on the guidance received from the remote operator station. It should be noted that although examples of guidance that may be provided when the AV is presented with certain traffic signal colors are provided herein, these are merely exemplary. For ex ample, the remote operator station may provide guidance for the AV to proceed as if the intersection is a partial-way stop when the traffic signal is green as well.
  • the remote operator may make determinations based on the information presented by the remote operator station
  • the remote op erator station itself may automatically make the same determinations made by the remote operator. That is, in some cases, the remote operator may not be necessary, and guidance may be provided through the use of only the remote operator station.
  • the remote operator station may include artificial intelligence, or the like, to al low the remote operator station to analyze the images and/or video feeds provided by the AV, as well as any other relevant information.
  • a remote operator may be able to indicate to the remote op erator station to provide a number of different types of guidance to the AV.
  • a first exam ple of a type of guidance may include an indication for the AV to proceed as if the inter section is an all-way stop (for example, the AV should proceed as if the intersection in cludes stop signs or flashing red lights at each of the roads in the intersection).
  • a second example of a type of guidance may include an indication for the AV to proceed as if the intersection is a partial-way stop (for example, the AV should proceed as if the road the AV is on includes a stop sign or a red light and the roads in the intersection that are per pendicular to the road that the AV is on have no stop sign or red light).
  • the all-way stop and partial-way stop scenarios may be further illustrated through the use cases depicted in FIGs. 2-3.
  • the AV may adjust how it approaches the intersection and/or predicts behaviors of other actors in the inter section, such as other vehicles.
  • the AV may adjust how it predicts when other ve hicles at the intersection will be traversing through the intersection or stopping in front of the intersection.
  • the remote guidance provided by the remote operator station to the AV can authorize an AV to perform certain maneuvers, but may not instruct the AV on how or when to perform the maneuvers.
  • Remote guidance may also not override the percep tion stack of the AV. That is, once the AV receives guidance from the remote operator, it may proceed or remain halted, but may proceed using its own sensors and internal pro cessing.
  • the AV may then use its own processing to identify when it may proceed into the intersection, and also to continuously monitor the environment while proceeding through the intersection.
  • another vehicle at the intersection may illegally drive through a red light and into the intersection.
  • the AV may still monitor for scenarios like this and adjust its movement correspondingly. This is just one non-limiting example of how the AV may still rely on internal processing even after being instructed to proceed.
  • the AV when provided with guidance to proceed as if the intersec tion is a partial-way stop, the AV may assume that cross-traffic in the intersection has right-of-way. In this mode of operation, when predicting the behavior of other vehicles at the intersection, the AV may assume that every cross-traffic vehicle may proceed through the intersection without stopping. The AV may assume this by default unless there is evi dence to the contrary that a specific vehicle in the intersection is yielding to the AV. For example, if a cross-traffic vehicle stops at the inlet to the intersection for a sufficiently long time, the AV may update its belief about that vehicle to assume that it is yielding right-of-way to the AV.
  • the AV may only proceed into the intersection when there are no non-yielding cross-traffic vehicles approaching the intersection.
  • the reason the remote operator may provide guidance to the AV to proceed as if the intersection is a partial-way stop when the traffic signal is flashing red is because neither the remote operator nor the AV can determine the state of the traffic signals that control the cross-traffic. If the cross- traffic has a traffic signal that is flashing yellow, the cross-traffic has right-of-way over the AV. If the cross-traffic also has a traffic signal that is flashing red, then the intersec tion is properly an all-way stop. Because the AV is operating under uncertainty, the safest action is the most conservative one, which is to treat the intersection as a partial-way stop.
  • the AV when provided with guidance to proceed as if the intersec tion is an all-way stop, the AV may assume that all vehicles approaching the intersection will yield mutually, and that right of way alternates around the intersection, such that ve hicles take turns proceeding.
  • the AV may assume that other vehicles at the intersection will use this right-of-way sharing behavior as a prior assumption and assume that the ve hicle whose turn it is will proceed through the intersection unless there is evidence that the vehicle is yielding.
  • the AV may assume other vehicles stopping at cross- traffic inlets do not have right of way and are yielding, whereas vehicles approaching the intersection and not slowing are assumed not to be stopping.
  • the AV may proceed on its own turn, assuming it is safe to do so and no other vehicles are observed to be proceeding out-of-turn.
  • the reason the remote operator provides guidance to the AV to proceed as if the intersection is an all-way stop when the traffic signal is green is to avoid unsafe situa tions caused by remote operator error or poor network communication between the AV and remote operator station. If the remote operator were allowed to command the AV to proceed as if it has right-of-way, the remote operator could accidentally cause the AV to proceed without caution through a solid red traffic signal. Treating the intersection as an all-way stop causes the AV to proceed with more caution than it would through a solid green traffic signal.
  • the AV’s perception of the traffic signal improves, the AV may become confident that it has detected the state of the traffic signal. If the AV confidently detects the traffic signal’s state before a remote operator station connects to the AV and sends a command, the AV may cancel the request for guidance and proceed as normal, acting on the traffic signal as the AV perceives it. If the traffic signal’s state is confidently detected after the AV receives a command from the remote operator station, the AV may act on the traffic signal as-perceived, and the com mand from the remote operator station may be disregarded.
  • FIG. 1 illustrates an example system 100, in accordance with one or more embodiments of this disclosure.
  • the system 100 may include one or more vehicles (for example, vehicle 112, vehicle 114, vehicle 116, and/or any other vehicle).
  • the system 100 may also include one or more remote operator stations 150.
  • any of the one or more vehicles and the one or more re mote operator stations 150 may communicate over a communications network 108.
  • the one or more vehicles may be autonomous vehicles (AVs).
  • the one or more vehicles may navigate one or more routes through a road net work including a number of intersections (for example, intersection 102, as well as any other number of intersections).
  • the one or more vehicles may have stored in local memory 135 (or at a remote server) a pre-established map of the road network, which may specify, among other things, the presence of traffic signals (for example, traffic sig nal 104 and/or traffic signal 106) at the intersections.
  • traffic signals for example, traffic sig nal 104 and/or traffic signal 106
  • the ve hicle may perform one or more checks.
  • the vehicle may determine whether the up coming traffic signal is in the field of view of its sensors (for example, cameras, LIDAR, RADAR, etc.). Second, if the traffic signal is in the field of view, the vehicle may deter mine whether it is able to identify the state of the traffic signal using one or more sen sors) 130. For example, the vehicle may capture one or more images and/or video feeds of the intersection and may use computer vision to determine the state of the traffic sig nal. The vehicle may also use any other sensors 130 (or multiple different types of sen sors) to determine the state of the traffic signal as well. If the traffic signal is within the field of view, but not detected, the state of the signal may be classified unknown.
  • sensors for example, cameras, LIDAR, RADAR, etc.
  • the traffic signal may be in the field of view, and the traffic signal itself may be identified, but a state of the traffic signal may be unknown.
  • the vehicle may then send a request for guidance to the remote operator station 150 over a communications network 108.
  • This re quest for guidance may include any information that may assist the remote operator in providing an instruction to the vehicle.
  • the information may indicate the reason for the request (for example, an unknown traffic signal), the type of the upcoming intersection (for example, traffic signal), a distance the vehicle is from the intersection, an identifier associated with the traffic signal, which can be used by the remote operator to identify the location of the traffic signal on a map (which, in some instances, may be the same map that is used by the vehicle to perform navigation).
  • the vehicle may also pro vide any other relevant information to the remote operator station 150. This information may be provided to the remote operator station 150 over the communications network 108 through a communications interface 140 (which may be the same as network inter face 712 depicted in FIG. 7).
  • the vehicle may also include any other elements (for exam ple, any elements depicted in the computing device 700 illustrated in FIG. 7 or other wise).
  • a remote operator station 150 may include at least one or more display interface(s) 154, memory 152, and one or more communication interface(s) 156.
  • the remote operator station 150 may receive a request for assistance from the vehicle through the communications interface(s) 156 (which may be the same as network interface 712 depicted in FIG. 7), the remote operator 158 may be presented with any of the information received in the request, as well as any other rele vant information.
  • the display interface(s) 154 that may be used to present one or more images and/or live video feeds from one or more cameras from the vehicle, a map of the location of the vehicle, and/or any other relevant data.
  • the remote operator 158 may determine the true state of the traffic signal using any of this information. That is, the remote operator 158 may view the one or more images and/or video feeds and may identify that the traffic signal is illuminating a particular color. The remote operator 158 may then provide guidance to the vehicle based on the color of the traffic signal. This may be accomplished by the remote operator 158 providing an input to the remote opera tor station 152, where the remote operator station 150 may subsequently provide guidance to the vehicle based on the input from the remote operator 158. For example, if the traffic signal is solid red, the operator 158 may wait for the light state to change before sending the guidance (or may send guidance for the vehicle to remain halted at the intersection).
  • the operator 158 may wait for the light state to change before sending guidance (or may send guidance and for the vehicle to remain halted at the intersection or proceed through the intersection depending on the length of time the yel low light has been illuminated). If the traffic signal is solid green or flashing yellow, the operator 158 may provide guidance to the vehicle to proceed as if the intersection is an all-way stop. If the traffic signal is flashing red, the operator 158 may provide guidance to the vehicle to proceed as if the intersection is a partial-way stop. When the vehicle re ceives the guidance, the vehicle may deactivate its hazard indicators and may proceed based on the action command received from the remote operator station.
  • the remote op erator station 150 may also include any other elements (for example, any elements de picted in the computing device 700 illustrated in FIG. 7 or otherwise). It should be noted that although examples of guidance that may be provided when the AV is presented with certain traffic signal colors are provided herein, these are merely exemplary. For example, the remote operator station may provide guidance for the AV to proceed as if the intersec tion is a partial-way stop when the traffic signal is green as well.
  • the communications network 108 may include any one or a combination of multiple different types of networks, such as cable networks, the Internet, wireless networks, and other private and/or public networks.
  • the com munications network 108 may include cellular, Wi-Fi, or Wi-Fi direct.
  • the network may involve communications between vehicles in the network and/or between vehicles in the network and elements external to the network. For example, Ve- hi cl e-to- Vehicle (V2V), Vehicle-to-Infrastructure (V2I), Vehicle-to-Everything (V2X), and/or Dedicated Short Range Communications (DSRC), to name a few, may be used.
  • V2V Ve- hi cl e-to- Vehicle
  • V2I Vehicle-to-Infrastructure
  • V2X Vehicle-to-Everything
  • DSRC Dedicated Short Range Communications
  • FIGs. 2A-2B depict an example use case including a first scene 200 and a second scene 250.
  • the use case may depict a first vehicle 216 and a second vehicle 212.
  • the use case may be described from the perspective of the first vehicle 216. That is, the first vehicle 216 may be an autonomous vehicle (AV) that may require guidance from a remote operator station 230 as described herein.
  • the first vehicle 216 may be approaching an intersection 202 that may include one or more traffic signals (for example, traffic signal 206 and traffic signal 208).
  • traffic signals for example, traffic signal 206 and traffic signal 208
  • the traf fic signal 206 may currently be displaying a flashing red color
  • the traffic signal 208 may currently be displaying a flashing yellow color.
  • the first vehicle 216 may need to know a state of the traffic signal 206. For example, the first vehicle 216 may need to know the color of the traffic signal 206 (which may be red in this case).
  • the first vehicle 216 may include one or more cameras that may be used to capture one or more images and/or video feeds of the traffic signal 206 and the intersection 202 as a whole. Under normal operation, the first vehicle 216 may analyze the one or more images and/or video feeds of the traffic signal 206 to determine the current color of the traffic signal. Based on the color of the traffic signal, the first vehicle 216 may determine whether to proceed through the intersection 202 or to stop in front of the intersection 202.
  • the first vehicle 216 may be unable to determine through analy sis of captured images and/or video feed the color of the traffic signal 206.
  • the first vehi cle 216 may then classify the state of the traffic signal 206 as unknown. Based on the state of the traffic signal 206 being set to unknown, the first vehicle 216 may stop at the intersection 202, and may send a request to a remote operator station 230 for guidance.
  • the remote operator station 230 may present any of the information received in the request, as well as any other relevant information.
  • the remote operator station 230 may include an interface that may be used to present one or more images and/or live video feeds from one or more cameras from the first vehicle 216, a map of the location of the first vehicle 216, and/or any other relevant data.
  • This information may be presented to a remote operator 232 who may be responsible for monitoring the remote operator station 230.
  • the remote operator 232 may determine the true state of the traffic signal using any of this information. That is, the remote operator 232 may view the one or more images and/or video feeds and may identify that the traffic signal 206 is flashing red and the traffic signal 208 is flashing yel low. The remote operator 232 may then provide guidance to the first vehicle 216.
  • the remote operator 232 may provide guidance to the first vehicle 216 to treat the intersection as a partial-way stop given that the traffic signals are displaying flashing lights.
  • the first vehicle 216 may then wait for all cross-traffic to exit the intersec tion 202 before proceeding through the intersection. For example, as shown in scene 250, the first vehicle 216 may wait for the vehicle 212 to proceed through the intersection 212 before the first vehicle 216 itself proceeds through the intersection.
  • FIGs. 3A-3B depict an example use case including a first scene 300 and a second scene 350.
  • the use case may depict a first vehicle 316 and a second vehicle 312.
  • the use case may be described from the perspective of the first vehicle 316 That is, the first vehicle 316 may be an autonomous vehicle (AV) that may require the assistance of a remote operator station 330 as described herein.
  • the first vehi cle 316 may be approaching an intersection 302 that may include one or more traffic sig nals (for example, traffic signal 306 and traffic signal 308).
  • the traffic signal 306 may currently be displaying a green color
  • the traffic signal 308 may currently be displaying a red color.
  • the first vehicle 316 may need to know a state of the traffic signal 306. For example, the first vehicle 316 may need to know the color of the traffic signal 306 (which may be red in this case).
  • the first vehicle 316 may include one or more cameras that may be used to capture one or more images and/or video feeds of the traffic signal 306 and the intersection 302 as a whole. Under normal operation, the vehicle 316 may analyze the one or more images and/or video feeds of the traffic signal 306 to determine a current color of the traffic signal. Based on the color of the traffic signal, the first vehicle 316 may determine whether to proceed through the in tersection 302 or to stop in front of the intersection 302.
  • the first vehicle 316 may be unable to determine through analy sis of captured images and/or video feed the color of the traffic signal 306.
  • the first vehi cle 316 may then classify the state of the traffic signal 306 as unknown. Based on the state of the traffic signal 306 being set to unknown, the first vehicle 316 may stop at the intersection 302, and may send a request to a remote operator station 330 for instruction.
  • the remote operator station 330 may present any of the information received in the request, as well as any other relevant information.
  • the remote operator station 330 may include an interface that may be used to present one or more images and/or live video feeds from one or more cameras from the first vehicle 316, a map of the location of the first vehicle 316, and/or any other relevant data.
  • This information may be presented to a remote operator 332 who may be responsible for monitoring the remote operator station 330.
  • the remote operator 332 may determine the true state of the traffic signal using any of this information. That is, the remote operator 332 may view the one or more images and/or video feeds and may identify that the traffic signal 306 is green and the traffic signal 308 is red. The remote operator 332 may then provide guidance to the first vehicle 316.
  • the remote operator 332 may provide the guidance to the first vehicle 316 to treat the in tersection as an stop given that the traffic signals are displaying flashing lights.
  • the first vehicle 316 may then wait for all cross-traffic to exit the intersection 302 before proceed ing through the intersection.
  • the first vehicle 316 may wait for the second vehicle 312 to proceed through the intersection 302 before the first vehicle 316 itself proceeds through the intersection.
  • the remote operator station may provide guidance for the AV to proceed as if the intersection is a partial-way stop when the traffic signal is green as well.
  • the remote operator may make determinations based on the infor mation presented by the remote operator station
  • the remote operator sta tion itself may automatically make the same determinations made by the remote operator. That is, in some cases, the remote operator may not be necessary, and guidance may be provided through the use of only the remote operator station.
  • the remote operator station may include artificial intelligence, machine learning, or the like to allow the remote operator station to analyze the images and/or video feeds pro vided by the AV, as well as any other relevant information.
  • FIG. 4 depicts an example flowchart 400.
  • the flowchart 400 may illustrate exam ple high level operations performed by an AV that may encounter a traffic signal that is an unknown state to the AV (for example, the AV may be unable to determine a color the traffic signal as described herein).
  • the flowchart 400 may begin at condi tion 414.
  • Condition 414 may involve a determination as to whether the AV has cleared (for example, traversed through) an intersection at its current location.
  • the flowchart 400 may illustrate a scenario starting from a point when the AV is already at an intersection.
  • the flowchart 400 illustrates con dition 414 as being the starting point
  • the operations described with respect to the flowchart 400 may similarly begin before the AV encounters an intersection as well. That is, the flowchart 400 may be modified such that the beginning point may be the condition 404.
  • the flowchart 400 may pro ceed to operation 402. If it is determined that the AV has not cleared the intersection, then the flowchart may proceed to operation 412. Operation 402 may involve the AV being in communication with a remote operator station, but has not yet received any guidance from the remote operator station.
  • Operation 412 may involve the AV hav ing received guidance from the remote operator station to proceed as if the intersection is an all-way stop or a partial-way stop. Thus, the AV may continue traversing through the intersection unless the condition 410 is met.
  • Condition 410 may involve a determination as to whether guidance has been provided from the remote operator station to the AV in dicating that the AV should disregard prior guidance for the AV to proceed through the intersection.
  • the flowchart 400 may continue to loop back to condition 412, and then AV may continue to traverse the intersection until it ultimately clears the intersection. However, if the guidance to disre gard prior guidance is received, then the flowchart may proceed to operation 406.
  • Condition 404 may involve a determination as to whether the AV is unable to identify the state of a current color of a traffic signal.
  • the AV may be ap proaching an intersection (which may be a subsequent intersection along the AV’s navi gation route beyond the initial intersection that was cleared) including one or more traffic signals, and may be unable to determine if the color of the traffic signal is red, yellow, or green (or any other color that may be used depending on the geographic location of the traffic signal).
  • the AV may make such determinations based on im ages and/or video feeds captured by one or more cameras included on the AV.
  • the AV may halt movement and may send a request to a remote operator station for further guidance. In this case, the flowchart 400 may proceed to operation 406. However, if the AV is able to determine the color of the traffic signal, then the AV may proceed through the intersection without the assistance of the remote operator station. In such a case, the flowchart 400 to operation 402.
  • the condition 404 may be continuously monitored as the AV is navigating an environment. For example, the AV may determine at a first time that it is able to identify the color of the traffic signal, but at a second time may be unable to determine the color of the traffic signal.
  • the AV may proceed to operation 406 at any time the vehicle is unable to determine the status of the traffic signal. This may, how ever, depend on a number of factors, such as the current location of the AV. For example, if the AV detects that the traffic signal is green and proceeds through the intersection, but is unable to determine the color of the traffic signal while already in the intersection, then the AV may simply proceed through the intersection so as to prevent the AV from halting in the middle of an intersection to request guidance from a remote operator station.
  • An other example factor may include the amount of time the color of the traffic signal re mains unknown.
  • the color of the traffic signal may only be unknown to the AV for a short duration of time, and after this short duration of time, the traffic signal color may again become known to the AV.
  • a threshold amount of time may be used. That is, the AV may be required to wait a threshold amount of time without being able to detect the color of the traffic signal before halting movement and requesting guidance from the remote operator station.
  • the remote operator station may be used to provide instructions to the AV when the AV may be unsure how to proceed through a given intersection.
  • the flowchart 400 may proceed to condition 408.
  • Condition 408 may in volve a determination as to whether a message including the guidance was transmitted back to the AV from the remote operator station.
  • the remote operator may be able to pro vide several different types guidance to the AV.
  • a first example type of guidance may in clude an indication for the AV to proceed as if the intersection is an all-way stop (for ex ample, the AV should proceed as if the intersection includes stop signs or flashing red lights at each of the roads in the intersection).
  • a second example type of guidance may include an indication for the AV to proceed as if the intersection is a partial-way stop (for example, the AV should proceed as if the road the AV is on includes a stop sign or a red light and the roads in the intersection that are perpendicular to the road that the AV is on have no stop sign or red light).
  • the AV may adjust how it approaches the intersection and/or predicts behaviors of other actors in the intersection, such as other vehicles. That is, the AV may adjust how it pre dicts when other vehicles at the intersection will be traversing through the intersection or stopping in front of the intersection.
  • the remote guidance provided by the remote opera tor to the AV can authorize the AV to perform certain maneuvers, but may not instruct the AV on how or when to perform the maneuvers. Remote guidance may also not over- ride the perception stack of the AV. That is, once the AV receives guidance from the re mote operator, it may proceed or remain halted, but may proceed using its own sensors and internal processing. For example, if the AV received guidance to proceed as if the in tersection were a partial-way stop, then the AV may then use its own processing to iden tify when it may proceed into the intersection, and also to continuously monitor the envi ronment while proceeding through the intersection.
  • the flowchart 400 may loop through the condition 408 until it is met. Once the condition 408 is met, the flowchart may either proceed back to operation 406 or may proceed to operation 412. The flowchart 400 may proceed back to operation 406 if the guidance provided by the remote operator station indicates that the AV should remain halted (for example, if the remote operator at the remote operator station determines that the color of the traffic signal is currently red or yellow).
  • FIG. 5. depicts an example flowchart 500.
  • the example flowchart 500 may in volve similar operations as the flowchart 400, but may be a more detailed illustration than the high-level operations illustrated in the flowchart 400.
  • the flowchart 500 may begin with operation 502, which may involve an AV approaching an intersection that may in clude one or more traffic signals.
  • the AV may determine it is approaching an intersection based on a map stored in local memory or at a remote location (for exam ple, the AV may determine its current location with respect to one or more intersections on the map).
  • the AV may determine it is approaching the intersection us ing information from one or more sensors on the AV (for example, the AV may use a camera to capture an image or video feed of an external environment to identify any up coming intersections. In other cases, the AV may determine that is approaching an inter section using any other suitable methods. Additionally, a remote system may also provide an indication to the AV that it is approaching an intersection rather than the AV making the determination itself. From operation 502, the flowchart 500 may proceed to condition 504.
  • Condition 504 may involve a determination as to whether the AV is able to identify a color of a traffic signal at the intersection. Condition 504 may be similar to conditions 510, 522, and 542 described below.
  • This condition may also be continuously looped throughout any of the operations being described with respect to the flowchart 500. That is, although the flowchart 500 depicts the condition as being included in specific locations in the flowchart 500, the AV may continuously determine whether it is able to discern the color of the traffic signal at any point as well. If it is determined in condition 504 that the AV is able to identify the color of the traffic signal, then the flowchart 500 may proceed to operation 548. However, if the AV is not able to identify the color of the traffic signal, then the flowchart 500 may proceed to operation 506. At operation 506, the AV may send a request for guidance to a remote operator station. The request may include any images and/or video feed captured by the AV.
  • the AV may transmit any images and/or video feed captured of the traffic signal at the intersection.
  • the request may also include any other information that may be relevant to the remote operator station, such as a current location of the AV, a speed of the AV, and/or any other information relating to the AV itself, the environment external to the AV, or any other information.
  • the flowchart 500 may then proceed to operation 508, which may involve the AV stopping at the intersection. That is, if the AV is unable to determine the color of a traffic signal at the intersection, the AV may halt movement instead of proceeding through the intersection, and may send the request for guidance to a remote operator station as indicated in opera tion 506.
  • operations 506 and/or 508 may occur in any order, and may also occur simultaneously.
  • the AV may continuously loop condition 510.
  • Condition 510 may involve determining if the color of the traffic signal is known by the AV. For example, the AV may determine at a first time that it is able to identify the color of the traffic signal, but at a second time may be unable to determine the color of the traf fic signal. In some cases, condition 510 may only be checked once, or a finite number of times as well (for example, instead of being continuously looped). If at any time the AV is able to identify the color of the traffic signal, the flowchart 500 may proceed to opera tion 544.
  • the AV may send a subsequent message to the remote opera tor station indicating that the AV no longer requires instructions. If the AV has already received instructions from the remote operator station, the AV may also disregard the in structions and proceed using the local processing of the AV. From operation 544, the flowchart 500 may proceed to operation 546 and operation 548, in which the AV may de activate its hazard lights and proceed through the intersection.
  • the flowchart 500 may proceed to operation 512.
  • the AV may halt movement at the intersection.
  • the AV may also activate its hazard lights (or provide any other type of indication to other vehicles at the intersection that the AV is stopped).
  • the information pro vided to the remote operator station may be presented on an interface of the remote opera tor station.
  • a remote operator managing the remote operator station may then examine the information presented on the interface of the remote operator station.
  • the remote operator may pro vide an input to the remote operator station to provide guidance to the AV to remain halted at the intersection.
  • the remote operator may simply refrain from providing an input to the remote operator station, as the AV may remain halted at the in tersection until either guidance is received from the remote operator station or the AV is able to subsequently determine the color of the traffic signal. If it is determined that the color of the traffic signal is not red, then, at operation 520, the remote operator may pro vide an input to the remote operator station, which may trigger the remote operator station to send guidance to the vehicle.
  • the flowchart 500 may proceed to condition 522, which may involve a determination as to whether the color of the traffic signal at the intersection is unknown to the AV. If the condition 522 is met (the AV is able to determine the color of the traffic signal), then the flowchart may proceed to opera tion 540. In operation 540, the AV may ignore guidance provided by the remote operator station, and may instead proceed to operation 544 and beyond. However, if the condition 522 is not met (the AV is still unable to determine the color of the traffic signal), then the flowchart 500 may proceed to condition 524. Condition 524 may involve a determination as to what type of guidance was provided by the remote operator station.
  • the condition 524 may involve a determination as to whether the remote operator station indicated that the light is green or unlit. If the condition 524 is met, then the flowchart 500 may proceed to operation 526. If the condition 528 is not met, then the flowchart 500 may proceed to operation 528. Operation 526 may involve the AV treating the intersection as an all-way stop. To this end, the remote operator station may provide guidance to the AV indicating that the AV should treat the intersection as an all way stop. Operation 528 may involve the remote operator providing an input to the re mote operator station indicating that the light is blinking red. From operation 528, the flowchart 500 may proceed to operation 530.
  • the AV may treat the in tersection as a partial-way stop.
  • the remote operator station may provide guidance to the AV indicating that the AV should treat the intersection as a partial-way stop.
  • guidance may be provided when the AV is presented with certain traffic signal colors are provided herein, these are merely exemplary.
  • the remote operator station may provide guidance for the AV to proceed as if the intersection is a partial-way stop when the traffic signal is green as well.
  • the remote operator may make deter minations based on the information presented by the remote operator station, in some cases, the remote operator station itself may automatically make the same determinations made by the remote operator.
  • the remote operator may not be nec essary, and guidance may be provided through the use of only the remote operator station.
  • the remote operator station may include artificial intelli gence, machine learning, or the like to allow the remote operator station to analyze the images and/or video feeds provided by the AV, as well as any other relevant information.
  • the flowchart may proceed to operation 532.
  • Operation 532 may involve updating the AV’s perception stack (for example, internal processing) to indicate that the AV may proceed through the inter section while yielding to other vehicles at the intersection based on the particular guid ance provided to the AV.
  • the flowchart 500 may proceed to condition 524.
  • Condition 534 may involve determining if there are any conflict movers in or at the intersection.
  • a conflict mover may be any other actor that may be moving in or at the intersection, such as another vehicle, a pedestrian, or any other moving actor. If it is determined that there are no conflict movers, then the flowchart 500 may proceed to oper ation 550.
  • the flowchart may pro ceed to condition 542.
  • the AV may proceed into the intersection, and then may ultimately clear the intersection at operation 552.
  • the AV may again determine if the color of the traffic signal is determined. If the color of the traffic signal is determined, then the flowchart may proceed to operation 544. If the color of the traffic signal is still not determined, then the flowchart 500 may proceed to operation 538.
  • the AV may wait for conflicting movers to leave the intersection (or may wait for a prediction that the conflict movers currently in the intersection will yield to the AV). From operation 538, the flowchart 500 may proceed to condition 536.
  • FIG. 6 is an example method 600.
  • the method may include capturing, at a first time and by a camera of an autonomous vehicle, at least one of: an image or video feed of a first traffic signal at an intersection.
  • the method 600 may include classifying, based on the image or the video feed of the first traffic signal, a state of a color of the first traffic signal as unknown.
  • the method 600 may include halting movement of the autonomous vehicle at the intersec tion based on classifying the state of the color of the first traffic signal as unknown.
  • the method 600 may include sending a request for guidance to a remote opera tor device, the request including the image or video feed of the first traffic signal.
  • the method 600 may include receiving, from the remote operator device, a first guidance.
  • the method 600 may include performing, based on the first guid ance, a first action including at least one of: remaining halted at the intersection or pro ceeding through the intersection.
  • the method 600 may also include capturing, at a second time and by the camera of an autonomous vehicle, at least one of: a second image or sec ond video feed of the first traffic signal.
  • the method 600 may also include classifying, by the autonomous vehicle and based on the second image or video feed, the state of the color of the first traffic signal as known.
  • the method 600 may also include performing, based on the state of the color of the first traffic signal being known, the first action.
  • the method 600 may also include capturing, at a second time and by the camera of an autonomous vehicle, a second image or video feed of the first traffic signal.
  • the method 600 may also include classifying, by the autonomous vehi cle and based on the second image or video feed, the state of the color of the first traffic signal as being known.
  • the method 600 may also include performing, based on the state of the color of the first traffic signal being known, a second action, the second action be ing different than the first action based on the first guidance.
  • the first guidance includes guidance for the autonomous vehicle to operate in a partial-way stop mode, and wherein operating in the partial-way stop mode involves the autonomous vehicle predicting that cross-traffic at the intersection will proceed without yielding to the autonomous vehicle.
  • the first guidance includes guidance for the autonomous vehicle to operate in an all-way stop mode, wherein operating in the all-way stop mode involves the autonomous vehicle pre dicting that cross-traffic at the intersection will yield to the autonomous vehicle.
  • the first guidance includes guidance for the vehicle to remain halted at the intersection.
  • performing the second action further comprises disre garding the first guidance received from the remote operator device and proceeding through the intersection.
  • FIG. 7 is an example method 700.
  • the method may include receiving, from an autonomous vehicle and by a remote operator de vice, a request for guidance, the request including at least one of: an image or video feed of a first traffic signal at an intersection captured by the autonomous vehicle, the request also indicating that the autonomous vehicle has classified a state of a color of the first traffic signal as unknown.
  • the method 700 may include receiving, by the remote operator device, an input selection of a first guidance to provide to the autono mous vehicle.
  • the method 700 may include providing, by the remote opera tor device and to the autonomous vehicle, the first guidance.
  • FIGs. 4- 7 may be carried out or performed in any suitable order as desired in various example em bodiments of the disclosure. Additionally, in certain example embodiments, at least a por tion of the operations may be carried out in parallel. Furthermore, in certain example em bodiments, less, more, or different operations than those depicted in FIGs. 4-7 may be performed.
  • FIG. 8 illustrates an example computing device 800, in accordance with one or more embodiments of this disclosure.
  • the computing 800 device may be representative of any number of elements described herein, such as any of the remote operator station(s), the vehicle(s), and/or any other element described herein.
  • the computing device 800 may include at least one processor 802 that executes instructions that are stored in one or more memory devices (referred to as memory 804).
  • the instructions can be, for instance, in structions for implementing functionality described as being carried out by one or more modules and systems disclosed above or instructions for implementing one or more of the methods disclosed above.
  • the processor(s) 802 can be embodied in, for example, a CPU, multiple CPUs, a GPU, multiple GPUs, a TPU, multiple TPUs, a multi-core processor, a combination thereof, and the like. In some embodiments, the processor(s) 802 can be ar ranged in a single processing device. In other embodiments, the processor(s) 802 can be distributed across two or more processing devices (e.g., multiple CPUs; multiple GPUs; a combination thereof; or the like). A processor can be implemented as a combination of processing circuitry or computing processing units (such as CPUs, GPUs, or a combina tion of both).
  • a processor can refer to a single-core processor; a single processor with software multithread execution capability; a multi-core processor; a multi-core processor with software multithread execution capability; a multi core processor with hardware multithread technology; a parallel processing (or compu ting) platform; and parallel computing platforms with distributed shared memory.
  • a processor can refer to an integrated circuit (IC), an ASIC, a digital signal processor (DSP), an FPGA, a PLC, a complex programmable logic device (CPLD), a discrete gate or transistor logic, discrete hardware components, or any combination thereof designed or otherwise configured (e.g., manufactured) to perform the functions described herein.
  • the processor(s) 802 can access the memory 804 by means of a communication architecture 806 (e.g., a system bus).
  • the communication architecture 806 may be suita ble for the particular arrangement (localized or distributed) and type of the processor(s) 802.
  • the communication architecture 806 can include one or many bus architectures, such as a memory bus or a memory controller; a peripheral bus; an ac celerated graphics port; a processor or local bus; a combination thereof, or the like.
  • such architectures can include an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MCA) bus, an Enhanced ISA (EISA) bus, a Video Electron ics Standards Association (VESA) local bus, an Accelerated Graphics Port (AGP) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express bus, a Personal Computer Memory Card International Association (PCMCIA) bus, a Universal Serial Bus (USB), and/or the like.
  • ISA Industry Standard Architecture
  • MCA Micro Channel Architecture
  • EISA Enhanced ISA
  • VESA Video Electron ics Standards Association
  • AGP Accelerated Graphics Port
  • PCI Peripheral Component Interconnect
  • PCMCIA Personal Computer Memory Card International Association
  • USB Universal Serial Bus
  • the memory components or memory devices can be removable or non-removable, and/or internal or external to a computing device or component.
  • Exam ples of various types of non-transitory storage media can include hard-disc drives, zip drives, CD-ROMs, digital versatile disks (DVDs) or other optical storage, magnetic cas settes, magnetic tape, magnetic disk storage or other magnetic storage devices, flash memory cards or other types of memory cards, cartridges, or any other non-transitory me dia suitable to retain the desired information and which can be accessed by a computing device.
  • non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory.
  • Volatile memory can include random access memory (RAM), which acts as external cache memory.
  • RAM is available in many forms such as synchronous RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), Synchlink DRAM (SLDRAM), and direct Rambus RAM (DRRAM).
  • SRAM synchronous RAM
  • DRAM dynamic RAM
  • SDRAM synchronous DRAM
  • DDR SDRAM double data rate SDRAM
  • ESDRAM enhanced SDRAM
  • SLDRAM Synchlink DRAM
  • DRRAM direct Rambus RAM
  • the disclosed memory devices or memories of the operational or computational environments described herein are intended to include one or more of these and/or any other suitable types of memory.
  • the memory 804 also can retain data.
  • Each computing device 800 also can include mass storage 808 that is accessible by the processor(s) 802 by means of the communication architecture 806.
  • the mass stor age 808 can include machine-accessible instructions (e.g., computer-readable instructions and/or computer-executable instructions).
  • the machine-accessible instructions may be encoded in the mass storage 808 and can be arranged in components that can be built (e.g., linked and compiled) and retained in computer-executable form in the mass storage 808 or in one or more other machine-accessible non-transitory storage media included in the computing device 800.
  • Such components can embody, or can con stitute, one or many of the various modules disclosed herein. Such modules are illustrated as modules 814.
  • modules may also be included within the memory 804 as well.
  • Execution of the modules 814, individually or in combination, by at least one of the processor(s) 802, can cause the computing device 800 to perform any of the opera tions described herein (for example, the operations described with respect to FIG. 4, as well as any other operations).
  • Each computing device 800 also can include one or more input/output interface devices 810 (referred to as I/O interface 810) that can permit or otherwise facilitate exter nal devices to communicate with the computing device 800.
  • I/O interface 810 input/output interface devices 810
  • the I/O inter face 810 may be used to receive and send data and/or instructions from and to an external computing device.
  • the computing device 800 also includes one or more network interface devices 812 (referred to as network interface(s) 812) that can permit or otherwise facilitate func tionally coupling the computing device 800 with one or more external devices.
  • Function ally coupling the computing device 800 to an external device can include establishing a wireline connection or a wireless connection between the computing device 800 and the external device.
  • the network interface devices 812 can include one or many antennas and a communication processing device that can permit wireless communication between the computing device 800 and another external device. For example, between a vehicle and a smart infrastructure system, between vehicles, etc. Such a communication processing de vice can process data according to defined protocols of one or several radio technologies.
  • the radio technologies can include, for example, 3G, Long Term Evolution (LTE), LTE- Advanced, 5G, IEEE 802.11, IEEE 802.16, Bluetooth, ZigBee, near-field communication (NFC), and the like.
  • the communication processing device can also process data accord ing to other protocols as well, such as vehicle-to-infrastructure (V2I) communications, vehi cl e-to- vehicle (V2V) communications, and the like.
  • V2I vehicle-to-infrastructure
  • V2V vehi cl e-to- vehicle
  • the network interface(s) 812 may also be used to facilitate peer-to-peer ad-hoc network connections as described herein.
  • the terms “environment,” “system,” “unit,” “module,” “architecture,” “interface,” “component,” and the like refer to a computer-related entity or an entity related to an operational apparatus with one or more defined functionalities.
  • the terms “environment,” “system,” “module,” “component,” “architecture,” “interface,” and “unit,” can be utilized interchangeably and can be generically referred to functional ele ments.
  • Such entities may be either hardware, a combination of hardware and software, software, or software in execution.
  • a module can be embodied in a pro cess running on a processor, a processor, an object, an executable portion of software, a thread of execution, a program, and/or a computing device.
  • both a software application executing on a computing device and the computing device can em body a module.
  • one or more modules may reside within a pro cess and/or thread of execution.
  • a module may be localized on one computing device or distributed between two or more computing devices.
  • a module can execute from various computer-readable non-transitory storage media having various data structures stored thereon. Modules can communicate via local and/or remote processes in accordance, for example, with a signal (either analogic or digital) having one or more data packets (e.g., data from one component interacting with another component in a local sys tem, distributed system, and/or across a network such as a wide area network with other systems via the signal).
  • a module can be embodied in or can include an apparatus with a defined functionality provided by mechanical parts operated by electric or elec tronic circuitry that is controlled by a software application or firmware application exe cuted by a processor.
  • a processor can be internal or external to the apparatus and can execute at least part of the software or firmware application.
  • a module can be embodied in or can include an apparatus that provides defined functional ity through electronic components without mechanical parts.
  • the electronic components can include a processor to execute software or firmware that permits or otherwise facili tates, at least in part, the functionality of the electronic components.
  • modules can communicate via local and/or remote pro Obs in accordance, for example, with a signal (either analog or digital) having one or more data packets (e.g., data from one component interacting with another component in a local system, distributed system, and/or across a network such as a wide area network with other systems via the signal).
  • modules can communicate or otherwise be coupled via thermal, mechanical, electrical, and/or electro mechanical coupling mechanisms (such as conduits, connectors, combinations thereof, or the like).
  • An interface can include input/output (I/O) components as well as associated processors, applications, and/or other programming components.
  • Conditional language such as, among others, “can,” “could,” “might,” or “may,” unless specifically stated otherwise, or otherwise understood within the context as used, is generally intended to convey that certain implementations could include, while other im plementations do not include, certain features, elements, and/or operations. Thus, such conditional language generally is not intended to imply that features, elements, and/or op erations are in any way required for one or more implementations or that one or more im plementations necessarily include logic for deciding, with or without user input or prompting, whether these features, elements, and/or operations are included or are to be performed in any particular implementation.

Landscapes

  • Engineering & Computer Science (AREA)
  • Automation & Control Theory (AREA)
  • Transportation (AREA)
  • Mechanical Engineering (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Aviation & Aerospace Engineering (AREA)
  • Traffic Control Systems (AREA)

Abstract

La présente invention concerne des systèmes, des procédés et des supports lisibles par ordinateur pour des systèmes et des procédés de guidage à distance pour véhicules autonomes. Un procédé à titre d'exemple peut consister à capturer, à un premier instant et par une caméra d'un véhicule autonome, au moins l'un parmi : une image ou un flux vidéo d'un premier feu de circulation à une intersection. Le procédé à titre d'exemple peut également consister à classifier, sur la base de l'image ou du flux vidéo du premier feu de circulation, un état d'une couleur du premier feu de circulation comme étant inconnu. Le procédé à titre d'exemple peut également consister à arrêter le déplacement du véhicule autonome au niveau de l'intersection sur la base de la classification de l'état de la couleur du premier feu de circulation comme étant inconnu. Le procédé à titre d'exemple peut également consister à envoyer une demande de guidage à un dispositif d'opérateur à distance, la demande comprenant l'image ou le flux vidéo du premier feu de circulation. Le procédé à titre d'exemple peut également consister à recevoir, à partir du dispositif d'opérateur à distance, un premier guidage. Le procédé à titre d'exemple peut également consister à effectuer, sur la base du premier guidage, une première action comprenant au moins l'une des opérations suivantes : rester arrêté à l'intersection ou traverser l'intersection.
EP22772009.1A 2021-03-17 2022-03-14 Guidage à distance pour véhicules autonomes Pending EP4308430A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US202117204066A 2021-03-17 2021-03-17
PCT/US2022/020231 WO2022197628A1 (fr) 2021-03-17 2022-03-14 Guidage à distance pour véhicules autonomes

Publications (1)

Publication Number Publication Date
EP4308430A1 true EP4308430A1 (fr) 2024-01-24

Family

ID=83320926

Family Applications (1)

Application Number Title Priority Date Filing Date
EP22772009.1A Pending EP4308430A1 (fr) 2021-03-17 2022-03-14 Guidage à distance pour véhicules autonomes

Country Status (3)

Country Link
EP (1) EP4308430A1 (fr)
CN (1) CN116745192A (fr)
WO (1) WO2022197628A1 (fr)

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2983153B1 (fr) * 2013-04-04 2018-12-19 Nissan Motor Co., Ltd Dispositif de reconnaissance de signal
KR101724868B1 (ko) * 2015-06-18 2017-04-07 현대자동차주식회사 영상기반 교통표시 인식장치 및 방법
KR20180068262A (ko) * 2016-12-13 2018-06-21 현대자동차주식회사 교통신호 인식방법 및 시스템
US20190208136A1 (en) * 2017-12-29 2019-07-04 Waymo Llc High-speed image readout and processing
US11046317B2 (en) * 2019-05-31 2021-06-29 Waymo Llc Multi-way stop intersection precedence for autonomous vehicles

Also Published As

Publication number Publication date
CN116745192A (zh) 2023-09-12
WO2022197628A1 (fr) 2022-09-22

Similar Documents

Publication Publication Date Title
US10983520B2 (en) Teleassistance data prioritization for self-driving vehicles
EP2407951B1 (fr) Dispositif d' assistance à la conduite
US10202126B2 (en) Teleassistance data encoding for self-driving vehicles
US11619936B2 (en) Method and apparatus for remotely controlling self-driving vehicle
CN109455180B (zh) 用于控制无人车的方法和装置
CN111383475A (zh) 用于自动代客泊车的系统、方法、基础设施和车辆
KR20200023661A (ko) 자율 주행 차량을 보조하기 위한 방법 및 이를 위한 장치
KR20180064491A (ko) 차량에 대한 운전 의도를 결정하기 위한 방법 및 차량 통신 시스템
US20210024095A1 (en) Method and device for controlling autonomous driving of vehicle, medium, and system
US11654823B2 (en) Light irradiation control apparatus and method of light irradiation control
CN112542055B (zh) 用于辅助机动车的方法
US10684135B2 (en) Method for updating an electronic card of a vehicle
CN111376897A (zh) 支持自动代客泊车的系统和方法以及用于其的基础设施和车辆
KR20210057886A (ko) 차량 충돌 방지 장치 및 그 방법
AU2019339418A1 (en) Remote safe driving methods and systems
CN113129629A (zh) 支持自动代客停车的系统和方法，及其基础设施和车辆
CN110598563A (zh) 可移动设备行进的处理方法、装置及存储介质
US11628861B2 (en) Method and apparatus for controlling vehicle, device and storage medium
JP6245083B2 (ja) 管制装置
EP4308430A1 (fr) Guidage à distance pour véhicules autonomes
CN112537311A (zh) 用于安全可靠地至少部分自动化地引导机动车的方法
CN112365730A (zh) 自动驾驶方法、装置、设备、存储介质以及车辆
EP3943352A1 (fr) Procédé de commande de conduite, appareil, dispositif, support et système
JP2018067140A (ja) 交通制御装置、交通制御方法、および自動運転車両
US20220406187A1 (en) Control apparatus, movable object, control method, and terminal

Legal Events

Date Code Title Description
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE

PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

17P Request for examination filed

Effective date: 20230626

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

DAV Request for validation of the european patent (deleted)
DAX Request for extension of the european patent (deleted)