US20190344799A1 - Vehicle control system and method - Google Patents

Vehicle control system and method Download PDF

Info

Publication number
US20190344799A1
US20190344799A1 US16/409,308 US201916409308A US2019344799A1 US 20190344799 A1 US20190344799 A1 US 20190344799A1 US 201916409308 A US201916409308 A US 201916409308A US 2019344799 A1 US2019344799 A1 US 2019344799A1
Authority
US
United States
Prior art keywords
failure
tier
task
node
vehicle
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US16/409,308
Inventor
Kartik Tiwari
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Gatik AI Inc
Original Assignee
Starsky Robotics Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Starsky Robotics Inc filed Critical Starsky Robotics Inc
Priority to US16/409,308 priority Critical patent/US20190344799A1/en
Assigned to Starsky Robotics, Inc. reassignment Starsky Robotics, Inc. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: TIWARI, Kartik
Publication of US20190344799A1 publication Critical patent/US20190344799A1/en
Assigned to GATIK AI INC. reassignment GATIK AI INC. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: Starsky Robotics, Inc.
Assigned to STARSKY ROBOTICS (ABC), LLC reassignment STARSKY ROBOTICS (ABC), LLC ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: Starsky Robotics, Inc.
Assigned to GATIK AI INC. reassignment GATIK AI INC. CORRECTIVE ASSIGNMENT TO CORRECT THE NAME OF ASSIGNOR PREVIOUSLY RECORDED ON REEL 054956 FRAME 0773. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Assignors: STARSKY ROBOTICS (ABC), LLC
Abandoned legal-status Critical Current

Links

Images

Classifications

    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60WCONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W50/00Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
    • B60W50/02Ensuring safety in case of control system failures, e.g. by diagnosing, circumventing or fixing failures
    • B60W50/0205Diagnosing or detecting failures; Failure detection models
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60WCONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W10/00Conjoint control of vehicle sub-units of different type or different function
    • B60W10/18Conjoint control of vehicle sub-units of different type or different function including control of braking systems
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60WCONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W10/00Conjoint control of vehicle sub-units of different type or different function
    • B60W10/20Conjoint control of vehicle sub-units of different type or different function including control of steering systems
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60WCONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W50/00Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
    • B60W50/02Ensuring safety in case of control system failures, e.g. by diagnosing, circumventing or fixing failures
    • B60W50/0225Failure correction strategy
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60WCONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W50/00Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
    • B60W50/02Ensuring safety in case of control system failures, e.g. by diagnosing, circumventing or fixing failures
    • B60W50/029Adapting to failures or work around with other constraints, e.g. circumvention by avoiding use of failed parts
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60WCONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W60/00Drive control systems specially adapted for autonomous road vehicles
    • B60W60/001Planning or execution of driving tasks
    • B60W60/0015Planning or execution of driving tasks specially adapted for safety
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60WCONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W50/00Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
    • B60W50/02Ensuring safety in case of control system failures, e.g. by diagnosing, circumventing or fixing failures
    • B60W50/0205Diagnosing or detecting failures; Failure detection models
    • B60W2050/021Means for detecting failure or malfunction
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60WCONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W50/00Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
    • B60W50/02Ensuring safety in case of control system failures, e.g. by diagnosing, circumventing or fixing failures
    • B60W50/0205Diagnosing or detecting failures; Failure detection models
    • B60W2050/022Actuator failures
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60WCONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W50/00Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
    • B60W50/02Ensuring safety in case of control system failures, e.g. by diagnosing, circumventing or fixing failures
    • B60W50/029Adapting to failures or work around with other constraints, e.g. circumvention by avoiding use of failed parts
    • B60W2050/0292Fail-safe or redundant systems, e.g. limp-home or backup systems
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60WCONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2520/00Input parameters relating to overall vehicle dynamics
    • B60W2520/10Longitudinal speed

Definitions

  • the list of task nodes is defined with respect to task nodes that are not functioning nominally (e.g., as a list of nonfunctional task nodes).
  • the list of nonfunctional task nodes corresponding to a tier one failure and associated response action includes any task node in that has a shortest path segment of one between it and an actuation element (e.g., is directly communicatively coupled to and/or publishes output to an actuation element in order to actuate the vehicle, wherein the published output of at least one task node 111 on the list controls an actuator of the actuation subsystem, etc.).
  • the aggregator node 120 can determine the failure tier in response to a task node in of the set of task nodes 110 generating a failure flag for a time period exceeding a threshold duration.
  • the threshold duration can be predetermined (e.g., a fixed threshold that applies to each task node, a different fixed threshold that is associated with each task node, a 0.5 second threshold, a 1 second threshold, a 50-millisecond threshold, etc.).
  • the threshold duration can be dynamically determined and/or otherwise depend on the characteristics of the one or more failure flags received by the aggregator node 120 .

Abstract

A vehicle control system for controlling a vehicle, including a set of task nodes, wherein each of the set of task nodes compares a failure criterion to input to the task node to generate a failure flag and provides output to at least one other task node of the set of task nodes; an aggregator node directly communicatively coupled to each of the set of task nodes, and wherein the aggregator node: receives a set of failure flags from the set of task nodes, determines a failure tier based on the set of failure flags, and generates a failure tier message defining the failure tier; and, a behavior planning node communicatively coupled to the aggregator node, wherein the behavior planning node: receives the failure tier message, and in response to the failure tier defined by the failure tier message, generates vehicle control instructions.

Description

    CROSS REFERENCE TO RELATED APPLICATIONS
  • This application claims the benefit of U.S. Provisional Application Ser. No. 62/669,586, filed 10-MAY-2018, and incorporates it herein in its entirety by this reference.
  • TECHNICAL FIELD
  • This invention relates generally to the vehicle control field, and more specifically to a new and useful system and method in the vehicle control field.
  • BACKGROUND
  • In autonomous and semi-autonomous control of automotive vehicles, it can be important to manage control flow in the event of system or component failure. If system or component failure is not identified and analyzed, adequate corrective action and/or response often cannot be taken in a timely and safe manner. Unlike many other robotics applications, the ability to immediately shut down operation and physically cease moving substantially instantaneously is often unavailable or undesirable for moving vehicles, and in particular for freight-bearing vehicles (e.g., semi-trucks) on roadways shared with smaller, more vulnerable vehicles. In addition, control and operation of automotive vehicles can require the impact of each component on vehicle operation (e.g., safe operation, nominal operation, efficient operation, etc.) to be monitored and recorded, and for system and/or component failure to express high traceability; however, these requirements are often inadequately met by conventional solutions to autonomous and/or semi-autonomous vehicle control.
  • Thus, there is a need in the vehicle control field to create a new and useful system and method to manage system or component failure. This invention provides such a new and useful system and method.
  • BRIEF DESCRIPTION OF THE FIGURES
  • FIG. 1 depicts a schematic illustration of a variation of the vehicle control system;
  • FIG. 2 depicts a flowchart of a variation of the method of using the vehicle control system;
  • FIG. 3 depicts an example time evolution of connectivity of a variation of a task node network of the vehicle control system;
  • FIG. 4 depicts an example architecture of the vehicle control system including a tiered response structure; and
  • FIG. 5 depicts an example dataset corresponding to tiered failure recognition in accordance with the vehicle control system and method of use thereof.
  • DESCRIPTION OF THE PREFERRED EMBODIMENTS
  • The following description of the preferred embodiments of the invention is not intended to limit the invention to these preferred embodiments, but rather to enable any person skilled in the art to make and use this invention.
  • 1. Overview
  • As shown in FIG. 1, the control system 100 includes a set of task nodes no, an aggregator node 120 communicatively coupled to each of the set of task nodes, and a behavior planning node 130 communicatively coupled to the aggregator node 120. The control system 100 can optionally include: an actuation subsystem 140, a perception subsystem 150, and any other components suitable for vehicle control. The control system 100 functions to control the operation of a vehicle 900 (e.g., an automotive vehicle). The control system 100 can additionally or alternatively function to manage system and/or component failure during vehicle operation. In variations, the control system 100 can function to apply a tiered failure mode framework to the operation of the system, for at least the practical application of safely and responsibly degenerating the operation of an unmanned vehicle. The control system can also function to provide real-time and/or subsequent (e.g., post-failure) insight into system and/or component failure modes. However, the control system 100 can additionally or alternatively have any other suitable function related to monitoring, managing, and/or mitigating failures in an autonomous and/or unmanned vehicle.
  • As shown in FIG. 2, the method 200 can include: monitoring a set of task nodes for a set of failure flags S210; determining a failure tier based on the set of failure flags S220; selecting a failure action based on the failure tier S230; and executing the failure action S240. The method 200 can optionally include any other suitable blocks and/or processes for vehicle control. The method 200 functions to manage system and/or component failure in real-time (e.g., near real-time, substantially real-time, etc.) during vehicle operation. The method 200 can also function to record and report system and/or component failure. In variations, the method 200 can function to apply a tiered failure mode framework to the operation of the system, for at least the practical application of safely and responsibly degenerating the operation of an unmanned vehicle. However, the method can additionally or alternatively have any other suitable function related to monitoring, managing, and/or mitigating failures in an autonomous and/or unmanned vehicle.
  • The control system 100 can include, be implemented by, and/or execute at various related systems. The control system 100 preferably controls a vehicle, such as an automotive vehicle (e.g., a passenger vehicle, a commercial vehicle, a car, a truck, a semi-truck, a tractor-trailer, etc.), a non-automotive vehicle (e.g., a locomotive, a mobile robot, an aerial vehicle, etc.), and any other suitable vehicle. In particular, the vehicle is preferably configured for unmanned operation wherein a human is not in a position to exercise physical control over the operation of the vehicle (e.g., wherein the human is collocated with the vehicle but not in such a position, wherein the human is remotely located and teleoperating or monitoring the vehicle, etc.). However, the vehicle can be otherwise suitably configured. The control system 100 can include or interface with a database (e.g., an onboard database, a remote database, a distributed database, a cloud database, a server, etc.), for such purposes as logging or recording information (e.g., in relation to component failure, failure event data, etc.), retrieving information (e.g., mapping data, environmental data, etc.), and/or any other suitable purpose and/or in relation to any other suitable data. The control system 100 can include and/or interface with a communication system, for such purposes as transmitting and receiving data (e.g., to or from a remote computing system), and/or any other suitable purpose. However, the control system 100 can include, be implemented by, and/or execute at any other suitable related systems.
  • The method 200 is preferably implemented by a control system substantially analogous to the control system 100 as described herein, but can additionally or alternatively be implemented by any other suitable vehicle control system.
  • 2. Benefits
  • Variations of the system and method can afford various benefits and/or advantages.
  • First, variations of the system can enable a transparent control system architecture, which enables system improvement and development without compromising system reliability or safety. The control system architecture can be transparent to control system developers; for example, the responsibility of each piece of the system (e.g., task node) for generating failure flags can be substantially identical across all components, irrespective of the task associated with each component. The mechanism for generating failure flags can also be substantially identical for each system component (e.g., task node) irrespective of the task associated with the component, though the criteria for generating various types of failure flags can differ by the type of task; this property can enhance transparency to system developers during system development (e.g., iterative development, ongoing development, continuous development, etc.). For example, a task node in that performs an aspect of lane line detection can define a different implementation in software than a task node in that performs an aspect of lateral position control, but both task nodes can apply a failure criterion (e.g., a bounds check, a value check, etc.) on the output that generates a failure flag with substantially identical characteristics.
  • Second, variations of the system and method can enable efficient and transparent review of vehicle control performance (e.g., event data related to discrete occurrences during vehicle operation that triggered a behavioral response of the control system; system computational performance metrics such as accuracy, latency, etc.; component performance; etc.). For example, variations of the system can log traceable failures on a component by component basis to enable system failures to be reviewed and diagnosed subsequent to a failure (e.g., a failure event). In another example, variations of the system and method can enable the frequency of failure of system components (e.g., software modules, task nodes, hardware components driven by the control system, etc.) to be determined (e.g., in relation to standards associated with automotive safety and/or control systems). In another example, disengagements of the vehicle control system can be traced back to the root cause (e.g., tracing the progression of a component and/or system failure from an initial failure flag to promotion of the failure flag through a set of tiered responses to an ultimate disengagement of the vehicle control system and associated need for human intervention).
  • Third, variations of the system and method can enable responsible degeneration (e.g., wind-down, cessation, etc.) of autonomous operation in the event of system and/or component failure. Responsible degeneration preferably includes safe and managed de-escalation of autonomous behaviors until the vehicle is in an inert state (e.g., non-moving and not capable of initiating movement or other behaviors without a human providing control input). Responsible degeneration (e.g., operation in a degenerate mode) can include taking immediate, irreversible action to stop the vehicle (e.g., in the current lane of travel on the highway), which can, in some examples, be referred to as an “uncontrolled action” because once the action is initiated (e.g., an emergency stop condition is reached) it preferably cannot be reversed (e.g., controlled) by the control system. Responsible degeneration can also include taking controlled action (e.g., reversible action) to stop the vehicle (e.g., pulling over to a shoulder, making a controlled stop in a lane, etc.) and/or otherwise mitigate the component failure (e.g., slowing to below a threshold speed and transitioning to teleoperation control of the vehicle in the event of failure of long-range object detection task nodes).
  • Fourth, variations of the system and method can enable high speed (e.g., low latency) response to system and/or component failure. Monitoring of failure flags can be continuously performed, such that system failures, component failures, and/or recurrent errors are not missed and/or overlooked until mitigation actions must be aggressive in order to responsibly manage failure. For example, substantially continuous monitoring of the task nodes responsible for long range object detection can enable a failure of one or more of those task nodes to be mitigated prior to any undetected objects entering a region around the vehicle that would require aggressive evasive action that could in turn reduce the safety of the vehicle and/or surrounding entities.
  • However, variations of the system and/or method can otherwise suitable afford any suitable benefits and/or advantages.
  • 3. System
  • As shown in FIG. 1, the control system 100 includes a set of task nodes 110, an aggregator node 120 communicatively coupled to each of the set of task nodes, and a behavior planning node 130 communicatively coupled to the aggregator node 120 and at least one of the set of task nodes. The control system 100 can optionally include: an actuation subsystem 140, a perception subsystem 150, and any other components suitable for vehicle control (e.g., perception, planning, decision making, actuation, etc.). The control system 100 functions to control the operation of a vehicle (e.g., an automotive vehicle, a semi-tractor trailer, any other suitable vehicle, etc.). The control system 100 can also function to operate the vehicle in response to a failure according to a tiered response structure, which preferably includes three tiers wherein the highest tier (e.g., most severe failure) is tier I, middle tier is tier II, and lowest tier (e.g., least severe failure) is tier III. The tiered response structure can additionally or alternatively include any suitable number of tiers. The control system 100 can additionally or alternatively function to manage system and/or component failure during vehicle operation, provide real-time and/or subsequent insight into system and/or component failure modes, and have any other suitable function.
  • 3.1 Task Nodes
  • The set of task nodes functions to collectively perform the set of tasks used to operate a vehicle in any suitable driving environment, and in response to any suitable inputs (e.g., environmental inputs such as weather, object perception inputs, other perception inputs, planning inputs, control inputs, etc.). The set of task nodes can be interconnected (e.g., between inputs and outputs of task nodes) in any suitable manner, and the inter-node connections can dynamically change in response to any suitable situational change (e.g., as shown in FIG. 3). Each task node in preferably “subscribes” to the output of at least one other task node in and/or an external sensor 151, and each task node in preferably “publishes” its own output to at least one other task node in and/or external actuator 141 or other device; task nodes can subscribe to a plurality of other task nodes and/or devices and publish to a plurality of other task nodes and/or devices. The interconnected set of task nodes can additionally or alternatively be referred to as a network of task nodes (e.g., a task node network). Each of the set of task nodes can connect to the aggregator node 120 for data transfer (e.g., transmission of failure flags to the aggregator node), such that failure of any node of the set of nodes is observed by the aggregator node 120 of the system. The set of task nodes can be redundant (e.g., the set of task nodes can include subsets of task nodes that are duplicates and perform the same task as at least one other task node), but can alternatively be a set of unique task nodes (e.g., wherein each task is performed by only one task node).
  • A task node in of the set of task nodes functions to perform a task (e.g., execute a series of instructions, execute a model or portion of a model, etc.) related to vehicle control and/or operation. Each task node 111 can also function to raise a failure flag of any suitable tier (e.g., a tier I, tier II, or tier III flag). The task node 111 can also function to subscribe to the output of a different node (e.g., another task node of the set of task nodes), wherein the task node in receives data (e.g., processed data, computational output, etc.) from the node to which it is subscribed. Similarly, the task node 111 can function to publish its output, wherein its output is provided to a different node that subscribes to its published output (e.g., for further processing by the network). However, a task node in of the set of task nodes can additionally or alternatively have any other suitable function.
  • In some variations, the task node 111 is defined as a software module with distinct expected inputs and outputs (e.g., a software driver for a camera). A task node 111 can be defined as a collection of other task nodes (e.g., a lane-change task node can include a turn signal task node, an accelerate task node, and a steering task node) and/or define a single primitive task (e.g., accelerate to a target speed, apply the brakes by a target amount, etc.). The expected inputs are preferably based on the subscriptions of the task node 111 to other task node(s), and the outputs are preferably the published outputs provided to the set of other task nodes that subscribe to the task node; however, the inputs and outputs can be otherwise suitably defined. The task node 111 can be defined either purely as software (e.g., a set of predetermined, computer-executable instructions that process digital input into digital output), as a combination of software and hardware (e.g., a set of predetermined, computer-executable instructions that process digital input into the physical output of a hardware device such as the position of a servo or content of a display, and can include the generation of analog output), purely as hardware (e.g., the physical manifestation of executable instructions in an integrated circuit, the electromagnetic transducer associated with an actuator, etc.), and/or be otherwise suitably defined.
  • The task node in preferably includes a node-specific predetermined set of instructions (e.g., internal logic, comparator with a failure criterion, bounds check, etc.) that functions to analyze its own operation, and in particular to determine whether the task node in itself has failed (e.g., met a failure condition, failed to meet a pass condition, etc.). For example, the task node 111 can include instructions that evaluate its own output (e.g., the published output) to determine whether or not the output is out of bounds (e.g., wherein the output includes values that should be numerical but are infinite, represented by an Inf character, and/or undefined) by comparing the output to a failure criterion defining the bounds. In another example, the task node 111 can include output check logic that evaluates its own output to determine whether the output is anomalous (e.g., different from an expected value or range of values by greater than a threshold magnitude). However, the task node 111 can additionally or alternatively perform operational checks on itself in any other suitable manner.
  • The task node in preferably raises a failure flag in response to failure of the task node 111 (e.g., within or during the execution of the task defined by the task node, processing of data by the task node, etc.), and communicates the failure flag to the aggregator node 120. In some variations, the failure flag does not have an associated tier and is identical across task nodes, and the failure tier is determined at the aggregator node 120 based on characteristics of the flag (e.g., the frequency with which it is generated over an interval of time or time window, the task associated with the task node in that raised the failure flag, etc.). In additional or alternative variations, the failure flag itself can be associated with various tiers (e.g., a tier I failure flag, a tier II failure flag, a tier III failure flag, etc.) upon generation (e.g., as metadata), and the determination of the failure tier by the aggregator node 120 can be based on the metadata associated with the failure flag. The tier of the failure flag can, in some instances, correspond to the type of failure designated by the tier (e.g., according to a tiered response structure as described below). However, the task node 111 can additionally or alternatively raise a failure flag in any other suitable manner.
  • In a first variation, every task node in of the set of task nodes can raise a lowest-level failure flag (e.g., a tier III failure flag), and any task node in of the set of task nodes that interfaces with a hardware touchpoint (e.g., a task node including a hardware driver for an actuator 141 of the actuation subsystem 140, a task node in direct communication with a portion of the actuation subsystem, etc.) can raise a highest-level failure flag (e.g., a tier I failure flag), and no task nodes can raise a mid-level failure flag (e.g., a tier II failure flag), which is instead raised by the aggregator node 120 in response to receiving one or more tier III failure flags (e.g., by determining that a tier II failure has occurred in response to the characteristics of the received failure flags). In an example of such a variation, the aggregator node 120 can determine that a tier II failure has occurred in response to receiving one or more failure flags (e.g., tier III failure flags) over a period (e.g., a time period, a time window) exceeding a threshold duration (e.g., a predetermined threshold duration, a dynamically determined threshold duration based on the context or operating environment of the vehicle, etc.). In another example of such a variation, the aggregator node 120 can determine that a tier II failure has occurred in response to receiving a number of failure flags above a threshold number (e.g., a predetermined threshold number, a dynamically determined threshold number based on the context or operating environment of the vehicle, etc.). However, in alternative variations, any suitable task node in of the set of task nodes can raise a failure flag, which can have any suitable level or tier and/or be independent of the level or tier.
  • 3.2 Aggregator Node
  • The aggregator node 120 functions to monitor the failure flag outputs of each of the set of task nodes. The aggregator node 120 can also function to report a failure to the behavior planning node 130 along with an associated failure tier and/or other failure metadata. The aggregator node 120 can also function to determine a list of operable task nodes (and/or a list of inoperable task nodes, nonfunctional task nodes, functional task nodes, etc.), and perform an action based on the list of task nodes (e.g., report the list of task nodes to the behavior planning node 130, determine a failure tier based on the list of task nodes, compare the list of task nodes against a failure criterion, etc.). The aggregator node 120 can also function to record and/or report a failure description (e.g., metadata associated with the failure flag). The aggregator node 120 can also function to promote a failure flag from a lower tier to a higher tier (e.g., a tier III flag to a tier II flag, a tier II flag to a tier I flag, etc.), or otherwise suitably determine a failure tier based on one or more received failure flags. However, the aggregator node 120 can additionally or alternatively have any other suitable function.
  • The aggregator node 120 is preferably communicatively coupled directly to each node of the set of task nodes, such that the shortest communication path between a task node in and the aggregator node 120 is one path segment (e.g., as shown in FIG. 1). However, in additional or alternative variations, the aggregator node 120 can be indirectly communicatively coupled to one or more of the nodes of the set of task nodes. The communication pathways (e.g., data transfer connections, data pathways, etc.) between the aggregator node 120 and each of the set of task nodes are preferably one-way such that the aggregator node 120 can receive information (e.g., failure flags, failure flag metadata, etc.) from the task nodes but does not transmit information thereto. However, in alternative variations, the communication pathways between the aggregator node 120 and each of the set of task nodes can be two-way such that information can be both received and transmitted.
  • In variations, the aggregator node 120 can generate a list of functional task nodes based on the received failure flags from each of the set of task nodes. The list of functional task nodes can be positively defined, such that each task node 111 on the list of functional task nodes is an operational (e.g., correctly functioning) task node in and the receipt of a failure flag from a task node in removes the task node in (and/or an identifier thereof) from the list. The list of functional task nodes can additionally or alternatively be negatively defined, such that each task node 111 on the list of functional task nodes is a failed task node 111 (e.g., a task node from which a failure flag has been received); in such cases, the list of functional task nodes can be referred to as a list of nonfunctional task nodes. For example, the aggregator node 120 can generate a list of nonfunctional task nodes from the set of task nodes, based on the set of failure flags, wherein the set of failure flags received correspond to the task nodes on the list of nonfunctional task nodes. The list of task nodes (e.g., functional, nonfunctional) can be adjusted at the frequency at which failure flags are received by the aggregator node 120 (e.g., once per clock cycle, once per update period, etc.). However, the list of functional task nodes can be otherwise suitably defined such that it can be used to generate a comparison between available task nodes (e.g., task nodes that are functioning nominally, correctly, etc.) and desired task nodes for a selected failure action (e.g., wherein the behavior planning node selects a failure action in response to a report from the aggregator node).
  • In a specific example, the list of task nodes is defined with respect to task nodes that are not functioning nominally (e.g., as a list of nonfunctional task nodes). In this example, the list of nonfunctional task nodes corresponding to a tier one failure and associated response action (e.g., failure action, failure response action, etc.) includes any task node in that has a shortest path segment of one between it and an actuation element (e.g., is directly communicatively coupled to and/or publishes output to an actuation element in order to actuate the vehicle, wherein the published output of at least one task node 111 on the list controls an actuator of the actuation subsystem, etc.). In a related example, the list of nonfunctional task nodes corresponding to a tier one failure and associated response action includes any task node in that has a shortest path segment of one between it and a sensor 151 (e.g., sensing element) of the perception subsystem 150 (e.g., is directly communicatively coupled to and/or subscribes to input from a sensing element).
  • The aggregator node 120 can, in variations, determine a failure tier based on promoting a low-level failure tier (e.g., a tier III failure determined based on receipt of a single failure flag) to a higher tier (e.g., a tier II failure, a tier I failure, etc.). Promoting a failure tier can be based on a duration over which the failure flag is repeatedly received from the same task node; for example, the aggregator node 120 can determine a tier II failure has occurred (e.g., and generate a tier II failure message) based on receiving a failure flag periodically (e.g., every computational cycle, at 10 Hz, at any other suitable frequency or with any other suitable period, etc.) for a period of time (e.g., a predetermined period of time, a predetermined time window, a specified period of time, a dynamically determined period of time, etc.), above a threshold duration. For example, the aggregator node 120 can determine the failure tier in response to a task node in of the set of task nodes 110 generating a failure flag for a time period exceeding a threshold duration. The threshold duration can be predetermined (e.g., a fixed threshold that applies to each task node, a different fixed threshold that is associated with each task node, a 0.5 second threshold, a 1 second threshold, a 50-millisecond threshold, etc.). The threshold duration can be dynamically determined and/or otherwise depend on the characteristics of the one or more failure flags received by the aggregator node 120. For example, the threshold duration can be inversely proportional to the number of task nodes that receive the published output of the task node in generating the failure flag (e.g., wherein the number of task nodes subscribed to the task node that raised the failure flag can be received by the aggregator node 120 along with the failure flag as metadata).
  • Promoting a failure tier can additionally or alternatively be based on the number of failure flags received from different task nodes (e.g., wherein the aggregator node determines the failure tier based on a number of failure flags of the set of failure flags received within a predetermined time window). For example, the aggregator can determine that a tier II failure has occurred in response to receiving a number of failure flags (e.g., wherein each failure flag received individually indicates a tier III failure), from unique task nodes, above a threshold number within a time window. The threshold number and the length of the time window can be predetermined, dynamically determined, and/or otherwise suitably determined.
  • However, promoting a failure tier to a higher failure tier can be otherwise suitably performed with any other suitable basis.
  • The aggregator node 120 can generate a failure message based on received failure flags (e.g., characteristics of the received failure flag). The generated failure message can be of the same tier or level as that associated with the received failure flags, or of a different (e.g., higher, more severe, etc.) tier or level. For example, the aggregator node 120 can generate a tier I failure message based on a number of tier II failure flags received within a time window (e.g., from the same task node, from different task nodes, etc.). The failure message is provided to the behavior planning node 130 by the aggregator node 120, and functions to initiate a failure response (e.g., a response action, a failure response action, a failure action, etc.) by the behavior planning node 130. However, the aggregator node 120 can generate a failure message based on received failure flags in any other suitable manner.
  • 3.3 Behavior Planning Node
  • The behavior planning node 130 functions to operate the vehicle in a degenerate mode (e.g., failure management mode) in response to a report from the aggregator node 120 (e.g. a report including a failure message, the failure message itself, a report including a list of operable task nodes, etc.). The behavior planning node 130 can also function to process input from the perception subsystem 150 to plan vehicle behavior. The behavior planning node 130 can also function to process input from the set of task nodes 110 to plan vehicle behavior. The behavior planning node 130 can also function to generate control outputs to operate the vehicle (e.g., using the actuation subsystem) during nominal vehicle operation (e.g., wherein no failure flags above a threshold failure level are received), wherein the control outputs can be provided directly to the actuation subsystem 140 and/or to the actuation subsystem 140 or other devices by way of the set of task nodes 110. The behavior planning node 130 can additionally or alternatively have any other suitable function.
  • The degenerate mode can include a tiered response structure defining a set of tiers, each tier of the set of tiers associated with a response that defines a set of failure actions (e.g., response actions, failure response actions, etc.). In a first variation of the behavior planning node 130, as shown in FIG. 4, the tiered response structure includes three tiers, wherein a tier I failure corresponds to a critical failure (e.g., wherein the vehicle cannot be successfully autonomously operated within nominal limits defined by the operational design domain and should be brought to a stop immediately to avoid exceeding the safe operation limitations of the system), a tier II failure corresponds to a manageable failure (e.g., wherein the vehicle is functionally capable of being brought to a controlled stop), and a tier III failure corresponds to a nominal component-level failure (e.g., that does not adversely affect autonomous vehicle operation, due to the component being redundant with another component or set of components or for any other suitable reason). However, the tiered response structure can have any suitable number of tiers, wherein each tier corresponds to any suitable failure type that in turn calls for any suitable response and associated set of failure actions.
  • In the first variation of the behavior planning node 130, receipt of a tier I failure message at the behavior planning node 130 results in the behavior planning node 130 executing a failure action that includes taking an irreversible action (e.g., bringing the vehicle to an immediate stop), wherein no vehicle system (e.g., onboard control system, teleoperator, etc.) can abort (e.g., reverse) the irreversible action once initiated. Control authority can be applied to action-related (e.g., stopping-related) systems during an irreversible stop (e.g., a feedback control loop can be applied to a brake actuator to prevent lockup during stoppage); however, unrelated control inputs (e.g., control inputs related to any task that is not stopping the vehicle) are ignored until the action is fully executed (e.g., the vehicle is brought to a complete stop) in this and related variations. In a first example of an irreversible action, a default-closed relay is closed in response to a tier I failure (e.g., a loss of power to the brake actuator) and the brakes are automatically applied by the default-closed relay that closes upon the occurrence of the failure. However, the irreversible action can be any other suitable action that is automatically (e.g., without active intervention by the control system during execution of the action) executed to completion in response to a tier I failure.
  • Further examples of failure actions that can be taken in addition to executing an irreversible stop can include deploying and/or activating emergency signals (e.g., emergency flashers of the vehicle, strobes, sirens, etc.), transmitting a request for human assistance (e.g., notifying a teleoperator that the vehicle has automatically brought itself to a halt in response to a failure of tier one or tier two, etc.), and/or any other suitable action in response to failure.
  • A tier I failure flag is preferably raised (e.g., by any task node of the set of task nodes, by the aggregator node 120, by the behavior planning node 130, etc.) in response to a critical failure (e.g., a critical failure substantially as described above) of the system or a component thereof. Such critical failures can include: failure of any hardware touch points 141 of the system (e.g., actuator touch points), wherein a task node in interfaces directly with hardware that physically manipulates control inputs of the vehicle (e.g., a task node in including a driver for a steering input actuator, a task node in including an encoder for a pedal actuator position, etc.); failure of the behavior planning node 130 itself (e.g., wherein any diagnostic check associated with operation of the behavior planning node 130 itself fails, wherein the behavior planning node 130 computationally freezes or crashes, etc.); promotion of a tier II failure to a tier I failure based on promotion criteria (e.g., as described below); and, any other suitable failure that prevents successful vehicle operation (e.g., successful autonomous vehicle operation).
  • In the first variation of the behavior planning node 130, receipt of a tier II failure report at the behavior planning node 130 results in the behavior planning node 130 executing a failure action that includes taking a reversible action (e.g., bringing the vehicle to a controlled stop). This failure action can include selecting a controlled stop task from a prioritized list of controlled stop tasks (e.g., degradation behaviors), wherein the controlled stop task is selected based on the availability of functional task nodes. Each controlled stop task is associated with a list of required task nodes, and the behavior planning node 130 selects the controlled stop task in this variation by comparing a list of functional task nodes (e.g., received in a report or message from the aggregator node) to the list of required task nodes. Upon determination that at least one of the required task nodes is not represented in the list of functional task nodes associated with the first controlled stop task, the behavior planning node 130 performs the same operation (i.e., comparing the list of required and functional task nodes) with respect to the second controlled stop task in the prioritized list, and repeats this until the end of the prioritized list is reached. In this variation, in the event that no controlled stop task is successfully validated (e.g., wherein all the required task nodes for the controlled stop task are not represented in the list of functional task nodes), the behavior planning node 130 promotes the tier II failure to a tier I failure and executes a suitable tier I failure action (e.g., an irreversible action as described above).
  • In a first example, the prioritized list of controlled stop tasks includes: first, pulling over to the shoulder; second, slowing down to less than a threshold speed and transferring control of the vehicle to a teleoperator; and, third, performing a controlled stop in the lane in which the vehicle is traveling. However, in alternative variations and/or examples, the prioritized list of controlled stop tasks can include any other suitable tasks, and/or any number of suitable tasks (e.g., more than three, fewer than three, etc.).
  • A tier II failure flag is preferably raised (e.g., by any task node of the set of task nodes, by the aggregator node 120, by the behavior planning node 130, etc.) in response to a duration of a tier III failure flag exceeding a threshold duration. The threshold duration is preferably based on the nature of the task node in raising the tier III failure flag; for example, a tier II failure flag can be raised in response to a task node in that operates a turn signal raising a tier III failure flag over an interval of at least 1 second, whereas a tier II failure flag can be raised in response to a task node in that processes wide-view image data raising a tier III failure flag over an interval of 500 milliseconds. However, the threshold duration can additionally or alternatively be based on the number of other task nodes to which the failing task node 111 is subscribed (e.g., wherein a greater number of nodes corresponding to a shorter duration), a category associated with the failing task node 111 (e.g., a perception-based task node, an actuation-based task node, a speed governor task node, etc.), and have any other suitable basis. The threshold duration is preferably a time duration (e.g., a number of milliseconds, a number of seconds, etc.), but can additionally or alternatively be a cycle duration (e.g., a number of computational cycles performed by the failing node), a sample duration (e.g., a number of frames sampled from a camera of the perception subsystem 150, a number of samples of an onboard gyroscope, etc.), and any other suitably defined duration. However, a tier II failure flag can additionally or alternatively be otherwise suitably raised.
  • In the first variation of the behavior planning node 130, receipt of a tier III failure report at the behavior planning node 130 results in the behavior planning node 130 executing a failure action that includes recording (e.g., logging, a recording action, etc.) information associated with the tier III failure report (e.g., traceback information, elapsed time since the last tier III failure associated with the node that raised the tier III failure, the perception subsystem state associated with the tier III failure, etc.). The behavior planning node 130 can also log (e.g., record) the time point associated with receipt of the tier III failure (e.g., in order to determine that a failure flag should be promoted to a higher tier, such as a tier II failure, due to having received the tier III failure repeatedly over a predetermined time period as shown by example in FIG. 5).
  • A tier III failure flag is preferably raised (e.g., by a task node of the set of task nodes) in response to any off-nominal operation of the task node in raising the tier III failure flag. Such off-nominal operation, preferably determined by comparison of the input or output to the node with a failure criterion but alternatively otherwise suitably determined, can include: a bounds check failure of published output or subscribed input, a logic check failure of internal computations (e.g., computations performed by the task node) or other data, a momentary lane-following failure (e.g., a spiked lane line detection output as shown in FIG. 5), an object detection failure in an image frame or image sequence, and any other suitable failure of any other component that does not interface with hardware actuation components of the system (e.g., wherein task nodes interfacing with hardware actuation components raise tier I failure flags in response to failures instead of tier III failure flags). However, a tier III failure flag can additionally or alternatively be otherwise suitably raised.
  • However, in alternative variations, receipt of a failure report associated with any suitable tier at the behavior planning node 130 can result in execution of any suitable failure actions.
  • 3.4 Actuation Subsystem
  • The control system 100 can include an actuation subsystem 140 that functions to actuate the control interfaces of the vehicle. The actuation subsystem 140 can also function to execute a selected vehicle action (e.g., a failure action determined by the behavior planning node 130). Control interfaces actuated by the actuation subsystem 140 can include human-oriented control interfaces, computer control interfaces, and any other suitable control interfaces. Human-oriented control interfaces can include, for example, a steering wheel, pedals (e.g., clutch pedal, gas pedal, brake pedal, etc.), a shifter (e.g., shift lever, shift paddle, etc.), and any other suitable control mechanism configured for operation by a human. Computer control interfaces can include, for example, a throttle actuation interface (e.g., a controllable fuel flow rate regulator), a brake actuation interface (e.g., redundant brake calipers, electronic control mechanism for existing brakes, etc.), a transmission actuation interface (e.g., an electronically-controllable automatic transmission), and any other suitable control interface configured to receive control instructions from a computing system.
  • The actuation subsystem 140 can receive inputs, including control instructions and/or operator inputs. Control instructions are preferably received from the behavior planning node 130, and can include voltage levels, current levels, time-varying signals, constant signals, trigger- and/or event-based signals, and any other suitable instructions and/or parameters that can effect an output of the actuation subsystem 140 (e.g., an actuation of a control interface by the actuation subsystem). However, control instructions can additionally or alternatively be received from any suitable system component or subsystem (e.g., a task node, a failsafe mechanism, etc.). Operator inputs are preferably received from an operator (e.g., a driver in the vehicle, a teleoperator, etc.), and can include a human moving the steering wheel by hand, a human depressing a pedal manually, a transmission from a teleoperator including instructions to actuate a portion of the actuation subsystem 140, and any other suitable inputs from an operator.
  • In variations, the actuation subsystem 140 can be controlled by the behavior planning node 130 in combination with the set of task nodes 110. The actuation subsystem 140 preferably actuates control inputs of the vehicle to navigate the vehicle along a roadway in response to the vehicle control instructions (e.g., generated by the behavior planning node 130, received from a teleoperator at the vehicle control system, etc.). However, the actuation subsystem 140 can additionally or alternatively be directly controlled by a human operator (e.g., a teleoperator, a human within the vehicle in a passenger position, a human within the vehicle in a driver position, etc.) and/or otherwise suitably controlled.
  • In a specific example, the vehicle control instructions corresponding to the reversible response action (e.g., controllable stop, reversible action, reversible response, etc.) actuate a first actuator 141 (e.g., a steering actuator) of the actuation subsystem 140 and a second actuator 141 (e.g., a brake actuator) of the actuation subsystem 140 to bring the vehicle to a stopped condition at a righthand side of the roadway (e.g., a shoulder, a rightmost lane, etc.).
  • In another specific example, the vehicle control instructions corresponding to the irreversible response action actuate the brake actuator to bring the vehicle to a stopped condition at a current instantaneous lateral position of the vehicle in the roadway (e.g., at its present lateral location within the lane, which can be the center of the lane and/or anywhere within the lane) and within a longitudinal position equal to the stopping distance once the brakes are applied (e.g., execute an immediate emergency stop in the roadway). In this example, the irreversible response action is executed immediately upon receipt of a failure message defining the failure tier corresponding to the irreversible response action (e.g., a message from the aggregator node 120 that a tier one failure has occurred, a determination by the behavior planning node that a tier one failure has occurred, etc.). However, in additional or alternative examples, the irreversible response action can include any other suitable response to failure that renders the vehicle inert (e.g., stopped and only able to be reactivated with human intervention).
  • 3.5 Perception Subsystem
  • The control system 100 can include a perception subsystem 150 that functions to perceive the surroundings of the vehicle. The perception subsystem 150 can also function to collect localization data and mapping data (e.g., simultaneously, in order to perform simultaneous localization and mapping/SLAM). The perception subsystem 150 can additionally function to collect vehicle operation data (e.g., time series data corresponding to the state of vehicle components during operation), and to record measurements indicative of the driving context (e.g., ambient environment parameters, vehicle location, etc.) and provide outputs (e.g., recorded measurements, processed sensor data, etc.) to the task node network and/or behavior planning node 130 (e.g., as signals, messages, etc.). Inputs to the perception subsystem 150 include sensor data, such as image data (e.g., still images, video streams, compressed image sequences, etc.), range data (e.g., time-of-flight/ToF data, LIDAR data, radar data, stereocamera data, optical flow data, point cloud data, etc.), environmental data (e.g., position data, temperature data, humidity data, etc.), and any other suitable sensor data. Inputs are preferably received from one or more sensors 151, which can be on-board the vehicle, removed from the vehicle (e.g., remote), or in any other suitable location. Outputs of the perception subsystem 150 can include the sensor data (e.g., raw data, unprocessed digitized image data, etc.), as well as data derived from the sensor data, such as fused data (e.g., image and range data that has been combined at a processor to increase accuracy or for any other suitable reason), otherwise processed data (e.g., classified objects, position of object center-of-mass/CoM, object extent in three- or two-dimensional space, object trajectory, etc.), and any other suitable data. Outputs of the perception subsystem 150 and/or components thereof (e.g., sensors) are provided to one or more task nodes of the set of task nodes 110, and/or to the behavior planning node 130; exemplary sensor measurements provided to the set of task nodes 100 and/or behavior planning node 130 include visual markers on the roadway, navigation data (e.g., map position data, GPS data), RADAR data, orientation sensor data, ambient environmental data (e.g., acoustic data, temperature data, etc.), and any other suitable sensor data.
  • Examples of sensors 151 and/or data sources on-board the vehicle include: an inertial measurement unit (IMU), ultrasonic sensors, a data port (e.g., on-board diagnostic module port/OBD port), GPS sensor(s) and modules, cameras (e.g., stereoscopic cameras, single lens cameras, etc.), navigation sensors (e.g., LiDAR, radar, ToF, etc.), position sensors (e.g., actuator position sensors, LVDT sensors, etc.), encoders (e.g., rotary encoders that measure the angular position and/or velocity of rotary actuators of the vehicle and/or vehicle control systems), and any other suitable on-board sensors. The aforementioned sensors of the perception subsystem 150 can additionally or alternatively be located remote from the vehicle (e.g., a stationary roadside sensor, a traffic camera, etc.) and transmit sensor data to the vehicle.
  • Sensor data can be pre-processed by one or more task nodes in communication with the perception subsystem 150, prior to provision to the behavior planning node 130 and/or other components of the control system. Pre-processing can be performed on camera data, GPS data, radar and/or other range data, and any other suitable data. For example, camera data (e.g., images) can be processed to extract lane markings, vehicle objects, visual landmarks (e.g., stop signs, pedestrian crossing signs, buildings, etc.), and any other suitable features. In another example, GPS coordinates can be combined with a map of the vehicle route (e.g., retrieved from a remote server, stored in an onboard database, etc.) to provide navigation data. In another example, radar signatures (e.g., point cloud data) are reduced to trajectories of vehicles in the field of view (FoV) including the velocity and range of the vehicles.
  • 3.6 Additional Specific Examples
  • In a specific example, the vehicle control system 100 for controlling a vehicle includes a set of task nodes 110, an aggregator node 120 directly communicatively coupled to each of the set of task nodes 110, and a behavior planning node 130 communicatively coupled to the aggregator node 120. In this example, each of the set of task nodes 110 is communicatively coupled to and receives subscription input from either another task node in of the set of task nodes 110 or a sensor of the perception subsystem 150 of the vehicle; compares a failure criterion to the subscription input to generate a failure flag and transforms the subscription input into published output, and provides the published output to either another task node in of the set of task nodes 100 or an actuator 141 of the actuation subsystem 140 of the vehicle. In this example, the aggregator node 120 receives a set of failure flags from the set of task nodes 110 and determines a failure tier based on the set of failure flags. The failure tier determined by the aggregator node 120 in this example is one of a first tier, a second tier, and a third tier, wherein the first tier is associated with an irreversible response action, the second tier is associated with a reversible response action, and the third tier is associated with a logging action. The aggregator node 120 determines the failure tier and generates a failure tier message that defines the failure tier, and provides the failure tier message to the behavior planning node 130. In this example, the behavior planning node 130 receives the failure tier message, and in response to the failure tier defined by the failure tier message, generates vehicle control instructions corresponding to one of the logging action, the reversible response action, and the irreversible response action.
  • 4. Method
  • As shown in FIG. 2, the method 200 includes: monitoring a set of task nodes for failure flags S210; determining a failure tier based on the failure flag S220; selecting a failure action based on the failure tier S230; and executing the failure action S240. The method 200 can optionally include any other suitable blocks and/or processes for vehicle control. The method 200 functions to manage system and/or component failure in real-time (e.g., near real-time, substantially real-time, etc.) during vehicle operation. The method 200 can also function to record and report system and/or component failure, and have any other suitable function.
  • Block S210 includes monitoring a set of task nodes for failure flags. Block S210 functions to detect system and/or component failure in an operating vehicle control system. Block S210 is preferably performed by an aggregator node substantially as described above, but can additionally or alternatively be performed by any other suitable system or system component.
  • In variations, Block S210 can include generating a list of nonfunctional task nodes based on the set of failure flags (e.g., wherein each task node that raises a failure flag is placed on the list of nonfunctional task nodes). However, Block S210 can additionally or alternatively include performing any suitable function of an aggregator node or variation thereof substantially as described above, and/or any other suitable action.
  • Block S220 includes determining a failure tier based on the set of failure flags (e.g., wherein the set of failure flags can include a single failure flag). Block S220 functions to associate the failure flag with its corresponding failure tier (e.g., based on metadata of the failure flag, such as its tier designation as tier I, tier II, tier III, etc.) so that a suitable response can be determined (e.g., a failure action as selected in Block S230). Block S220 is preferably performed by an aggregator node substantially as described above, but can additionally or alternatively be performed by a behavior planning node as described above, a task node as described above, and/or any other suitable system or system component.
  • In a variation, Block S220 includes determining that the failure tier is the first tier (e.g., a tier I failure) based on receiving at least one of the failure flags from a task node publishing output to an actuator.
  • In another variation, Block S220 includes determining that the failure tier is the second tier (e.g., a tier II failure) based on receiving a failure flag from a task node of the set of task nodes for a time period exceeding a threshold duration. The threshold duration can, in examples, be proportional to the number of task nodes subscribing to the output of the task node raising the flag, inversely or directly proportional to the speed at which the vehicle is traveling and/or any other suitable dynamical property of the vehicle, and/or otherwise suitably based.
  • In variations wherein Block S210 includes generating a list of nonfunctional task nodes, Block S220 can include determining the failure tier based on the list of nonfunctional task nodes. For example, Block S220 can include determining the failure tier is the first tier (e.g., a tier I failure) by determining that the list of nonfunctional task nodes includes at least one task node that publishes output to the actuator of the actuation subsystem (e.g., controls an actuator).
  • In another example, Block S220 can include determining the failure tier based on comparing a list of nonfunctional task nodes to a prioritized list of degradation behaviors, wherein each degradation behavior requires a subset of task nodes of the set of task nodes for execution. For example, the prioritized list of degradation behaviors can include first, pulling over to the shoulder; second, slowing down to less than a threshold speed and transferring control of the vehicle to a teleoperator; and, third, performing a controlled stop in the lane in which the vehicle is traveling. In this example, Block S220 can include determining the failure tier is the first tier based on the subset of task nodes of each of the degradation behaviors of the prioritized list of degradation behaviors sharing at least one task node with the list of nonfunctional task nodes (e.g., none of the prioritized list of degradation behaviors can be safely executed to bring the vehicle to a controlled stop or otherwise under the control of a human operator). However, in alternative variations and/or examples, the prioritized list of degradation behaviors can include any other suitable tasks, and/or any number of suitable tasks (e.g., more than three, fewer than three, etc.).
  • In another variation, Block S220 includes determining that the failure tier is the second tier (e.g., a tier II failure) based on the number of failure flags received within a predetermined time window from a single task node. In another variation, Block S220 includes determining that the failure tier is the second tier (e.g., a tier II failure) based on the number of failure flags received from a unique set of task nodes (e.g., the number of failed task nodes). In another variation, Block S220 includes determining that the failure tier is the second tier (e.g., a tier II failure) based on the number of task nodes subscribing to the output of the failing task node (e.g., which generated the failure flag) exceeding a threshold number. However, Block S220 can additionally or alternatively include otherwise suitably determining the failure tier.
  • Block S230 includes selecting a failure action (e.g., response action, failure response action, etc.) based on the failure tier. Block S230 functions to determine a suitable response to a system or component failure based on the type and/or severity of the failure (e.g., as defined by the failure tier). The failure action can include an irreversible action, a controlled action, and/or a logging action, substantially as described above in relation to tier I failure responses, tier II failure responses, and tier III failure responses, respectively. However, selecting the failure action can additionally or alternatively include selecting any other suitable action in response to a failure of any suitable type. Block S230 is preferably performed by a behavior planning node substantially as described above, but can additionally or alternatively be performed by any suitable system or system component.
  • Block S240 includes executing the failure action (e.g., response action failure response action, etc.). Block S240 functions to implement the selected action (e.g., selected in accordance with Block S230). Block S240 is preferably implemented by a behavior planning node in conjunction with an actuation subsystem, substantially as described above. However, Block S240 can additionally or alternatively be performed by any suitable system or system component. In some variations, Block S240 can include recording and/or reporting system and/or component failure (e.g., at an onboard database, to a remote operator, etc.).
  • In variations, Block S240 includes executing the response action associated with the first tier and the second tier, which includes navigating the vehicle along a roadway. In related variations, Block S240 includes executing the response action associated with the third tier, which includes recording the failure flag and the corresponding task node generating the failure flag.
  • In further variations, Block S240 includes executing the irreversible response action, which includes controlling the vehicle to stop in the roadway in direct response to determining the failure tier is the first tier, and wherein executing the reversible response action comprises controlling the vehicle to traverse the roadway and stop proximal a side region adjacent to the roadway in direct response to determining the failure tier is the second tier.
  • However, in additional or alternative variations, Block S240 can include executing any other suitable action in response to a detected failure of any suitable tier or severity.
  • In a specific example, the method for managing component failure in a vehicle control system of a vehicle includes: at each of a set of task nodes of the vehicle control system: subscribing to input from at least one of a first task node of the set of task nodes and a sensor of a perception subsystem of the vehicle, transforming the input into an output according to a set of predetermined rules associated with the task node, publishing the output to at least one of a second task node of the set of task nodes and an actuator of an actuation subsystem of the vehicle, and comparing at least one of the subscription input and the published output to a failure criterion to generate a failure flag; at an aggregator node directly communicatively coupled to each of the set of task nodes: receiving a set of failure flags from the set of task nodes; determining a failure tier based on the set of failure flags, wherein the failure tier is one of a first tier, a second tier, and a third tier; determining a response action based on the failure tier, wherein the first tier corresponds to an irreversible response action, the second tier corresponds to a reversible response action, and the third tier corresponds to a logging action; and executing the response action corresponding to the failure tier.
  • The method 200 can additionally or alternatively include any other suitable Blocks, such as blocks implementing any functionality of a system substantially identical to the control system 100 described above in Section 3.
  • The system and methods of the preferred embodiment and variations thereof can be embodied and/or implemented at least in part as a machine configured to receive a computer-readable medium storing computer-readable instructions. The instructions are preferably executed by computer-executable components preferably integrated with the system. The computer-readable medium can be stored on any suitable computer-readable media such as RAMs, ROMs, flash memory, EEPROMs, optical devices (CD or DVD), hard drives, floppy drives, or any suitable device. The computer-executable component is preferably a general or application specific processor, but any suitable dedicated hardware or hardware/firmware combination device can alternatively or additionally execute the instructions.
  • Although omitted for conciseness, the preferred embodiments include every combination and permutation of the various system components, and the various method blocks.
  • As a person skilled in the art will recognize from the previous detailed description and from the figures and claims, modifications and changes can be made to the preferred embodiments of the invention without departing from the scope of this invention defined in the following claims.

Claims (20)

What is claimed is:
1. A vehicle control system for controlling a vehicle, comprising:
a set of task nodes, wherein each of the set of task nodes:
is communicatively coupled to and receives subscription input from at least one of a task node of the set of task nodes and a sensor of a perception subsystem of the vehicle;
compares a failure criterion to the subscription input to generate a failure flag and transforms the subscription input into published output, and
provides the published output to at least one of another task node of the set of task nodes and an actuator of an actuation subsystem of the vehicle;
an aggregator node directly communicatively coupled to each of the set of task nodes, and wherein the aggregator node:
receives a set of failure flags from the set of task nodes,
determines a failure tier based on the set of failure flags, wherein the failure tier is one of a first tier, a second tier, and a third tier, wherein the first tier is associated with an irreversible response action, the second tier is associated with a reversible response action, and the third tier is associated with a logging action, and
generates a failure tier message defining the failure tier;
a behavior planning node communicatively coupled to the aggregator node, wherein the behavior planning node:
receives the failure tier message, and
in response to the failure tier defined by the failure tier message, generates vehicle control instructions corresponding to one of the logging action, the reversible response action, and the irreversible response action.
2. The system of claim 1, wherein the aggregator node determines the failure tier based on a number of failure flags of the set of failure flags received within a predetermined time window.
3. The system of claim 1, wherein the aggregator node generates a list of nonfunctional task nodes from the set of task nodes based on the set of failure flags, and determines the failure tier based on the list of nonfunctional task nodes.
4. The system of claim 3, wherein the list of nonfunctional task nodes associated with the first tier comprises at least one task node wherein the published output of the at least one task node controls the actuator of the actuation subsystem.
5. The system of claim 1, wherein the aggregator node determines the failure tier in response to a task node of the set of task nodes generating a failure flag for a time period exceeding a threshold duration.
6. The system of claim 5, wherein the threshold duration is inversely proportional to a number of task nodes that receive the published output of the task node.
7. The system of claim 1, wherein the actuation subsystem is controlled by the behavior planning node in combination with the set of task nodes, and wherein the actuation subsystem actuates control inputs of the vehicle to navigate the vehicle along a roadway in response to the vehicle control instructions.
8. The system of claim 7, wherein the vehicle control instructions corresponding to the reversible response action actuate a steering actuator of the actuation subsystem and a brake actuator of the actuation subsystem to bring the vehicle to a stopped condition at a righthand side of the roadway.
9. The system of claim 8, wherein the vehicle control instructions corresponding to the irreversible response action actuate the brake actuator to bring the vehicle to the stopped condition at a current instantaneous lateral position of the vehicle in the roadway, and wherein the irreversible response action is executed immediately upon receipt of the failure message defining the failure tier corresponding to the irreversible response action.
10. A method for managing component failure in a vehicle control system of a vehicle, comprising:
at each of a set of task nodes of the vehicle control system:
subscribing to input from at least one of a first task node of the set of task nodes and a sensor of a perception subsystem of the vehicle,
transforming the input into an output according to a set of predetermined rules associated with the task node,
publishing the output to at least one of a second task node of the set of task nodes and an actuator of an actuation subsystem of the vehicle, and
comparing at least one of the subscription input and the published output to a failure criterion to generate a failure flag;
at an aggregator node directly communicatively coupled to each of the set of task nodes:
receiving a set of failure flags from the set of task nodes;
determining a failure tier based on the set of failure flags, wherein the failure tier is one of a first tier, a second tier, and a third tier;
determining a response action based on the failure tier, wherein the first tier corresponds to an irreversible response action, the second tier corresponds to a reversible response action, and the third tier corresponds to a logging action; and
executing the response action corresponding to the failure tier.
11. method of claim 10, wherein determining the failure tier based on the set of failure flags comprises determining that the failure tier is the first tier based on receiving at least one of the failure flags from a task node publishing output to the actuator.
12. The method of claim 10, wherein determining the failure tier based on the set of failure flags comprises determining that the failure tier is the second tier based on receiving a failure flag from a task node of the set of task nodes for a time period exceeding a threshold duration.
13. The method of claim 12, wherein the threshold duration is proportional to a speed of the vehicle.
14. The method of claim 10, further comprising generating a list of nonfunctional task nodes based on the set of failure flags, and determining the failure tier based on the list of nonfunctional task nodes.
15. The method of claim 14, wherein determining the failure tier is the first tier comprises determining that the list of nonfunctional task nodes comprises at least one task node that publishes the output to the actuator of the actuation subsystem.
16. The method of claim 14, further comprising determining the failure tier based on comparing the list of nonfunctional task nodes to a prioritized list of degradation behaviors, wherein each degradation behavior requires a subset of task nodes of the set of task nodes for execution.
17. The method of claim 16, further comprising determining the failure tier is the first tier based on the subset of task nodes of each of the degradation behaviors of the prioritized list of degradation behaviors sharing at least one task node with the list of nonfunctional task nodes.
18. The method of claim 10, wherein determining the failure tier based on the set of failure flags comprises determining that the failure tier is the second tier based on a number of failure flags of the set of failure flags received within a predetermined time window.
19. The method of claim 10, wherein executing the response action associated with the first tier and the second tier comprises navigating the vehicle along a roadway, and wherein executing the response action associated with the third tier comprises recording the failure flag and the corresponding task node generating the failure flag.
20. The method of claim 19, wherein executing the irreversible response action comprises controlling the vehicle to stop in the roadway in direct response to determining the failure tier is the first tier, and wherein executing the reversible response action comprises controlling the vehicle to traverse the roadway and stop proximal a side region adjacent to the roadway in direct response to determining the failure tier is the second tier.
US16/409,308 2018-05-10 2019-05-10 Vehicle control system and method Abandoned US20190344799A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US16/409,308 US20190344799A1 (en) 2018-05-10 2019-05-10 Vehicle control system and method

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US201862669586P 2018-05-10 2018-05-10
US16/409,308 US20190344799A1 (en) 2018-05-10 2019-05-10 Vehicle control system and method

Publications (1)

Publication Number Publication Date
US20190344799A1 true US20190344799A1 (en) 2019-11-14

Family

ID=68463935

Family Applications (2)

Application Number Title Priority Date Filing Date
US17/259,151 Pending US20210155253A1 (en) 2018-05-10 2019-05-10 Vehicle control system and method
US16/409,308 Abandoned US20190344799A1 (en) 2018-05-10 2019-05-10 Vehicle control system and method

Family Applications Before (1)

Application Number Title Priority Date Filing Date
US17/259,151 Pending US20210155253A1 (en) 2018-05-10 2019-05-10 Vehicle control system and method

Country Status (2)

Country Link
US (2) US20210155253A1 (en)
WO (1) WO2019217859A1 (en)

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20220063641A1 (en) * 2020-08-27 2022-03-03 Hyundai Motor Company Device and method for detecting failure of actuator of vehicle
CN114348025A (en) * 2022-01-30 2022-04-15 中国第一汽车股份有限公司 Vehicle driving monitoring system, method, equipment and storage medium
US11334081B2 (en) * 2020-03-20 2022-05-17 Ford Global Technologies, Llc Vehicle anomalous-condition response during autonomous driving
US20220204003A1 (en) * 2019-05-07 2022-06-30 Kontrol Gmbh Formal Verification for the Development and Real-Time Application of Autonomous Systems
US11427175B2 (en) 2019-12-16 2022-08-30 Waymo Llc Vehicle braking systems and methods
US20220289242A1 (en) * 2021-03-12 2022-09-15 Honda Motor Co., Ltd. Movement control system and moving object
US11797294B2 (en) * 2021-03-05 2023-10-24 Toyota Jidosha Kabushiki Kaisha In-vehicle device, information processing device, download execution feasibility determination method, program, recording medium
US11919526B1 (en) * 2021-04-28 2024-03-05 Zoox, Inc. Model-based vehicle drive system monitors

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20230176573A1 (en) * 2021-12-06 2023-06-08 Gatik Ai Inc. Method and system for operating an autonomous agent with a remote operator

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102006017824B4 (en) * 2006-04-13 2018-10-11 Dspace Digital Signal Processing And Control Engineering Gmbh Method for constructing a diagnostic function
US8868447B1 (en) * 2008-05-10 2014-10-21 Charlotte Intellectual Properties, LLC Sensor and control node publishing and subscription system
US8618922B2 (en) * 2010-03-30 2013-12-31 GM Global Technology Operations LLC Method and system for ensuring operation of limited-ability autonomous driving vehicles
US9275093B2 (en) * 2011-01-28 2016-03-01 Cisco Technology, Inc. Indexing sensor data
US20130018692A1 (en) * 2011-07-13 2013-01-17 Siemens Aktiengesellschaft Apparatus, method, and computer program product for scenario-based identification of complete safety-based requirements specification
US10397019B2 (en) * 2015-11-16 2019-08-27 Polysync Technologies, Inc. Autonomous vehicle platform and safety architecture
CN110291477B (en) * 2016-12-02 2022-08-16 嘉蒂克人工智能公司 Vehicle control system and method of use

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20220204003A1 (en) * 2019-05-07 2022-06-30 Kontrol Gmbh Formal Verification for the Development and Real-Time Application of Autonomous Systems
US11427175B2 (en) 2019-12-16 2022-08-30 Waymo Llc Vehicle braking systems and methods
US11334081B2 (en) * 2020-03-20 2022-05-17 Ford Global Technologies, Llc Vehicle anomalous-condition response during autonomous driving
US20220063641A1 (en) * 2020-08-27 2022-03-03 Hyundai Motor Company Device and method for detecting failure of actuator of vehicle
US11858522B2 (en) * 2020-08-27 2024-01-02 Hyundai Motor Company Device and method for detecting failure of actuator of vehicle
US11797294B2 (en) * 2021-03-05 2023-10-24 Toyota Jidosha Kabushiki Kaisha In-vehicle device, information processing device, download execution feasibility determination method, program, recording medium
US20220289242A1 (en) * 2021-03-12 2022-09-15 Honda Motor Co., Ltd. Movement control system and moving object
US11919526B1 (en) * 2021-04-28 2024-03-05 Zoox, Inc. Model-based vehicle drive system monitors
CN114348025A (en) * 2022-01-30 2022-04-15 中国第一汽车股份有限公司 Vehicle driving monitoring system, method, equipment and storage medium

Also Published As

Publication number Publication date
WO2019217859A1 (en) 2019-11-14
US20210155253A1 (en) 2021-05-27

Similar Documents

Publication Publication Date Title
US20190344799A1 (en) Vehicle control system and method
US10710602B2 (en) Systems and methods for a vehicle controller safety monitor
CN109863500B (en) Event driven region of interest management
US10437247B2 (en) Multi-stage operation of autonomous vehicles
CN109421738B (en) Method and apparatus for monitoring autonomous vehicles
CN110268413B (en) Low level sensor fusion
US11560690B2 (en) Techniques for kinematic and dynamic behavior estimation in autonomous vehicles
US20200209848A1 (en) Service degradation in an autonomous driving system
US10532740B2 (en) Method and arrangement for monitoring and adapting the performance of a fusion system of an autonomous vehicle
US9290186B2 (en) Messaging via vehicle steering wheel
EP3577528B1 (en) Enabling remote control of a vehicle
US20200057441A1 (en) Systems and methods for intelligent arbitration between autonomous and manual operation signals of an autonomous agent
WO2020259705A1 (en) Autonomous driving handoff systems and methods
US11724708B2 (en) Fail-safe handling system for autonomous driving vehicle
CN104108395A (en) Method And Device For Modifying The Configuration Of A Driving Assistance System Of A Motor Vehicle
CN108614566B (en) Operation method for parallel driving
JP2018205940A (en) Mobile body behavior prediction device
CN111897304B (en) Method, apparatus and system for real-time diagnostics and fault monitoring in a machine system
US20220254204A1 (en) Checkpoint-Based Tracing for Monitoring a Robotic System
US20220153293A1 (en) Method and device for operating an automated vehicle
US20220274628A1 (en) A method for quantifying extreme traffic behavior
US20240067216A1 (en) Verification of vehicle prediction function
EP4177775A1 (en) A method for quantifying object detection performance
WO2023043305A1 (en) An autonomous vehicle and a method of operation thereof
CN116893948A (en) Computer-implemented system and method for monitoring functional capabilities of an autopilot function

Legal Events

Date Code Title Description
AS Assignment

Owner name: STARSKY ROBOTICS, INC., CALIFORNIA

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:TIWARI, KARTIK;REEL/FRAME:049752/0697

Effective date: 20190710

STPP Information on status: patent application and granting procedure in general

Free format text: NON FINAL ACTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER

STPP Information on status: patent application and granting procedure in general

Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS

STCB Information on status: application discontinuation

Free format text: ABANDONED -- FAILURE TO PAY ISSUE FEE

AS Assignment

Owner name: GATIK AI INC., CALIFORNIA

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:STARSKY ROBOTICS, INC.;REEL/FRAME:054956/0773

Effective date: 20201216

AS Assignment

Owner name: STARSKY ROBOTICS (ABC), LLC, CALIFORNIA

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:STARSKY ROBOTICS, INC.;REEL/FRAME:057992/0294

Effective date: 20200527

Owner name: GATIK AI INC., CALIFORNIA

Free format text: CORRECTIVE ASSIGNMENT TO CORRECT THE NAME OF ASSIGNOR PREVIOUSLY RECORDED ON REEL 054956 FRAME 0773. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT;ASSIGNOR:STARSKY ROBOTICS (ABC), LLC;REEL/FRAME:057991/0844

Effective date: 20201216