JP2020504860A - Navigation systems with imposed liability constraints - Google Patents

Abstract

Systems and methods are provided for navigating a host vehicle. In some embodiments, the system receives at least one image representative of the environment of the host vehicle from the image capture device and performs at least one driving operation of the planned navigation action to achieve the host vehicle's navigation goals. Determining based on the policy, analyzing the at least one image to identify a target vehicle in the environment of the host vehicle, and determining a potential accident liability of the host vehicle with respect to the identified target vehicle. Testing the planned navigation action against at least one accident liability rule, and testing the planned navigation action against the at least one accident liability rule is performed on the host vehicle when the planned navigation action is performed. Planned navigation should be used to indicate potential accident liability. That the host vehicle does not perform any navigation actions and that testing of the planned navigation actions for at least one accident liability rule will not result in accident liability for the host vehicle if the planned navigation actions are performed. If indicated, it may include at least one processing device programmed to cause the host vehicle to perform a planned navigation operation.

Description

Cross-reference of related applications
[001] This application is related to US Provisional Patent Application No. 62 / 438,563, filed December 23, 2016, US Provisional Patent Application No. 62 / 546,343, filed August 16, 2017, Claims the priority benefit of US Provisional Patent Application No. 62 / 565,244 filed September 29, 2017 and US Provisional Patent Application No. 62 / 582,687 filed November 7, 2017. Is what you do. All of the above applications are incorporated herein by reference in their entirety.

Background technology field
[002] The present disclosure relates generally to autonomous vehicle navigation. In addition, the present disclosure relates to systems and methods for navigating according to potential (possible) accident liability constraints.

Background information
[003] As technology continues to evolve, the goal of a fully autonomous vehicle navigable on the road is becoming a reality. An autonomous vehicle may need to consider various factors and, based on those factors, may make appropriate decisions to safely and accurately reach the intended destination. For example, an autonomous vehicle may need to process and interpret visual information (eg, information captured from a camera), radar, or lidar, as well as other sources (eg, GPS devices, speed sensors). , Accelerometers, suspension sensors, etc.). At the same time, to navigate to the destination, the autonomous vehicle identifies its location on a particular road (eg, a particular lane in a multi-lane road), navigates alongside other vehicles, Avoid objects and pedestrians, observe traffic signals and signs, go from one road to another at appropriate intersections or interchanges, and respond to any other situations that occur or evolve during vehicle operation It may be necessary. Further, navigation systems may have to adhere to certain imposed constraints. In some cases, those constraints may relate to interactions between the host vehicle and one or more other objects, such as other vehicles or pedestrians. In other cases, those constraints may relate to liability rules to be followed when performing one or more navigation operations for the host vehicle.

  [004] In the field of autonomous driving, there are two important considerations for a viable autonomous vehicle system. The first consideration is the standardization of security assurance, including the requirements that all autonomous vehicles must meet to ensure safety, and how those requirements can be verified. The second consideration is scalability, because the engineering solutions that drive up costs do not scale to millions of vehicles, but rather the widespread adoption of autonomous vehicles or Furthermore, even less widespread recruitment can be hindered. Therefore, there is a need for an interpretable mathematical model for security and the design of a system that is scalable to millions of vehicles while complying with security requirements.

Overview
[005] Embodiments according to the present disclosure provide systems and methods for autonomous vehicle navigation. The disclosed embodiments may use a camera to provide autonomous vehicle navigation features. For example, according to embodiments of the present disclosure, the disclosed system may include one, two, or more than one camera that monitors the environment of the vehicle. The disclosed system may provide a navigation response based on, for example, an analysis of images captured by one or more of the cameras. The navigation response may also take into account other data, including, for example, global positioning (GPS) data, sensor data (eg, from accelerometers, speed sensors, suspension sensors, etc.), and / or other map data.

  [006] Systems and methods for navigating a host vehicle are provided. In some embodiments, the system receives at least one image representative of the environment of the host vehicle from the image capture device and performs at least one driving operation of the planned navigation action to achieve the host vehicle's navigation goals. Determining based on the policy, analyzing at least one image to identify a target vehicle in the environment of the host vehicle, and determining a potential accident liability of the host vehicle with respect to the identified target vehicle. Testing the planned navigation action against at least one accident liability rule, and testing the planned navigation action against the at least one accident liability rule, for the host vehicle when the planned navigation action is performed Planned navigation should be used to indicate potential accident liability. That the host vehicle does not perform any of the planned navigation actions, and that testing of the planned navigation actions for at least one accident liability rule will not result in accident liability for the host vehicle if the planned navigation actions are performed. If indicated, it may include at least one processing device programmed to cause the host vehicle to perform the planned navigation operation.

  [007] In some embodiments, a navigation system for a host vehicle receives at least one image representing an environment of the host vehicle from an image capture device, and a plurality of potential images for the host vehicle. Determining a (possible) navigation action based on at least one driving policy; analyzing at least one image to identify a target vehicle in the environment of the host vehicle; and a host vehicle for the identified target vehicle. Testing a plurality of potential navigation actions against at least one accident liability rule, and selecting one of the potential navigation actions to determine a potential accident liability of For that one of the potential navigation actions, the test is executed when the selected potential navigation action is taken. Including at least one processing device programmed to perform a selection and to cause the host vehicle to perform the selected potential navigation action, indicating that no liability will occur for the strike vehicle. obtain.

  [008] In some embodiments, a system for navigating a host vehicle includes receiving at least one image representative of an environment of the host vehicle from an image capture device and achieving a navigation goal for the host vehicle. Determining at least one planned navigation behavior based on at least one driving policy, analyzing at least one image to identify a target vehicle within the environment of the host vehicle, Testing each of the two or more planned navigation actions against at least one accident liability rule to determine responsibilities; and for each of the two or more planned navigation actions, testing Potential accident liability for the host vehicle if a particular one of more than one planned navigation action is taken If there is, the host vehicle is not allowed to perform a particular one of the planned navigation actions, and for each of the two or more planned navigation actions, a test is performed for two or more planned navigation actions. If one of the actions indicates that no accident liability will occur for the host vehicle, a particular one of the two or more planned navigation actions is a viable candidate for implementation. And selecting a navigation action to be performed from among the viable candidates for implementation based on at least one cost function, and causing the host vehicle to perform the selected navigation action. At least one processing device programmed as follows.

  [009] In some embodiments, the accident liability tracking system for the host vehicle receives at least one image representative of the environment of the host vehicle from an image capture device, and analyzes the at least one image to analyze the host vehicle. Identifying a target vehicle in the environment of the vehicle; determining one or more characteristics of a navigation state of the identified target vehicle based on analysis of the at least one image; and navigating the identified target vehicle. Comparing the determined one or more characteristics of the state to at least one accident liability rule; and comparing the determined one or more characteristics of the identified target vehicle navigation state to the at least one accident liability rule. Storing at least one value indicative of the potential accident liability of the identified target vehicle based on the comparison of After an accident with one target vehicle, it may include at least one processing device programmed to output at least one stored value to determine responsibility for the accident.

  [010] In some embodiments, a system for navigating a host vehicle includes receiving at least one image representative of an environment of the host vehicle from an image capture device and achieving a navigation goal for the host vehicle. Determining at least one planned navigation behavior based on at least one driving policy, analyzing at least one image to identify a target vehicle in the environment of the host vehicle, and determining potential accident liability. Testing the planned navigation action against at least one accident liability rule, and testing the planned navigation action against the at least one accident liability rule, when the planned navigation action occurs. Planned navigation to indicate potential liability for the vehicle That the host vehicle does not perform any of the planned navigation actions, and that testing of the planned navigation actions for at least one accident liability rule will not result in accident liability for the host vehicle if the planned navigation actions are performed. When indicated, it may include at least one processing device programmed to cause the host vehicle to perform a planned navigation operation, wherein the at least one processing device is based on an analysis of the at least one image, Determining one or more characteristics of the identified target vehicle navigation state and comparing the determined one or more characteristics of the identified target vehicle navigation state to at least one accident liability rule. And the determined one of the navigation states of the identified target vehicle Based on a comparison of the at least one accident liability rules more properties, is further programmed to perform the method comprising: storing at least one value indicating a potential accident responsibility of the identified side of the target vehicle.

  [011] In some embodiments, a system for navigating a host vehicle includes receiving at least one image representative of an environment of the host vehicle from an image capture device, and achieving a navigation goal for the host vehicle. Determining at least one planned navigation action based on at least one driving policy; analyzing at least one image to identify a target vehicle within the environment of the host vehicle; and performing the planned navigation action. Determining the distance of the next state between the host vehicle and the target vehicle, which would occur if Determining the current speed of the vehicle and assuming a maximum braking capability of the target vehicle based on at least one recognized characteristic of the target vehicle Given the maximum braking capacity of the host vehicle and the current speed of the host vehicle, the distance traveled by the target vehicle determined based on the current speed of the target vehicle and the assumed maximum braking capacity of the target vehicle At least one processing device programmed to perform a planned navigation operation if the host vehicle can be stopped within a stopping distance that is less than the determined next state distance sum. May be included.

  [012] In some embodiments, a system for navigating a host vehicle includes receiving at least one image representing an environment of the host vehicle from an image capture device and achieving a navigation goal for the host vehicle. Determining a planned navigation behavior of the vehicle based on at least one driving policy, and analyzing the at least one image to determine a first target vehicle ahead of the host vehicle and a second target vehicle ahead of the first target vehicle. Identifying the next target vehicle between the host vehicle and the second target vehicle, which will occur when a planned navigation operation is performed; and Determining the current maximum braking capability of the host vehicle and the current speed of the host vehicle, and assuming the maximum braking capability of the host vehicle and the current speed of the host vehicle as given. Performing the planned navigation action if the host vehicle can be stopped within a stopping distance that is less than the determined next state distance between the host vehicle and the second target vehicle. It may include at least one processing device to be programmed.

  [013] In some embodiments, a system for navigating a host vehicle includes receiving at least one image representative of an environment of the host vehicle from an image capture device, analyzing the at least one image, and analyzing the at least one image. For identifying a target vehicle in the environment of the vehicle, determining two or more potential navigation actions to achieve the host vehicle's navigation goals, and for each of the two or more potential navigation actions. Testing each of the two or more potential navigation actions to at least one accident liability rule to determine an indicator of potential accident liability between the host vehicle and the identified target vehicle; Selecting one of two or more potential navigation actions for implementation, wherein the one is associated with the selected action. Selecting the one only if the potential accident liability indicator indicates that no potential accident liability is associated with the host vehicle as a result of performing the selected action. It may include at least one processing device to be programmed.

  [014] In some embodiments, a system for navigating a host vehicle includes receiving at least one image representative of an environment of the host vehicle from an image capture device, and an indicator of a current navigation state of the host vehicle. Determining from the at least one sensor and analyzing the at least one image and an indicator of a current navigational state of the host vehicle that a collision between the host vehicle and one or more objects is inevitable. Performing a first planned navigation action for the host vehicle with an expected collision with the first object and an expected collision with the second object based on at least one driving policy. Determining a second planned navigation action for the accompanying host vehicle and determining potential accident liability Testing the first planned navigation action and the second planned navigation action against at least one accident liability rule; and detecting the first planned navigation action against the at least one accident liability rule. If the test indicates that there is potential accident liability for the host vehicle when the first planned navigation operation is performed, then causing the host vehicle to not perform the first planned navigation operation; If the test of the second planned navigation action for one accident liability rule indicates that no accident liability will occur for the host vehicle if the second planned navigation action is performed, the second planned Causing the host vehicle to perform the selected navigation operation. It may include at least one processing device is.

  [015] According to another disclosed embodiment, the non-transitory computer-readable storage medium is executable by at least one processing device and implements any of the steps and / or methods described herein. Program instructions to be executed may be stored.

  [0161] The above summary and the following detailed description are merely exemplary and explanatory and are not restrictive of the claims.

BRIEF DESCRIPTION OF THE FIGURES
[017] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate various disclosed embodiments.

[018] FIG. 18 is a diagrammatic representation of an exemplary system according to the disclosed embodiments. [019] FIG. 9 is a side view representation of an exemplary vehicle including a system according to the disclosed embodiments. [020] FIG. 2B is a top view representation of the vehicle and system shown in FIG. 2A according to a disclosed embodiment; [021] FIG. 26 is a top view representation of another embodiment of a vehicle including a system according to the disclosed embodiments. [022] FIG. 21 is a top view representation of yet another embodiment of a vehicle including a system according to the disclosed embodiments. [023] FIG. 26 is a top view representation of yet another embodiment of a vehicle including a system according to the disclosed embodiments. 1 is a diagrammatic representation of an exemplary vehicle control system according to a disclosed embodiment; [025] FIG. 14 is a diagrammatic representation of the interior of a vehicle including a rearview mirror and a user interface of a vehicle imaging system according to the disclosed embodiments. [026] FIG. 25 is an illustration of an example of a camera mount configured to be positioned behind a rearview mirror and opposite a vehicle windshield according to a disclosed embodiment. [027] FIG. 3B is an illustration of the camera mount shown in FIG. 3B from a different perspective, according to a disclosed embodiment. [028] FIG. 28 is an illustration of an example of a camera mount configured to be positioned opposite a vehicle windshield behind a rearview mirror, according to a disclosed embodiment; [029] FIG. 9 is an exemplary block diagram of a memory configured to store instructions for performing one or more operations according to the disclosed embodiments. [030] FIG. 10 is a flowchart illustrating an exemplary process for generating one or more navigation responses based on monocular image analysis, according to a disclosed embodiment; [031] FIG. 10 is a flowchart illustrating an example process for detecting one or more vehicles and / or pedestrians in a set of images, according to a disclosed embodiment; [032] FIG. 16 is a flowchart illustrating an exemplary process for detecting road marks and / or lane geometry information in a set of images, according to a disclosed embodiment; [033] FIG. 16 is a flowchart illustrating an exemplary process for detecting traffic lights in a set of images, according to a disclosed embodiment; [034] FIG. 10 is a flowchart of an exemplary process for generating one or more navigation responses based on a vehicle path, according to a disclosed embodiment; [035] FIG. 16 is a flowchart illustrating an exemplary process for determining whether a preceding vehicle is changing lanes, in accordance with a disclosed embodiment. [036] FIG. 40 is a flowchart illustrating an exemplary process for generating one or more navigation responses based on stereoscopic image analysis, in accordance with a disclosed embodiment. [037] FIG. 7 is a flowchart illustrating an exemplary process for generating one or more navigation responses based on an analysis of three sets of images, according to a disclosed embodiment; [038] FIG. 40 is a block diagram representation of a module that may be implemented by one or more specifically programmed processing devices of a navigation system for an autonomous vehicle, according to a disclosed embodiment. [039] FIG. 6 is a graph of navigation options according to a disclosed embodiment; [040] FIG. 40 is a graph of navigation options according to a disclosed embodiment; [041] FIG. 16 illustrates a schematic diagram of a navigation option for a host vehicle in a junction area according to a disclosed embodiment; [041] FIG. 16 illustrates a schematic diagram of a navigation option for a host vehicle in a junction area according to a disclosed embodiment; [041] FIG. 16 illustrates a schematic diagram of a navigation option for a host vehicle in a junction area according to a disclosed embodiment; [042] FIG. 4 illustrates a graphical representation of a dual polymerization flow scenario according to a disclosed embodiment. [043] Figure 4 illustrates a graph of potentially useful options in a dual polymerization flow scenario according to disclosed embodiments. [044] FIG. 41 illustrates a diagram of a representative image captured of a host vehicle environment with potential navigation constraints, according to a disclosed embodiment. [045] FIG. 9 illustrates a flowchart of an algorithm for navigating a vehicle, according to a disclosed embodiment; [046] FIG. 7 illustrates a flowchart of an algorithm for navigating a vehicle, according to a disclosed embodiment; [047] FIG. 9 illustrates a flowchart of an algorithm for navigating a vehicle, according to a disclosed embodiment; 4 illustrates a flowchart of an algorithm for navigating a vehicle, according to a disclosed embodiment. [049] FIG. 10 illustrates a view of a host vehicle navigating in a roundabout, according to a disclosed embodiment. [049] FIG. 10 illustrates a view of a host vehicle navigating in a roundabout, according to a disclosed embodiment. [050] FIG. 9 illustrates a flowchart of an algorithm for navigating a vehicle, according to a disclosed embodiment; [051] FIG. 14 illustrates an example of a host vehicle traveling on a multi-lane highway, according to a disclosed embodiment. [052] FIG. 18 illustrates an example of a vehicle interrupting another vehicle in accordance with a disclosed embodiment. [052] FIG. 18 illustrates an example of a vehicle interrupting another vehicle in accordance with a disclosed embodiment. [053] FIG. 52 illustrates an example of a vehicle following another vehicle, according to a disclosed embodiment. [054] FIG. 18 illustrates an example of a vehicle exiting a parking lot and merging on a potentially busy road, according to a disclosed embodiment. [055] FIG. 55 illustrates a vehicle traveling on a road, according to a disclosed embodiment. [056] shows an example of a scenario according to the disclosed embodiments. [056] shows an example of a scenario according to the disclosed embodiments. [056] shows an example of a scenario according to the disclosed embodiments. [056] shows an example of a scenario according to the disclosed embodiments. [057] shows an example of a scenario according to the disclosed embodiments. [058] FIG. 19 illustrates an example of a scenario according to the disclosed embodiments. [059] shows an example of a scenario according to the disclosed embodiments. [060] FIG. 10 illustrates an example of a scenario in which a vehicle follows another vehicle according to a disclosed embodiment. [060] FIG. 10 illustrates an example of a scenario in which a vehicle follows another vehicle according to a disclosed embodiment. [061] FIG. 14 illustrates an example of negligence in an interrupt scenario, according to a disclosed embodiment. [061] FIG. 14 illustrates an example of negligence in an interrupt scenario, according to a disclosed embodiment. [062] FIG. 14 illustrates an example of negligence in an interrupt scenario, according to a disclosed embodiment. [062] FIG. 14 illustrates an example of negligence in an interrupt scenario, according to a disclosed embodiment. [063] FIG. 14 illustrates an example of negligence in a drift scenario, according to disclosed embodiments. [063] FIG. 14 illustrates an example of negligence in a drift scenario, according to disclosed embodiments. [063] FIG. 14 illustrates an example of negligence in a drift scenario, according to disclosed embodiments. [063] FIG. 14 illustrates an example of negligence in a drift scenario, according to disclosed embodiments. [064] FIG. 9 illustrates an example of negligence in a two-way traffic scenario, according to a disclosed embodiment. [064] FIG. 9 illustrates an example of negligence in a two-way traffic scenario, according to a disclosed embodiment. [065] FIG. 9 illustrates an example of negligence in a root priority scenario, in accordance with the disclosed embodiments. [065] FIG. 9 illustrates an example of negligence in a root priority scenario, in accordance with the disclosed embodiments. [066] FIG. 9 illustrates an example of negligence in a root priority scenario, in accordance with a disclosed embodiment. [066] FIG. 9 illustrates an example of negligence in a root priority scenario, in accordance with a disclosed embodiment. [067] FIG. 9 illustrates an example of negligence in a root priority scenario, according to a disclosed embodiment. [067] FIG. 9 illustrates an example of negligence in a root priority scenario, according to a disclosed embodiment. [068] FIG. 9 illustrates an example of negligence in a route priority scenario, according to a disclosed embodiment. [068] FIG. 9 illustrates an example of negligence in a route priority scenario, according to a disclosed embodiment. [069] FIG. 9 illustrates an example of negligence in a root priority scenario, in accordance with a disclosed embodiment. [069] FIG. 9 illustrates an example of negligence in a root priority scenario, in accordance with a disclosed embodiment. [070] FIG. 7 illustrates an example of negligence in a root priority scenario, in accordance with a disclosed embodiment. [070] FIG. 7 illustrates an example of negligence in a root priority scenario, in accordance with a disclosed embodiment. [071] FIG. 7 illustrates an example of negligence in a traffic light scenario, according to a disclosed embodiment. [071] FIG. 7 illustrates an example of negligence in a traffic light scenario, according to a disclosed embodiment. [072] FIG. 7 illustrates an example of negligence in a traffic light scenario, in accordance with the disclosed embodiments. [072] FIG. 7 illustrates an example of negligence in a traffic light scenario, in accordance with the disclosed embodiments. [073] FIG. 7 illustrates an example of negligence in a traffic light scenario, in accordance with a disclosed embodiment. [073] FIG. 7 illustrates an example of negligence in a traffic light scenario, in accordance with a disclosed embodiment. [074] FIG. 7 illustrates an example of a traffic vulnerable (VRU) scenario according to a disclosed embodiment. [074] FIG. 7 illustrates an example of a traffic vulnerable (VRU) scenario according to a disclosed embodiment. [074] FIG. 7 illustrates an example of a traffic vulnerable (VRU) scenario according to a disclosed embodiment. [075] FIG. 17 illustrates an example of a traffic vulnerable (VRU) scenario according to a disclosed embodiment. [075] FIG. 17 illustrates an example of a traffic vulnerable (VRU) scenario according to a disclosed embodiment. [075] FIG. 17 illustrates an example of a traffic vulnerable (VRU) scenario according to a disclosed embodiment. 9 illustrates an example of a traffic vulnerable (VRU) scenario according to a disclosed embodiment. 9 illustrates an example of a traffic vulnerable (VRU) scenario according to a disclosed embodiment. 9 illustrates an example of a traffic vulnerable (VRU) scenario according to a disclosed embodiment. [077] FIG. 17 illustrates an example of a traffic vulnerable (VRU) scenario according to a disclosed embodiment. [077] FIG. 17 illustrates an example of a traffic vulnerable (VRU) scenario according to a disclosed embodiment. [077] FIG. 17 illustrates an example of a traffic vulnerable (VRU) scenario according to a disclosed embodiment. [077] FIG. 17 illustrates an example of a traffic vulnerable (VRU) scenario according to a disclosed embodiment.

Detailed description
[078] The following detailed description refers to the accompanying drawings. Wherever possible, the same reference numbers will be used in the drawings and the description to refer to the same or like parts. Some example embodiments are described herein, but modifications, adaptations, and other implementations are possible. For example, substitutions, additions, or modifications to the components shown in the figures may be made, and the exemplary methods described herein may replace, reorder, delete, or replace the steps of the disclosed methods. It can be changed by addition. Accordingly, the following detailed description is not limited to the disclosed embodiments and examples. Instead, the proper scope is defined by the appended claims.

[079] Overview of autonomous vehicles
[080] As used throughout this disclosure, the term "autonomous vehicle" refers to a vehicle that can implement at least one navigation change without driver input. "Navigation change" refers to one or more changes in steering, braking, or accelerating / decelerating a vehicle. To be autonomous, the vehicle does not need to be fully automatic (eg, fully operational without a driver or driver input). Rather, autonomous vehicles include vehicles that can operate under the control of a driver during certain periods of time and operate without control of the driver during other periods of time. An autonomous vehicle may control only some aspects of vehicle navigation, such as steering, (eg, to maintain a vehicle course between vehicle lane constraints), or may operate under certain (but not all) circumstances. It may also include a vehicle that controls the steering operation while leaving other aspects (eg, braking or braking under certain circumstances) to the driver. In some cases, an autonomous vehicle may handle some or all aspects of vehicle braking, speed control, and / or steering.

  [081] Since human drivers usually rely on visual cues and observations to control vehicles, the traffic infrastructure is built accordingly, and lane markings, traffic signs, and traffic lights are Designed to provide information to drivers. In view of these design features of the traffic infrastructure, an autonomous vehicle may include a camera and a processing unit that analyzes visual information captured from the environment of the vehicle. The visual information is, for example, an image representing traffic infrastructure components (eg, lane marks, traffic signs, traffic lights, etc.) and other obstacles (eg, other vehicles, pedestrians, rubble, etc.) observable by the driver. May be included. Furthermore, motor vehicles can use stored information, such as information that provides a model of the vehicle's environment when navigating. For example, a vehicle may use GPS data, sensor data (e.g., from accelerometers, speed sensors, suspension sensors, etc.), and / or other map data to control the environment of the vehicle while the vehicle is traveling. Related information may be provided, and the vehicle (and other vehicles) may use the information to locate itself on the model. Some vehicles may also be capable of communicating between vehicles, sharing information, changing peer vehicles for dangers or changes around the vehicle, and the like.

[082] System overview
[083] FIG. 1 is a block diagram representation of a system 100 according to a disclosed exemplary embodiment. System 100 may include various components depending on particular implementation requirements. In some embodiments, system 100 includes a processing unit 110, an image acquisition unit 120, a position sensor 130, one or more memory units 140, 150, a map database 160, a user interface 170, and a wireless transceiver 172. obtain. Processing unit 110 may include one or more processing devices. In some embodiments, processing unit 110 may include application processor 180, image processor 190, or any other suitable processing device. Similarly, image acquisition unit 120 may include any number of image acquisition devices and components depending on the requirements of a particular application. In some embodiments, the image acquisition unit 120 includes one or more image capture devices (eg, a camera, a CCD, any other type of image capture device 122, an image capture device 124, an image capture device 126, etc.). Image sensor). The system 100 can also include a data interface 128 that communicatively connects the processing unit 110 to the image acquisition unit 120. For example, data interface 128 may include one or more optional wired and / or wireless links for transmitting image data acquired by image acquisition unit 120 to processing unit 110.

  [084] The wireless transceiver 172 is configured to exchange transmissions with one or more networks (e.g., cellular or Internet, etc.) over a wireless interface using radio frequency, infrared frequency, magnetic or electric fields. One or more devices. Wireless transceiver 172 may transmit and / or receive data using any known standard (e.g., Wi-Fi, Bluetooth, Bluetooth Smart, 802.15.4, ZigBee ( Registered trademark) etc.). Such transmission may include communication from the host vehicle to one or more remotely located servers. Such transmissions may include one or more of the host vehicle and one or more of the host vehicle's environments (e.g., to facilitate coordination of the host vehicle's navigation with or in relation to the target vehicle in the host vehicle's environment). It may include communication (unidirectional or bidirectional) with the target vehicle, as well as broadcast transmission to unspecified recipients near the transmitting vehicle.

  [085] Both the application processor 180 and the image processor 190 may include various types of hardware-based processing devices. For example, one or both of the application processor 180 and the image processor 190 include a microprocessor, a preprocessor (such as an image preprocessor), a graphics processor, a central processing unit (CPU), a support circuit, a digital signal processor, an integrated circuit, and a memory. Or any other type of device suitable for running an application and processing and analyzing the images. In some embodiments, application processor 180 and / or image processor 190 may include any type of single-core or multi-core processor, mobile device microcontroller, central processing unit, etc. Various processing devices are available, including, for example, processors available from manufacturers such as Intel®, AMD®, etc., and various architectures (eg, x86 processors, ARM®, etc.). ).

  [086] In some embodiments, the application processor 180 and / or the image processor 190 may include any of the EyeQ series processors available from Mobileye®. These processor designs include a plurality of processing units each having a local memory and an instruction set. Such a processor may include a video input for receiving image data from multiple image sensors, and may also include a video output function. In one example, EyeQ2® uses 90 nm-micron technology operating at 332 MHz. The EyeQ2® architecture includes two floating point hyperthreaded 32-bit RISC CPUs (MIPS32® 34K® core), five vision computation engines (VCE), and three vector microcode processors (VMP ( ®), a Denali 64-bit mobile DDR controller, a 128-bit internal audio interconnect, dual 16-bit video input and 18-bit video output controllers, a 16-channel DMA, and some peripherals. The MIPS34K CPU manages five VCEs, three VMPs and DMAs, a second MIPS34K CPU and multi-channel DMA, and other peripherals. Five VCEs, three VMPs, and a MIPS34K CPU can perform the intensive vision calculations required by multi-functional bundled applications. As another example, in a disclosed embodiment, a third generation processor, EyeQ3®, which is six times stronger than EyeQ2®, may be used. In other examples, EyeQ4® and / or EyeQ5® can be used in the disclosed embodiments. Of course, newer or future EyeQ processing devices may be used with the disclosed embodiments.

  [087] Any of the processing devices disclosed herein can be configured to perform a particular function. Configuring a processing device, such as the described EyeQ processor or any other controller or microprocessor, to perform a particular function is to program computer-executable instructions to execute during operation of the processing device. Providing the instructions to the processing device. In some embodiments, configuring the processing device may include directly programming the processing device with architectural instructions. In other embodiments, configuring the processing device may include storing the executable instructions on a memory accessible by the processing device during operation. For example, a processing device may access memory during operation to retrieve and execute stored instructions. In any event, the processing device configured to perform the sensing, image analysis, and / or navigation functions disclosed herein requires dedicated hardware to control multiple hardware-based components of the host vehicle. Represents the base system.

  [088] Although FIG. 1 shows two separate processing devices included in the processing unit 110, more or fewer processing devices may be used. For example, in some embodiments, a single processing device may be used to accomplish the tasks of application processor 180 and image processor 190. In other embodiments, these tasks may be performed by more than two processing devices. Further, in some embodiments, the system 100 may include one or more of the processing units 110 without including other components such as the image acquisition unit 120.

  [089] The processing unit 110 may include various types of devices. For example, processing unit 110 may include a variety of controllers, image preprocessors, central processing units (CPUs), support circuits, digital signal processors, integrated circuits, memories, or any other type of device that processes and analyzes images. It may include a device. The image preprocessor may include a video processor that captures, digitizes, and processes images from the image sensor. A CPU may include any number of microcontrollers or microprocessors. The support circuits can be any number of circuits commonly known in the art, including cache, power, clock, and input / output circuits. The memory may store software that, when executed by the processor, controls the operation of the system. The memory may include a database and image processing software. The memory may include any number of random access memory, read only memory, flash memory, disk drives, optical storage, tape storage, removable storage, and other types of storage. In one example, the memory can be separate from the processing unit 110. In another example, the memory may be integrated into the processing unit 110.

  [090] Each memory 140, 150 may include software instructions that, when executed by a processor (eg, application processor 180 and / or image processor 190), may control the operation of various aspects of system 100. These memory units may include various databases and image processing software, as well as trained systems such as, for example, neural networks or deep neural networks. The memory unit may include random access memory, read-only memory, flash memory, disk drive, optical storage, tape storage, removable storage, and / or any other type of storage. In some embodiments, memory units 140, 150 may be separate from application processor 180 and / or image processor 190. In other embodiments, these memory units may be integrated into application processor 180 and / or image processor 190.

  [091] The position sensor 130 may include any type of device suitable for determining a position associated with at least one component of the system 100. In some embodiments, position sensor 130 may include a GPS receiver. Such a receiver can determine the position and speed of the user by processing the signal broadcast by the Global Positioning System satellite. Position information from position sensor 130 may be provided to application processor 180 and / or image processor 190.

  [092] In some embodiments, the system 100 may include components such as a speed sensor (eg, a speedometer) for measuring the speed of the vehicle 200. System 100 may also include one or more (single-axis or multi-axis) accelerometers for measuring acceleration of vehicle 200 along one or more axes.

  [093] The memory units 140, 150 may include data organized in a database or any other format that indicates locations of known landmarks. Processing environmental sensor information (images, radar signals, depth information obtained by lidaring or stereoscopically processing two or more images, etc.) together with position information such as GPS coordinates and vehicle self-motion, to obtain a known landmark The current position of the vehicle with respect to is obtained, and the position of the vehicle can be refined. A particular aspect of this technology is included in the location technology known as REM ™ sold by the assignee of the present application.

  [094] User interface 170 may include any device suitable for providing information or receiving input from one or more users of system 100. In some embodiments, the user interface 170 may include user input devices, including, for example, a touch screen, microphone, keyboard, pointer device, track wheel, camera, knob, buttons, and the like. With such an input device, a user can type commands or information, provide voice commands, use buttons, pointers, or eye tracking features, or communicate any other information to the system 100. By selecting on-screen menu choices through suitable techniques, it may be possible to provide information input or commands to the system 100.

  [095] The user interface 170 may provide one or more processes to provide information to or receive information from the user, for example, process the information for use by the application processor 180. A device may be provided. In some embodiments, such a processing device recognizes and tracks eye movements, receives and interprets voice commands, recognizes touches and / or gestures made on a touch screen. Commands to interpret, commands to respond to keyboard input or menu selections, etc. may be executed. In some embodiments, user interface 170 may include a display, speakers, haptic devices, and / or any other device that provides output information to a user.

  [096] The map database 160 may include any type of database that stores map data useful to the system 100. In some embodiments, the map database 160 stores data relating to locations in the reference coordinate system of various items, including roads, water features, geographic features, businesses, points of interest, restaurants, gas stations, and the like. May be included. Map database 160 may store not only the locations of such items, but also descriptors associated with those items, including, for example, the names associated with any of the stored features. In some embodiments, the map database 160 may be physically located with other components of the system 100. Alternatively or additionally, map database 160 or a portion thereof may be located remotely with respect to other components of system 100 (eg, processing unit 110). In such embodiments, information from the map database 160 may be downloaded to a network via a wired or wireless data connection (eg, via a cellular network and / or the Internet, etc.). In some cases, map database 160 may store a sparse data model that includes a polynomial representation of a particular road feature (eg, a lane mark) or a target trajectory of the host vehicle. Map database 160 may also include a stored representation of various recognized landmarks that may be used to determine or update the known position of the host vehicle with respect to the target trajectory. The landmark representation may include, among other potential identifiers, data fields such as landmark type and landmark location.

  [097] The image capture devices 122, 124, and 126 may each include any type of device suitable for capturing at least one image from an environment. Further, any number of image capture devices may be used to acquire images for input to an image processor. Some embodiments may include only a single image capture device, while other embodiments may include two, three, or even four or more image capture devices. Image capture devices 122, 124, and 126 are further described below with reference to FIGS. 2B-2E.

  [098] One or more cameras (eg, image capture devices 122, 124, and 126) may be part of a sensing block included on the vehicle. Various other sensors may be included in the sensing block, and any or all of the sensors may be utilized to develop a sensed navigation condition for the vehicle. In addition to the camera (front-facing, side-facing, back-facing, etc.), other sensors such as radar, lidar, acoustic sensors, etc. may be included in the sensing block. In addition, the sensing block may include one or more components configured to communicate and transmit and receive information related to the environment of the vehicle. For example, such components include a radio transceiver (eg, RF) that can receive sensor-based information or any other type of information related to the environment of the host vehicle from a source remotely located with respect to the host vehicle. obtain. Such information may include sensor output information or related information received from a vehicle system other than the host vehicle. In some embodiments, such information may include information received from remote computing devices, centralized servers, and the like. Further, the cameras can take many different configurations, such as a single camera unit, multiple cameras, camera clusters, long FOV, short FOV, wide angle, fisheye, and the like.

  [099] The system 100 or various components of the system 100 may be incorporated into a variety of different platforms. In some embodiments, system 100 may be included in vehicle 200, as shown in FIG. 2A. For example, vehicle 200 may include processing unit 110 and any other components of system 100, as described above with respect to FIG. In some embodiments, vehicle 200 may include only a single image capture device (e.g., a camera), while in other embodiments, such as the embodiments discussed in connection with FIGS. Image capture devices can be used. For example, as shown in FIG. 2A, either of the image capture devices 122 and 124 of the vehicle 200 may be part of an ADAS (Advanced Driver Assistance System) imaging set.

  [0100] The image capture device included in the vehicle 200 as part of the image acquisition unit 120 may be located at any suitable location. In some embodiments, as shown in FIGS. 2A-2E and 3A-3C, image capture device 122 may be located near a rearview mirror. This location may provide a line of sight similar to the driver of vehicle 200 and may assist the driver in determining what is visible and what is not. Although the image capture device 122 may be located at any location near the rearview mirror, locating the image capture device 122 on the driver side of the mirror further assists in obtaining images representing the driver's field of view and / or gaze. I can do it.

  [0101] Other locations may be used for the image capture device of the image acquisition unit 120. For example, the image capture device 124 may be located on or in a bumper of the vehicle 200. Such a location may be particularly suitable for image capture devices having a wide field of view. The line of sight of the image capture device located on the bumper can be different from the line of sight of the driver, so the bumper image capture device and the driver do not always see the same object. Image capture devices (eg, image capture devices 122, 124, and 126) can be located at other locations. For example, the image capture device may be located on one or both of the side mirrors of the vehicle 200, the roof of the vehicle 200, the hood of the vehicle 200, the trunk of the vehicle 200, the side of the vehicle 200, and mounted on any window of the vehicle 200. , Behind, or forward, and may be mounted at or near the front and / or rear lights of the vehicle 200.

  [0102] In addition to the image capture device, the vehicle 200 may include various other components of the system 100. For example, the processing unit 110 may be integrated into an engine control unit (ECU) of the vehicle or included in the vehicle 200 separately from the ECU. The vehicle 200 may include a position sensor 130 such as a GPS receiver, and the vehicle 200 may include a map database 160 and memory units 140 and 150.

  [0103] As described above, wireless transceiver 172 may receive and / or receive data via one or more networks (eg, a cellular network, the Internet, etc.). For example, wireless transceiver 172 may upload data collected by system 100 to one or more servers and download data from one or more servers. Via wireless transceiver 172, system 100 may receive updates to data stored in map database 160, memory 140, and / or memory 150, for example, periodically or on demand. Similarly, wireless transceiver 172 may include any data from system 100 (eg, images captured by image acquisition unit 120, position sensors 130, other sensors, or data received by vehicle control systems, etc.) and / or Or, any data processed by the processing unit 110 may be uploaded to one or more servers.

  [0104] The system 100 may upload data to a server (eg, cloud) based on privacy level settings. For example, the system 100 may implement a privacy level setting that regulates or restricts data (including metadata) transmitted to the server that may uniquely identify the vehicle and / or the driver / owner of the vehicle. Such settings may be set by a user, for example, via wireless transceiver 172, may be initialized by factory default settings, or may be set by data received by wireless transceiver 172. .

  [0105] In some embodiments, the system 100 may upload data according to a "high" privacy level, and under settings, the system 100 does not have any details about a particular vehicle and / or driver / owner. Data (eg, location information related to the route, captured images, etc.) may be transmitted. For example, when uploading data according to a “high” privacy level, the system 100 does not include the vehicle identification number (VIN) or the name of the driver or owner of the vehicle, but instead, as long as it is associated with the captured image and / or route. Data such as the obtained location information may be transmitted.

  [0106] Other privacy levels are also contemplated. For example, the system 100 may send data to a server according to a “medium” privacy level, and “high” privacy, such as the make and / or model of the vehicle and / or vehicle type (eg, passenger car, sports utility vehicle, truck, etc.). It may include additional information not included below the level. In some embodiments, system 100 may upload data according to a "low" privacy level. Under the “low” privacy level setting, the system 100 uploads enough data to uniquely identify a particular vehicle, owner / driver, and / or part or all of the route the vehicle has traveled, Such information may be included. Such “low” privacy level data may include, for example, VIN, driver / owner name, point of departure of the vehicle prior to departure, intended destination of the vehicle, make and / or model of the vehicle, type of vehicle, etc. It may include one or more.

  [0107] FIG. 2A is a side view representation of an exemplary vehicle imaging system according to the disclosed embodiments. FIG. 2B is a top view representation of the embodiment shown in FIG. 2A. As shown in FIG. 2B, the disclosed embodiment includes a first image capture device 122 positioned near a rearview mirror and / or near a driver of the vehicle 200 and a bumper region of the vehicle 200 (eg, a bumper region). One of 210) may show a vehicle 200 that includes in its body a system 100 having a second image capture device 124 positioned on or in the bumper area and a processing unit 110.

  [0108] As shown in FIG. 2C, both image capture devices 122 and 124 may be positioned near the rearview mirror of vehicle 200 and / or near the driver. Further, while two image capture devices 122 and 124 are shown in FIGS. 2B and 2C, it should be understood that other embodiments may include more than two image capture devices. For example, in the embodiment shown in FIGS. 2D and 2E, a first image capture device 122, a second image capture device 124, and a third image capture device 126 are included in system 100 of vehicle 200.

  As shown in FIG. 2D, the image capture device 122 may be positioned near the rearview mirror of the vehicle 200 and / or near the driver, and the image capture devices 124 and 126 may be positioned near the bumper area of the vehicle 200 (eg, , One of the bumper regions 210). Also, as shown in FIG. 2E, the image supplement devices 122, 124, and 126 may be positioned near the rearview mirror of the vehicle 200 and / or near the driver's seat. The disclosed embodiments are not limited to any particular number and configuration of image capture devices, which may be positioned within and / or on the vehicle 200 at any suitable location.

  [0110] It should be understood that the disclosed embodiments are not limited to vehicles and are applicable in other situations. It should also be understood that the disclosed embodiments are not limited to a particular type of vehicle 200, but may be applicable to all types of vehicles, including automobiles, trucks, trailers, and other types of vehicles.

  [0111] The first image capture device 122 may include any suitable type of image capture device. Image capture device 122 may include an optical axis. In one example, image capture device 122 may include an Aptina M9V024 WVGA sensor with a global shutter. In other embodiments, image capture device 122 may provide a resolution of 1280 × 960 pixels and may include a rolling shutter. Image capture device 122 may include various optical components. In some embodiments, one or more lenses may be included, for example, to provide a desired focal length and field of view of the image capture device. In some embodiments, a 6 mm lens or a 12 mm lens may be associated with the image capture device 122. In some embodiments, image capture device 122 may be configured to capture an image having a desired field of view (FOV) 202, as shown in FIG. 2D. For example, the image capture device 122 may be configured to have a normal FOV, such as in the range of 40 degrees to 56 degrees, including 46 degrees FOV, 50 degrees FOV, 52 degrees FOV, or more than 52 degrees FOV. . Alternatively, the image capture device 122 may be configured to have a narrow FOV in the range of 23-40 degrees, such as a 28 degree FOV or a 36 degree FOV. In addition, image capture device 122 may be configured to have a wide FOV in the range of 100-180 degrees. In some embodiments, the image capture device 122 may include a wide-angle bumper camera or a bumper camera with a maximum 180 degree FOV. In some embodiments, image capture device 122 may be a 7.2 M pixel image capture device with an aspect ratio of about 2: 1 (eg, H × V = 3800 × 1900 pixels) having a horizontal FOV of about 100 degrees. . Such an image capture device may be used instead of a three-dimensional image capture device configuration. Due to the large lens distortion, the vertical FOV of such an image capture device can be much lower than 50 degrees in implementations where the image capture device uses radially symmetric lenses. For example, such lenses are not radially symmetric, which allows a vertical FOV of greater than 50 degrees with a horizontal FOV of 100 degrees.

  [0112] The first image capture device 122 may acquire a plurality of first images for a scene associated with the vehicle 200. The plurality of first images may each be acquired as a series of image scan lines, which may be captured using a rolling shutter. Each scan line may include multiple pixels.

  [0113] The first image capture device 122 may have a scan rate associated with acquiring each of the first series of image scan lines. The scan rate may refer to the rate at which the image sensor can acquire image data associated with each pixel included in a particular scan line.

  [0114] The image capture devices 122, 124, and 126 may include any suitable type and number of image sensors, including, for example, a CCD sensor or a CMOS sensor. In one embodiment, a CMOS image sensor may be utilized with a rolling shutter, whereby each pixel in a row is read one at a time, and scanning of the row is advanced row by row until the entire image frame is captured. . In some embodiments, rows may be captured sequentially from top to bottom with respect to the frame.

  [0115] In some embodiments, one or more of the image capture devices (e.g., image capture devices 122, 124, and 126) disclosed herein may constitute a high-resolution imager and may have 5M pixels. It can have a resolution of more than 7M pixels, more than 10M pixels, or more.

  [0116] Through the use of a rolling shutter, pixels in different rows may be exposed and captured at different times, thereby causing skew and other image artifacts in the captured image frames. On the other hand, if the image capture device 122 is configured to operate with a global or synchronous shutter, all pixels may be exposed during a common exposure period for the same amount of time. As a result, image data in a frame collected from a system utilizing a global shutter represents a snapshot of the entire FOV (such as FOV 202) at a particular time. Conversely, when applying a rolling shutter, each row in the frame is exposed and data is captured at different times. Thus, a moving object may appear distorted on an image capture device having a rolling shutter. This phenomenon will be described in more detail below.

  [0117] The second image capture device 124 and the third image capture device 126 can be any type of image capture device. Like the first image capture device 122, each of the image capture devices 124 and 126 may include an optical axis. In one embodiment, each of the image capture devices 124 and 126 may include an Aptina M9V024 WVGA sensor with a global shutter. Alternatively, each of the image capture devices 124 and 126 may include a rolling shutter. Like image capture device 122, image capture devices 124 and 126 may be configured to include various lenses and optical elements. In some embodiments, the lenses associated with image capture devices 124 and 126 provide the same or narrower FOV (such as FOVs 204 and 206) as the FOV associated with image capture device 122 (such as FOV 202). I can do it. For example, image capture devices 124 and 126 may have a FOV of 40 degrees, 30 degrees, 26 degrees, 23 degrees, 20 degrees, or less than 20 degrees.

  [0118] Image capture devices 124 and 126 may acquire a plurality of second and third images for a scene associated with vehicle 200. Each of the plurality of second and third images may be acquired as a second and third series of image scan lines, which may be captured using a rolling shutter. Each scan line or each row may have multiple pixels. Image capture devices 124 and 126 may have second and third scan rates associated with acquiring each image scan line included in the second and third series.

  [0119] Each image capture device 122, 124, and 126 may be positioned at any suitable location with respect to the vehicle 200 and in any suitable orientation. The relative positions of the image capture devices 122, 124, and 126 may be selected to assist in fusing information obtained from the image capture devices together. For example, in some embodiments, the FOV associated with image capture device 124 (FOV 204) is associated with image capture device 122 (eg, FOV 202) and FOV associated with image capture device 126 (eg, FOV 206). And may partially or completely overlap.

[0120] Image capture devices 122, 124, and 126 may be positioned on vehicle 200 at any suitable relative height. In one example, there can be a height difference between the image capture devices 122, 124, and 126, and the height difference can provide sufficient disparity information to enable stereo analysis. For example, as shown in FIG. 2A, the two image capture devices 122 and 124 are at different heights. There may also be lateral displacement differences between the image capture devices 122, 124, and 126, for example, to provide additional parallax information to the stereo analysis by the processing unit 110. Lateral displacement difference, as shown in FIGS. 2C and 2D, may indicate at d _x. In some embodiments, a front displacement or a rear displacement (eg, a range displacement) may be present between the image capture devices 122, 124, 126. For example, image capture device 122 may be located more than 0.5 to 2 meters behind image capture device 124 and / or image capture device 126. With this type of displacement, one of the image capture devices may be able to cover a potential blind spot of another image capture device.

  [0121] The image capture device 122 may have any suitable resolution capability (eg, the number of pixels associated with the image sensor), and the resolution of the image sensor associated with the image capture device 122 may include the image capture device 124 and The resolution of the image sensor associated with 126 may be higher, lower, or the same. In some embodiments, the image capture device 122 and / or the image sensors associated with the image capture devices 124 and 126 have a resolution of 640 × 480, 1024 × 768, 1280 × 960, or any other suitable resolution. I can do it.

  [0122] The frame rate (eg, the rate at which an image capture device acquires a set of pixel data for one image frame before moving on to capture the pixel data associated with the next image frame) is controllable. obtain. The frame rate associated with image capture device 122 may be higher, lower, or the same as the frame rate associated with image capture devices 124 and 126. The frame rate associated with image capture devices 122, 124, and 126 may depend on various factors that can affect the timing of the frame rate. For example, one or more of the image capture devices 122, 124, and 126 may be prior to or before the acquisition of image data associated with one or more pixels of an image sensor in the image capture devices 122, 124, and / or 126. It may include a selectable pixel delay period imposed after acquisition. In general, image data corresponding to each pixel may be obtained according to the clock rate of the device (eg, one pixel per clock cycle). Further, in embodiments that include a rolling shutter, one or more of the image capture devices 122, 124, and 126 may include image data associated with pixel rows of image sensors within the image capture devices 122, 124, and / or 126. And a selectable horizontal blanking period imposed before or after the acquisition of Further, one or more of the image capture devices 122, 124, and / or 126 may be selectable imposed before or after the acquisition of image data associated with the image frames of the image capture devices 122, 124, and 126. A vertical blanking period may be included.

  [0123] With these timing controls, the frame rates associated with the image capture devices 122, 124, and 126 may be synchronized even when the line scan rates of each image capture device are different. Further, as discussed in more detail below, among these factors (eg, image sensor resolution, maximum line scan rate, etc.), these selectable timing controls allow the field of view of image capture device 122 to be increased by image capture device 124. And 126 may be able to synchronize image capture from an area where the FOV of image capture device 122 overlaps with one or more FOVs of image capture devices 124 and 126.

  [0124] Frame rate timing at the image capture devices 122, 124, and 126 may depend on the resolution of the associated image sensor. For example, assuming that the line scan rates of both devices are similar, and one device includes an image sensor having a resolution of 640 × 480 and the other device includes an image sensor having a resolution of 1280 × 960, a higher resolution may be used. A longer time is required to acquire a frame of image data from a sensor having the same.

  [0125] Another factor that can affect the timing of image data acquisition at the image capture devices 122, 124, and 126 is the maximum line scan rate. For example, obtaining rows of image data from image sensors included in image capture devices 122, 124, and 126 requires some minimum amount of time. Assuming no additional pixel delay period, this minimum amount of time to acquire a row of image data will be related to the maximum line scan rate for a particular device. Devices that provide higher maximum line scan rates have the potential to provide higher frame rates than devices that have lower maximum line scan rates. In some embodiments, one or both of image capture devices 124 and 126 may have a higher line scan rate than the maximum line scan rate associated with image capture device 122. In some embodiments, the highest line scan rate of image capture device 124 and / or 126 is 1.25, 1.5, 1.75, or 2 times the highest line scan rate of image capture device 122. Or more.

  [0126] In another embodiment, image capture devices 122, 124, and 126 may have the same highest line scan rate, but image capture device 122 may operate at a scan rate that is less than or equal to its highest scan rate. The system may be configured such that one or both of the image capture devices 124 and 126 operate at a line scan rate equal to the line scan rate of the image capture device 122. In other examples, the system may be configured such that the line scan rate of the image capture device 124 and / or 126 is 1.25 times, 1.5 times, 1.75 times, or more than 2 times the line scan rate of the image capture device 122. May be configured.

  [0127] In some embodiments, image capture devices 122, 124, and 126 may be asymmetric. That is, these image capture devices may include cameras having different fields of view (FOV) and focal lengths. The field of view of the image capture devices 122, 124, and 126 may include any desired area for the environment of the vehicle 200, for example. In some embodiments, one or more of the image capture devices 122, 124, and 126 may capture images from an environment in front of the vehicle 200, an environment behind the vehicle 200, an environment on both sides of the vehicle 200, or a combination thereof. It can be configured to obtain data.

  [0128] Further, the focal length associated with each image capture device 122, 124, and / or 126 may be selectable such that each device acquires an image of an object within a desired distance range from vehicle 200. (Eg, by inclusion of a suitable lens, etc.). For example, in some embodiments, image capture devices 122, 124, and 126 may capture images of nearby objects within a few meters of the vehicle. The image capture devices 122, 124, 126 can also be configured to capture images of objects at a greater distance from the vehicle (eg, 25m, 50m, 100m, 150m or more). Further, the focal length of the image capture devices 122, 124, and 126 may be such that some image capture devices (eg, image capture device 122) capture images of objects relatively close to the vehicle (eg, within 10 meters or 20 meters). While other image capture devices (e.g., image capture devices 124 and 126) capture images of objects further away from vehicle 200 (e.g., greater than 20 m, greater than 50 m, greater than 100 m, greater than 150 m, etc.). You can choose to be.

  [0129] According to some embodiments, the FOV of one or more image capture devices 122, 124, and 126 may have a wide angle. For example, it may be advantageous to have a 140 degree FOV for image capture devices 122, 124, and 126, which may be used, in particular, for image capture of areas near vehicle 200. For example, image capture device 122 may be used to capture an image of the right or left area of vehicle 200, and in such embodiments, image capture device 122 may have a wide FOV (eg, at least 140 degrees). It may be desirable.

  [0130] The field of view associated with each of the image capture devices 122, 124, and 126 may depend on each focal length. For example, as the focal length increases, the corresponding field of view decreases.

  [0131] Image capture devices 122, 124, and 126 may be configured to have any suitable field of view. In one particular example, image capture device 122 may have a horizontal FOV of 46 degrees, image capture device 124 may have a horizontal FOV of 23 degrees, and image capture device 126 may have a horizontal FOV of 23-46 degrees. In another example, image capture device 122 may have a horizontal FOV of 52 degrees, image capture device 124 may have a horizontal FOV of 26 degrees, and image capture device 126 may have a horizontal FOV of 26-52 degrees. In some embodiments, the ratio of the FOV of image capture device 122 to the FOV of image capture device 124 and / or image capture device 126 may vary between 1.5 and 2.0. In other embodiments, this ratio can vary from 1.25 to 2.25.

  [0132] System 100 may be configured such that the field of view of image capture device 122 at least partially or completely overlaps the field of view of image capture device 124 and / or image capture device 126. In some embodiments, the system 100 may be configured such that the field of view of the image capture devices 124 and 126 falls within, for example, the field of view of the image capture device 122 (eg, is smaller than the field of view of the image capture device 122). Can be configured to share a common center with the field of view. In other embodiments, image capture devices 122, 124, and 126 may capture adjacent FOVs or have partially overlapping FOVs. In some embodiments, the field of view of the image capture devices 122, 124, and 126 is such that the center of the narrower FOV image capture devices 124 and / or 126 is located in the lower half of the field of view of the wider FOV device 122. Can be aligned to obtain.

  [0133] FIG. 2F is a diagrammatic representation of an exemplary vehicle control system according to the disclosed embodiments. As shown in FIG. 2F, vehicle 200 may include a throttle system 220, a brake system 230, and a steering system 240. The system 100 may include one or more of a throttle system 220, a brake system 230, and a steering system 240 via one or more data links (eg, any wired and / or wireless links or data transmission links). (Eg, a control signal). For example, based on an analysis of images acquired by the image capture devices 122, 124, and / or 126, the system 100 may provide control signals for navigating the vehicle 200 to the throttle system 220, the brake system 230, and the steering system 240. One or more may be provided (eg, by accelerating, turning, lane shifting, etc.). Further, the system 100 may provide inputs (e.g., speed, whether the vehicle 200 is braking and / or turning, etc.) indicative of the operating status of the vehicle 200 to the throttle system 220, the braking system 230, and the steering system 24. It may be received from one or more. In the following, further details will be provided in connection with FIGS.

  As shown in FIG. 3A, vehicle 200 may also include a user interface 170 that interacts with a driver or occupant of vehicle 200. For example, the user interface 170 in the vehicle application may include a touch screen 320, a knob 330, a button 340, and a microphone 350. The driver or occupant of the vehicle 200 uses a handle (eg, located on or near the steering column of the vehicle 200, including, for example, a turn signal handle) and a button (eg, located on the handle of the vehicle 200). To interact with the system 100. In some embodiments, microphone 350 may be positioned adjacent to rearview mirror 310. Similarly, in some embodiments, the image capture device 122 may be located near the rearview mirror 310. In some embodiments, user interface 170 may also include one or more speakers 360 (eg, speakers of a vehicle audio system). For example, system 100 may provide various notifications (eg, alerts) via speaker 360.

  [0135] FIGS. 3B-3D illustrate an example camera mount configured to be positioned against a vehicle windscreen behind a rearview mirror (eg, rearview mirror 310), according to disclosed embodiments. 370 is a diagram of FIG. As shown in FIG. 3B, camera mount 370 may include image capture devices 122, 124, and 126. Image capture devices 124 and 126 may be positioned behind glare shield 380, which may directly contact the windshield and may include a composition of film and / or anti-reflective material. For example, glare shield 380 may be positioned to be aligned against a windshield having a matching slope. In some embodiments, each of image capture devices 122, 124, and 126 may be positioned behind glare shield 380, for example, as shown in FIG. 3D. The disclosed embodiments are not limited to any particular configuration of the image capture devices 122, 124 and 126, camera mount 370, and glare shield 380. FIG. 3C is a view of the camera mount 370 shown in FIG. 3B as viewed from the front.

  [0136] As will be appreciated by those skilled in the art having the benefit of this disclosure, many variations and / or modifications to the above disclosed embodiments may be made. For example, not all components are required for operation of system 100. Further, any components may be located in any suitable portion of system 100, and the components may be rearranged into various configurations while providing the functionality of the disclosed embodiments. Thus, the above-described configuration is an example, and regardless of the configuration described above, system 100 may provide a wide range of functions for analyzing the surroundings of vehicle 200 and navigating vehicle 200 in response to the analysis.

  [0137] As discussed in further detail below, in accordance with various disclosed embodiments, system 100 may provide various features related to autonomous driving and / or driver assistance technology. For example, the system 100 may analyze image data, location data (eg, GPS location information), map data, speed data, and / or data from sensors included in the vehicle 200. System 100 may, for example, collect data for analysis from image acquisition unit 120, position sensor 130, and other sensors. Further, the system 100 may analyze the collected data to determine whether the vehicle 200 should take a particular action and then automatically take the determined action without human intervention. For example, if the vehicle 200 navigates without human recruitment, the system 100 may automatically control the braking, acceleration, and / or steering of the vehicle 200 (eg, control signals may be transmitted to the throttle system 220, the brake system 230). , And by sending to one or more of the steering systems 240). Further, the system 100 may analyze the collected data and issue warnings and / or alerts to vehicle occupants based on the analysis of the collected data. Further details regarding various embodiments provided by the system 100 are provided below.

[0138] Forward-looking multi-imaging system
[0139] As described above, the system 100 can provide a driving assistance function using a multi-camera system. A multi-camera system may use one or more cameras facing the front of the vehicle. In other embodiments, a multi-camera system may include one or more cameras facing the side of the vehicle or the rear of the vehicle. In one embodiment, for example, system 100 may use a two-camera imaging system, where the first camera and the second camera (eg, image capture devices 122 and 124) are connected to a vehicle (eg, vehicle 200). May be positioned at the front and / or sides. Other camera configurations are consistent with the disclosed embodiments, and the configurations disclosed herein are examples. For example, system 100 may include any number (eg, one, two, three, four, five, six, seven, eight, etc.) of camera configurations. Further, system 100 may include a "cluster" of cameras. For example, a cluster of cameras (including any suitable number, e.g., one, four, eight, etc. cameras) may be forward facing the vehicle, or facing any other direction. (Eg, backwards, sideways, oblique, etc.). Thus, system 100 may include multiple clusters of cameras, with each cluster oriented in a particular direction to capture images from a particular region of the vehicle's environment.

  [0140] The first camera may have a field of view that is larger, smaller, or partially overlapping than the field of view of the second camera. Further, the first camera may be connected to the first image processor to perform monocular image analysis of an image provided by the first camera, and the second camera may be connected to the second image processor. , May perform a monocular image analysis of the image provided by the second camera. Outputs (eg, processed information) of the first and second image processors may be combined. In some embodiments, the second image processor may receive images from both the first camera and the second camera and perform stereo analysis. In another embodiment, system 100 may use a three camera imaging system, where each camera has a different field of view. Accordingly, such systems may make decisions based on information derived from objects at various distances, both forward and side of the vehicle. References to monocular image analysis may refer to cases where the image analysis is performed based on images captured from a single viewpoint (eg, a single camera). Stereoscopic image analysis may refer to the case where image analysis is performed based on two or more images captured with one or more of the image capture parameters changed. For example, captured images suitable for performing stereoscopic image analysis include images captured from two or more different locations, images captured from different fields of view, images captured using different focal lengths, with disparity information. It may include captured images and the like.

  [0141] For example, in one embodiment, system 100 may implement a three-camera configuration that uses image capture devices 122-126. In such a configuration, image capture device 122 may provide a narrow field of view (e.g., 34 degrees or other value selected from a range of about 20-45 degrees, etc.) and image capture device 124 may provide a wide field of view ( For example, the image capture device 126 may provide a 150 degree or other value selected from a range of about 100 to about 180 degrees, and the image capture device 126 may provide a medium field of view (e.g., 46 degrees or selected from a range of about 35 to about 60 degrees). Other values). In some embodiments, the image capture device 126 may operate as a primary or primary camera. Image capture devices 122-126 may be positioned substantially side-by-side (eg, 6 cm apart) behind rear-view mirror 310. Further, in some embodiments, as described above, one or more of the image capture devices 122-126 may be mounted behind a glare shield 380 that is flush with the windshield of the vehicle 200. Such a shield may operate to minimize the effect of any reflections from the interior of the vehicle on the image capture devices 122-126.

  [0142] In another embodiment, as described above in connection with FIGS. 3B and 3C, a wide-field camera (eg, image capture device 124 in the above example) is a narrow main-field camera (eg, image in the above example). It may be mounted lower than the capture devices 122 and 126). This configuration can provide free gaze from a wide-field camera. To reduce reflections, the camera may be mounted near the windshield of vehicle 200 and may include a polarizer in the camera to reduce reflected light.

  [0143] A three-camera system may provide certain performance features. For example, some embodiments may include the ability to verify detection of an object by one camera based on detection results from another camera. In the three camera configuration described above, the processing unit 110 may include, for example, three processing devices (eg, three EyeQ series processor chips as described above), each processing device being one of the image capture devices 122-126. One or more of the images captured.

  [0144] In a three camera system, the first processing device may receive images from both the main camera and the narrow field of view camera and perform vision processing of the narrow FOV camera, for example, for other vehicles, pedestrians, Lane marks, traffic signs, traffic lights, and other road objects can be detected. Further, the first processing device may calculate the parallax of the pixels between the image from the main camera and the image from the narrow camera, and create a 3D reconstruction of the environment of the vehicle 200. Next, the first processing device may combine the 3D reconstruction with the 3D map data or 3D information calculated based on information from another camera.

  [0145] The second processing device may receive images from the main camera and perform vision processing to detect other vehicles, pedestrians, lane marks, traffic signs, traffic lights, and other road objects. . Further, the second processing device may calculate camera displacement, calculate pixel disparity between successive images based on the displacement, and create a 3D reconstruction of the scene (eg, structure from motion). The second processing device may send the structure from motion based on the 3D reconstruction to the first processing device and combine the structure from motion with the stereoscopic 3D image.

  [0146] A third processing device may receive the image from the wide FOV camera and process the image to detect vehicles, pedestrians, lane marks, traffic signs, traffic lights, and other road objects. The third processing device may further execute additional processing instructions to analyze the image and identify moving objects in the image, such as vehicles changing lanes, pedestrians, and the like.

  [0147] In some embodiments, independently capturing and processing image-based information streams may provide an opportunity to provide redundancy in the system. Such redundancy verifies information obtained by capturing and processing image information from at least a second image capture device, for example, using a first image capture device and images processed from the device. And / or supplement.

  [0148] In some embodiments, the system 100 may use two image capture devices (eg, image capture devices 122 and 124) in providing navigation assistance to the vehicle 200 and a third image capture device (e.g., For example, an image capture device 126) may be used to provide redundancy and verify analysis of data received from the other two image capture devices. For example, in such a configuration, the image capture devices 122 and 124 may provide images for stereoscopic analysis by the system 100 for navigating the vehicle 200, while the image capture device 126 may provide a monocular by the system 100. Images may be provided for analysis to provide redundancy and verification of information obtained based on images captured from image capture device 123 and / or image capture device 124. That is, the image capture device 126 (and corresponding processing device) may provide a redundant sub that provides a check on the analysis derived from the image capture devices 122 and 124 (eg, to provide an automatic emergency braking (AEB) system). It can be considered to provide a system. Further, in some embodiments, the received data is based on information received from one or more sensors (eg, radar, lidar, acoustic sensors, information received from one or more transceivers outside the vehicle, etc.). Redundancy and verification.

  [0149] Those skilled in the art will recognize that the above camera configurations, camera locations, number of cameras, camera positions, etc. are merely exemplary. These components and the like described for the overall system may be assembled and used in a variety of different configurations without departing from the scope of the disclosed embodiments. Further details regarding the use of the multi-camera system to provide driver assistance and / or autonomous vehicle functions follow.

  [0150] FIG. 4 is an exemplary functional block diagram of a memory 140 and / or 150 that can store / program instructions for performing one or more operations in accordance with the disclosed embodiments. While reference is made below to memory 140, those skilled in the art will recognize that instructions can be stored in memory 140 and / or 150.

  [0151] As shown in FIG. 4, the memory 140 may store a monocular image analysis module 402, a stereoscopic image analysis module 404, a speed and acceleration module 406, and a navigation response module 408. The disclosed embodiments are not limited to any particular configuration of the memory 140. Further, application processor 180 and / or image processor 190 may execute instructions stored in any of modules 402-408 included in memory 140. Those skilled in the art will understand that references to processing unit 110 in the following discussion may refer to application processor 180 and image processor 190 individually or collectively. Accordingly, any step of the following process may be performed by one or more processing devices.

  [0152] In one embodiment, the monocular image analysis module 402 may store instructions (such as computer vision software) that, when executed by the processing unit 110, cause one of the image capture devices 122, 124, and 126 to execute. Performs a monocular image analysis of the set of images acquired by In some embodiments, processing unit 110 may combine information from the set of images with additional sensor information (eg, information from radar) to perform monocular image analysis. As described below in connection with FIGS. 5A-5D, monocular image analysis module 402 associates lane marks, vehicles, pedestrians, road signs, highway exit ramps, traffic lights, dangerous goods, and vehicle environments. May include instructions for detecting a set of features in the set of images, such as any other features provided. Based on the analysis, the system 100 may generate one or more navigation responses in the vehicle 200, such as turns, lane shifts, and acceleration changes, as discussed below in connection with the navigation response module 408 (e.g., , Via the processing unit 110).

  [0153] In one embodiment, the monocular image analysis module 402, when performed by the processing unit 110, performs monocular image analysis of a set of images acquired by one of the image capture devices 122, 124, and 126. Instructions (such as computer vision software) may be stored. In some embodiments, processing unit 110 may combine information from the set of images with additional sensor information (eg, information from radar, lidar, etc.) to perform monocular image analysis. As described below in connection with FIGS. 5A-5D, the monocular image analysis module 402 relates to lane marks, vehicles, pedestrians, road signs, highway exit ramps, traffic lights, hazardous materials, and vehicle environments. May include instructions for detecting a set of features in the set of images, such as any other feature to be performed. Based on the analysis, system 100 may include one or more navigation responses, such as turns, lane shifts, acceleration changes, etc., as discussed below in connection with determining the navigation responses (eg, by processing unit 110). May be caused in the vehicle 200.

  [0154] In one embodiment, the stereoscopic image analysis module 404 may store instructions (such as computer vision software) that, when executed by the processing unit 110, are selected from the image capture devices 122, 124, and 126. Performing a stereoscopic image analysis of the first and second sets of images acquired by the combined image capture device. In some embodiments, processing unit 110 may combine information from the first and second sets of images with additional sensor information (eg, information from radar) to perform stereoscopic image analysis. For example, the stereoscopic image analysis module 404 may issue an instruction to perform stereoscopic image analysis based on a first set of images acquired by the image capture device 124 and a second set of images acquired by the image capture device 126. May be included. As described below in connection with FIG. 6, the stereoscopic image analysis module 404 includes a first and a second set of lane marks, vehicles, pedestrians, road signs, highway exit ramps, traffic lights, dangerous goods, and the like. May include instructions for detecting a set of features in the image. Based on the analysis, processing unit 110 may generate one or more navigation responses in vehicle 200, such as turns, lane shifts, and acceleration changes, as described below in connection with navigation response module 408. Further, in some embodiments, the stereoscopic image analysis module 404 can implement techniques associated with a trained system (such as a neural network or deep neural network) or an untrained system.

  [0155] In one embodiment, the speed and acceleration module 406 includes data received from one or more computational and electromechanical devices in the vehicle 200 that are configured to change the speed and / or acceleration of the vehicle 200. Can be stored. For example, the processing unit 110 may execute instructions associated with the speed and acceleration module 406 to target the vehicle 200 based on data derived from execution of the monocular image analysis module 402 and / or the stereoscopic image analysis module 404. Speed can be calculated. Such data includes, for example, target position, speed, and / or acceleration, position and / or speed of vehicle 200 with respect to nearby vehicles, pedestrians, or road objects, and position information of vehicle 200 with respect to road lane marks. And the like. In addition, processing unit 110 may convert sensor inputs (eg, information from radar) and inputs from other systems of vehicle 200, such as throttle system 220, brake system 230, and / or steering system 240 of vehicle 200. Based on this, the target speed of the vehicle 200 can be calculated. Based on the calculated target speed, the processing unit 110 sends an electronic signal to the throttle system 220, the brake system 230, and / or the steering system 240 of the vehicle 200, for example, to physically weaken the brake of the vehicle 200. Alternatively, a change in speed and / or acceleration may be triggered by weakening the accelerator.

  [0156] In one embodiment, the navigation response module 408 is executable by the processing unit 110 and based on data derived from execution of the monocular image analysis module 402 and / or the stereoscopic image analysis module 404, a desired navigation. Software for determining the response may be stored. Such data may include position and speed information associated with nearby vehicles, pedestrians, and road objects, target position information for vehicle 200, and the like. Further, in some embodiments, the navigation response is detected 1 from the map data, the predetermined location of the vehicle 200, and / or from the execution of the vehicle 200 and the monocular image analysis module 402 and / or the stereo image analysis module 404. It may be based (partially or completely) on the relative speed or relative acceleration between one or more objects. The navigation response module 408 may generate a desired response based on sensor inputs (eg, information from radar) and inputs from other systems of the vehicle 200, such as the throttle system 220, the brake system 230, and the steering system 240 of the vehicle 200. Navigation response can be determined. Based on the desired navigation response, the processing unit 110 sends an electronic signal to the throttle system 220, the brake system 230, and the steering system 240 of the vehicle 200, for example, to turn the steering wheel of the vehicle 200, Achieving the rotation may trigger a desired navigation response. In some embodiments, the processing unit 110 uses the output of the navigation response module 408 (eg, the desired navigation response) as input to the execution of the speed and acceleration module 406 to calculate the speed change of the vehicle 200. I can do it.

  [0157] Further, any of the modules disclosed herein (eg, modules 402, 404, and 406) may utilize techniques associated with trained systems (such as neural networks or deep neural networks) or untrained systems. Can be implemented.

  [0158] FIG. 5A is a flowchart illustrating an example process 500A for generating one or more navigation responses based on monocular image analysis, according to a disclosed embodiment. At step 510, processing unit 110 may receive a plurality of images via data interface 128 between processing unit 110 and image acquisition unit 120. For example, a camera included in the image acquisition unit 120 (such as an image capture device 122 having a field of view 202) captures multiple images of an area in front of the vehicle 200 (or, for example, at the side or back of the vehicle), and connects to the data connection ( For example, the data may be transmitted to the processing unit 110 via digital, wired, USB, wireless, Bluetooth, or the like. The processing unit 110 may execute the monocular image analysis module 402 to analyze a plurality of images at step 520, as described in further detail below in connection with FIGS. 5B-5D. By performing the analysis, processing unit 110 may detect a set of features in the set of images such as lane marks, vehicles, pedestrians, road signs, highway exit ramps, and traffic lights.

  [0159] At step 520, the processing unit 110 executes the monocular image analysis module 402 to detect various road hazards such as truck tire parts, dropped road signs, loose cargo, and small animals. You can also. The structure, shape, size, and color of road hazards can vary, making detection of such hazards more difficult. In some embodiments, processing unit 110 may execute monocular image analysis module 402 to perform multi-frame analysis on multiple images to detect road hazards. For example, processing unit 110 may estimate camera movement between successive image frames, calculate the disparity of pixels between frames, and build a 3D map of the road. Next, the processing unit 110 may use the 3D map to detect a road surface and a dangerous substance existing on the road surface.

  [0160] At step 530, the processing unit 110 executes the navigation response module 408 to provide the vehicle 200 with one or more based on the analysis performed at step 520 and the techniques described above in connection with FIG. A navigation response may be generated. Navigation responses may include, for example, turns, lane shifts, acceleration changes, and the like. In some embodiments, processing unit 110 may use data derived from execution of velocity and acceleration module 406 to generate one or more navigation responses. Further, multiple navigation responses may occur simultaneously, sequentially, or in any combination thereof. For example, processing unit 110 may cause vehicle 200 to exceed one lane and then, for example, accelerate by, for example, sequentially transmitting control signals to steering system 240 and throttle system 220 of vehicle 200. Alternatively, processing unit 110 may cause vehicle 200 to apply a brake and simultaneously shift lanes, for example, by simultaneously transmitting a control signal to brake system 230 and steering system 240 of vehicle 200.

  [0161] FIG. 5B is a flowchart illustrating an exemplary process 500B for detecting one or more vehicles and / or pedestrians in a set of images, according to a disclosed embodiment. The processing unit 110 may execute the monocular image analysis module 402 to perform the process 500B. At step 540, processing unit 110 may identify a set of candidate objects representing potential vehicles and / or pedestrians. For example, the processing unit 110 scans one or more images, compares the images with one or more predetermined patterns, and within each image, a target object (eg, a vehicle, a pedestrian, or a portion thereof). ) May be identified. The predetermined pattern may be specified to achieve a low rate of “false hits” and a low rate of “missing”. For example, the processing unit 110 may use a low similarity threshold to a predetermined pattern to identify candidate objects as potential vehicles or pedestrians. By doing so, the processing unit 110 may be able to reduce the probability of missing (eg, not identifying) a candidate object representing a vehicle or pedestrian.

  [0162] At step 542, the processing unit 110 may filter the set of candidate objects to exclude certain candidates (eg, irrelevant or irrelevant objects) based on the classification criteria. Such criteria may be derived from various characteristics associated with the object type stored in a database (eg, a database stored in memory 140). The characteristics may include the shape, size, texture, and location (eg, for the vehicle 200) of the object, and the like. Accordingly, processing unit 110 may reject spurious candidates from the set of candidate objects using one or more sets of criteria.

  [0163] At step 544, the processing unit 110 may analyze the plurality of image frames to determine whether the objects in the set of candidate images represent vehicles and / or pedestrians. For example, processing unit 110 may track detected candidate objects over successive frames and accumulate per-frame data (eg, size, position relative to vehicle 200, etc.) associated with the detected objects. Further, the processing unit 110 may estimate the parameters of the detected object and compare the per-frame position data of the object with the predicted position.

  [0164] At step 546, processing unit 110 may construct a set of measurements for the detected object. Such measurements may include, for example, the position, velocity, and acceleration values (for vehicle 200) associated with the detected object. In some embodiments, processing unit 110 may include an estimation technique that uses a series of time-based observations, such as a Kalman filter or linear quadratic estimation (LQE) and / or different object types (eg, cars, trucks, pedestrians, etc.). Measurements can be constructed based on the modeling data available for a bicycle, road sign, etc.). The Kalman filter may be based on a scale measurement of the object, where the scale measurement is proportional to the time to collision (eg, the amount of time for the vehicle 200 to reach the object). Thus, by performing steps 540-546, processing unit 110 identifies vehicles and pedestrians appearing in the set of captured images and provides information (eg, position, speed, size) associated with the vehicles and pedestrians. Can be derived. Based on the identified and derived information, processing unit 110 may cause one or more navigation responses at vehicle 200, as described above in connection with FIG. 5A.

  [0165] At step 548, the processing unit 110 performs an optical flow analysis of the one or more images to reduce the probability of detecting "false hits" and missing candidate objects representing vehicles or pedestrians. I can do it. Optical flow analysis may refer to, for example, analyzing a movement pattern that is separate from road surface movement for vehicle 200 in one or more images associated with other vehicles and pedestrians. Processing unit 110 may calculate the movement of the candidate object by observing different positions of the object over multiple image frames captured at different times. The processing unit 110 may use the position and time values as inputs to the mathematical model to calculate the movement of the candidate object. Thus, optical flow analysis may provide another method of detecting vehicles and pedestrians near vehicle 200. The processing unit 110 may perform optical flow analysis in combination with steps 540-546 to provide redundancy for detecting vehicles and pedestrians and increase the reliability of the system 100.

  [0166] FIG. 5C is a flowchart illustrating an exemplary process 500C for detecting road marks and / or lane geometry information in a set of images, according to a disclosed embodiment. Processing unit 110 may execute monocular image analysis module 402 to perform process 500C. At step 550, processing unit 110 may detect the set of objects by scanning one or more images. To detect lane mark segments, lane geometry information, and other relevant road marks, processing unit 110 filters the set of objects to determine that they are irrelevant (eg, small holes, small rocks, etc.). May be excluded. At step 552, processing unit 110 may group together the segments detected at step 550 belonging to the same road mark or lane mark. Based on the grouping, processing unit 110 may develop a model, such as a mathematical model, to represent the detected segments.

  [0167] At step 554, processing unit 110 may construct a set of measurements associated with the detected segment. In some embodiments, processing unit 110 may create a projection of the detected segment from the image plane to the real world plane. The projection may be characterized using a cubic polynomial having coefficients corresponding to physical properties such as detected road position, slope, curvature, and derivative of curvature. In generating the projection, processing unit 110 may take into account road changes and pitch and roll rates associated with vehicle 200. In addition, the processing unit 110 may model the road height by analyzing the position and the motion cues present on the road surface. Further, processing unit 110 may estimate a pitch rate and a roll rate associated with vehicle 200 by tracking the set of feature points in one or more images.

  [0168] In step 556, the processing unit 110 may perform a multi-frame analysis, for example, by tracking the detected segments over successive image frames and accumulating per-frame data associated with the detected segments. If the processing unit 110 performs a multi-frame analysis, the set of measurements constructed in step 554 may be more reliable and may be associated with more and more confidence. Thus, by performing steps 550-556, processing unit 110 may identify roadmarks that appear in the set of captured images and derive lane geometry information. Based on the identified and derived information, processing unit 110 may cause one or more navigation responses at vehicle 200, as described above in connection with FIG. 5A.

  [0169] At step 558, the processing unit 110 may further develop a safety model of the vehicle 200 in situations surrounding the vehicle, taking into account additional sources of information. The processing unit 110 may use the safety model to define situations in which the system 100 may safely perform autonomous control of the vehicle 200. To develop a safety model, in some embodiments, processing unit 110 may include the location and movement of other vehicles, detected road edges and barriers, and / or map data (such as data from map database 160). ) May be considered. By considering additional sources of information, processing unit 110 may provide redundancy for detecting road marks and lane geometries, which may increase the reliability of system 100.

  [0170] FIG. 5D is a flowchart illustrating an exemplary process 500D for detecting a traffic light in a set of images, according to a disclosed embodiment. Processing unit 110 may execute monocular image analysis module 402 to perform process 500D. At step 560, processing unit 110 may scan the set of images and identify objects that appear at locations in the image that are likely to include traffic lights. For example, the processing unit 110 may filter the identified objects to build a set of candidate objects that exclude objects that are unlikely to correspond to a traffic light. Filtering may be based on various characteristics associated with the traffic light, such as shape, dimensions, texture, and location (eg, for vehicle 200). Such characteristics may be based on many examples of traffic lights and traffic control signals and may be stored in a database. In some embodiments, processing unit 110 may perform a multi-frame analysis on a set of candidate objects reflecting possible traffic lights. For example, processing unit 110 may track the candidate object over successive image frames, estimate the real-world position of the candidate object, and filter out moving (less likely traffic lights) objects. In some embodiments, the processing unit 110 may perform a color analysis on the candidate object to identify the relative positions of the detected colors represented within the potential traffic light.

  [0171] At step 562, processing unit 110 may analyze the geometry of the intersection. The analysis is extracted from (i) the number of lanes detected on both sides of the vehicle 200, (ii) marks (arrow marks etc.) detected on roads, and (iii) map data (data from map database 160 etc.). May be based on any combination of the intersection descriptions. The processing unit 110 may perform the analysis using information derived from the execution of the monocular analysis module 402. In addition, the processing unit 110 may identify the correspondence between the traffic light detected in step 560 and the lane appearing near the vehicle 200.

  [0172] As the vehicle 200 approaches the intersection, at step 564, the processing unit 110 may update the analyzed intersection geometry and the confidence associated with the detected traffic light. For example, the number of traffic lights estimated to appear at an intersection, as compared to the number actually appearing at an intersection, can affect reliability. Thus, based on the reliability, the processing unit 110 may delegate control to the driver of the vehicle 200 to improve the safety situation. By performing steps 560-564, processing unit 110 may identify traffic signals appearing in the set of captured images and analyze intersection geometry information. Based on the identification and analysis, the processing unit 110 may generate one or more navigation responses at the vehicle 200, as described above in connection with FIG. 5A.

[0173] FIG. 5E is a flowchart of an exemplary process 500E for producing one or more navigation responses in a vehicle 200 based on a vehicle path, according to a disclosed embodiment. At step 570, processing unit 110 may build an initial vehicle path associated with vehicle 200. Vehicle path, the coordinates (x, y) may represent using a set of points represented by the distance d _i between two points in the set of points may be in the range of 1 to 5 meters. In one embodiment, processing unit 110 may construct an initial vehicle path using two polynomials, such as left and right road polynomials. The processing unit 110 calculates the geometric midpoint between the two polynomials and, if there is a predetermined offset (offset 0 may correspond to running in the center of the lane), a predetermined offset (eg, smart lane) Offset) may offset each point included in the resulting vehicle path. The offset may be in a direction perpendicular to the segment between any two points in the vehicle path. In another embodiment, processing unit 110 uses one polynomial and estimated lane width to determine each point in the vehicle path by half the estimated lane width plus a predetermined offset (eg, a smart lane offset). Can be offset.

[0174] At step 572, the processing unit 110 may update the vehicle route constructed at step 570. Processing unit 110, the distance between two points d _k in the set of points representing a vehicle path, to be shorter than the distance d _i described above, using a higher resolution, the vehicle path built in 570 Can be rebuilt. For example, the distance d _k may range from 0.1 to 0.3 meters. Processing unit 110 may reconstruct the vehicle path using a parabolic spline algorithm, which may result in a cumulative distance vector S corresponding to the overall length of the vehicle path (ie, based on a set of points representing the vehicle path). .

[0175] At step 574, the processing unit 110 may identify a look-ahead point (represented by coordinates as ( _xl , _zl )) based on the updated vehicle path performed at step 572. The processing unit 110 may extract a look-ahead point from the accumulated distance vector S, and may associate the look-ahead point with a look-ahead distance and a look-ahead time. The look-ahead distance may have a lower range of 10-20 meters and may be calculated as the product of the speed of the vehicle 200 and the look-ahead time. For example, as the speed of the vehicle 200 decreases, the look-ahead distance may also decrease (eg, until a lower limit is reached). The look-ahead time, which may be in the range of 0.5 to 1.5 seconds, may be inversely proportional to the gain of one or more control loops associated with producing a navigation response in vehicle 200, such as a progress error tracking control loop. . For example, the gain of the progress error tracking control loop may depend on bandwidths such as yaw rate loops, steering actuator loops, and vehicle lateral dynamics. Therefore, the higher the gain of the progress error tracking control loop, the shorter the look-ahead time.

[0176] At step 576, processing unit 110 may determine a progress error and a yaw rate command based on the look-ahead point identified at step 574. Processing unit 110, the inverse tangent of prefetch points, by calculating, for example, _{arctan (x l / z l)} , may identify the progression errors. Processing unit 110 may determine the yaw rate command as a product of the progress error and the high level control gain. The high level control gain may be equal to (2 / look-ahead time) if the look-ahead distance is not at the lower limit. If the look-ahead distance is the lower limit, the high level control gain may be equal to (2 ^* speed of vehicle 200 / look-ahead distance).

  [0177] FIG. 5F is a flowchart illustrating an exemplary process 500F for determining whether a preceding vehicle is changing lanes, according to a disclosed embodiment. At step 580, processing unit 110 may identify navigation information associated with a preceding vehicle (eg, a vehicle traveling in front of vehicle 200). For example, processing unit 110 may determine the position, speed (eg, direction and speed), and / or acceleration of a leading vehicle using techniques described above in connection with FIGS. 5A and 5B. Processing unit 110 may use one or more road polynomials, look-ahead points (associated with vehicle 200), and / or snail trails (e.g., taken by a preceding vehicle) using the techniques described above in connection with FIG. 5E. A set of points that describe a route can also be specified.

  [0178] At step 582, the processing unit 110 may analyze the navigation information identified at step 580. In one embodiment, processing unit 110 may calculate the distance between the snail trail and the road polynomial (eg, along the trail). This difference in distance along the trail is determined by a predetermined threshold (e.g., 0.1-0.2 m for straight roads, 0.3-0.4 m for loosely curved roads, 0.5-0.4 m for sharply curved roads). If it is greater than 0.6 meters, the processing unit 110 may determine that the preceding vehicle is likely to be changing lanes. If it is detected that multiple vehicles are traveling in front of vehicle 200, processing unit 110 may compare the snail trail associated with each vehicle. Based on the comparison, processing unit 110 may determine that a vehicle whose snail trail does not match the snail trail of another vehicle is likely to be changing lanes. The processing unit 110 may further compare the curvature of the snail trail (associated with the preceding vehicle) with the expected curvature of the road segment on which the preceding vehicle is traveling. The expected curvature may be extracted from map data (eg, data from map database 160), road polynomials, snail trails of other vehicles, prior knowledge of the road, and the like. If the difference between the curvature of the snail trail and the expected curvature of the road segment exceeds a predetermined threshold, the processing unit 110 may determine that the preceding vehicle is likely to be changing lanes.

[0179] In another embodiment, the processing unit 110 compares the instantaneous position of the preceding vehicle with a look-ahead point (associated with the vehicle 200) over a specific time period (eg, 0.5 to 1.5 seconds). I can do it. The difference between the distance between the instantaneous position of the preceding vehicle and the look-ahead point during a specific time period and the cumulative sum of the differences are determined by a predetermined threshold value (e.g., 0.3 to 0.4 meters for a straight road, a road having a gentle curve). In this case, the processing unit 110 may determine that there is a high possibility that the preceding vehicle is changing lanes. In another embodiment, the processing unit 110 may analyze the snail trail geometry by comparing the lateral distance traveled along the trail with the expected curvature of the snail trail. The expected radius of curvature is calculated as:
(Δ _z ² + δ _x ² ) / 2 / (δ _x )
Where σ _x represents the horizontal travel distance and σ _z represents the vertical travel distance. If the difference between the lateral travel distance and the expected curvature exceeds a predetermined threshold (e.g., 500-700 meters), the processing unit 110 may determine that the preceding vehicle is likely to be changing lanes. In another embodiment, processing unit 110 may analyze the position of the leading vehicle. If the location of the preceding vehicle obscures the road polynomial (eg, the preceding vehicle overlaps the road polynomial), processing unit 110 may determine that the preceding vehicle is likely to be changing lanes. If the position of the preceding vehicle is such that another vehicle is detected in front of the preceding vehicle and the snail trails of the two vehicles are not parallel, processing unit 110 may determine that the (closer) preceding vehicle is changing lanes. Can be determined to be high.

  [0180] At step 584, processing unit 110 may determine whether preceding vehicle 200 is changing lanes based on the analysis performed at step 582. For example, processing unit 110 may make that determination based on a weighted average of the individual analyzes performed in step 582. Under such a scheme, for example, a determination by the processing unit 110 that the preceding vehicle is likely to be changing lanes based on a particular type of analysis may be assigned a value of “1” (“0”). Indicates that it is unlikely that the preceding vehicle is changing lanes). The different analyzes performed in step 582 may be assigned different weights, and the disclosed embodiments are not limited to any particular combination of analysis and weights. Further, in some embodiments, the analysis may utilize a trained system (e.g., a machine learning or deep learning system), such as a system based on images captured at the current location, rather than the current location of the vehicle. The future future path can be estimated.

  [0181] FIG. 6 is a flowchart illustrating an example process 600 for generating one or more navigation responses based on stereoscopic image analysis, according to a disclosed embodiment. At step 610, the processing unit 110 may receive the first and second plurality of images via the data interface 128. For example, cameras included in image acquisition unit 120 (such as image capture devices 122 and 124 having fields of view 202 and 204) capture first and second plurality of images of an area in front of vehicle 200 and provide digital connections ( (Eg, USB, wireless, Bluetooth, etc.). In some embodiments, processing unit 110 may receive the first and second plurality of images via more than one data interface. The disclosed embodiments are not limited to any particular data interface configuration or protocol.

  [0182] At step 620, the processing unit 110 executes the stereoscopic image analysis module 404 to perform stereoscopic image analysis of the first and second plurality of images to create a 3D map of a road in front of the vehicle. Then, features in the image such as lane marks, vehicles, pedestrians, road signs, highway exit ramps, traffic lights, and road hazards can be detected. Stereoscopic image analysis may be performed similarly to the steps described above in connection with FIGS. 5A-5D. For example, the processing unit 110 executes the stereoscopic image analysis module 404 to detect candidate objects (eg, vehicles, pedestrians, road marks, traffic lights, road hazards, etc.) in the first and second plurality of images. Then, a subset of candidate objects may be filtered out based on various criteria, multi-frame analysis may be performed, measurements may be constructed, and confidence of the remaining candidate objects may be determined. In performing the above steps, processing unit 110 may consider information from both the first and second plurality of images, rather than information from only one set of images. For example, processing unit 110 may analyze differences in pixel-level data (or other data subsets from among the two streams of captured images) of the candidate object appearing in both the first and second plurality of images. As another example, processing unit 110 may detect that the object appears in one of the images but not in the other image, or other differences that may exist for the object appearing in the two image streams. , The position and / or velocity (eg, for vehicle 200) of the candidate object may be estimated. For example, the position, speed, and / or acceleration for vehicle 200 may be determined based on the trajectory, position, movement characteristics, etc., of features associated with objects appearing in one or both of the image streams.

  [0183] At step 630, the processing unit 110 executes the navigation response module 408 to perform one or more operations on the vehicle 200 based on the analysis performed at step 620 and the techniques described above in connection with FIG. A navigation operation may occur. Navigation responses may include, for example, turns, lane shifts, acceleration changes, speed changes, braking, and the like. In some embodiments, processing unit 110 may use data derived from execution of velocity and acceleration module 406 to generate one or more navigation responses. Further, multiple navigation responses may occur simultaneously, sequentially, or in any combination thereof.

  FIG. 7 is a flowchart illustrating an exemplary process 700 for generating one or more navigation responses based on an analysis of three sets of images, according to a disclosed embodiment. At step 710, processing unit 110 may receive the first, second, and third plurality of images via data interface 128. For example, the cameras included in the image acquisition unit 120 (such as image capture devices 122, 124, and 126 having fields of view 202, 204, and 206) may be first, second, and / or second areas of the front and / or side of the vehicle 200. , And a third plurality of images may be captured and transmitted to the processing unit 110 via a digital connection (eg, USB, wireless, Bluetooth, etc.). In some embodiments, processing unit 110 may receive the first, second, and third plurality of images via more than two data interfaces. For example, each of the image capture devices 122, 124, and 126 may have an associated data interface for communicating data to the processing unit 110. The disclosed embodiments are not limited to any particular data interface configuration or protocol.

  [0185] In step 720, the processing unit 110 analyzes the first, second, and third plurality of images to determine lane marks, vehicles, pedestrians, road signs, highway exit ramps, traffic lights, and roads. Features in the image, such as dangerous goods, can be detected. The analysis may be performed similarly to the steps described above in connection with FIGS. 5A-5D and FIG. For example, the processing unit 110 may perform a monocular image analysis on each of a first, second, and third plurality of images (e.g., with reference to performing the monocular image analysis module 402 and FIGS. 5A-5D). And based on the steps described above). Alternatively, processing unit 110 performs stereoscopic image analysis on the first and second plurality of images, the second and third plurality of images, and / or the first and third plurality of images. (Eg, via execution of the stereoscopic image analysis module 404 and based on the steps described above in connection with FIG. 6). The processed information corresponding to the analysis of the first, second, and / or third plurality of images may be combined. In some embodiments, processing unit 110 may perform a combination of monocular image analysis and stereoscopic image analysis. For example, the processing unit 110 may perform monocular image analysis on the first plurality of images (eg, via execution of the monocular image analysis module 402) and perform stereoscopic image analysis on the second and third plurality of images. (Eg, via execution of the stereoscopic image analysis module 404). The configuration of the image capture devices 122, 124 and 126-including each position and field of view 202, 204 and 206-affects the type of analysis performed on the first, second and third plurality of images. Can be effected. The disclosed embodiments are not limited to the particular configuration of the image capture devices 122, 124, and 126 or the type of analysis performed on the first, second, and third plurality of images.

  [0186] In some embodiments, processing unit 110 may perform tests on system 100 based on the images acquired and analyzed in steps 710 and 720. Such a test may provide an indicator of the overall performance of the system 100 with a particular configuration of the image capture devices 122, 124, and 126. For example, the processing unit 110 may identify the percentage of “false hits” (eg, when the system 100 incorrectly determines the presence of a vehicle or pedestrian) and “overlooks”.

  [0187] At step 730, the processing unit 110 generates one or more navigation responses at the vehicle 200 based on information derived from two of the first, second, and third plurality of images. I can make it. The two choices of the first, second, and third plurality of images may depend on various factors, such as, for example, the number, type, and size of objects detected in each of the plurality of images. The processing unit 110 may determine the quality and resolution of the image, the effective field of view reflected in the image, the number of captured frames, and the extent to which the object or objects of interest actually appear in the frames (eg, the percentage of frames in which the objects appear, The selection can be made based on, for example, the rate at which objects appear in each such frame).

  [0188] In some embodiments, the processing unit 110 may determine the degree to which information derived from one image source is consistent with information derived from other image sources, such that the first, second, And information derived from two of the third plurality of images. For example, processing unit 110 combines the processed information (whether monocular analysis, stereoscopic analysis, or any combination of the two) derived from each of image capture devices 122, 124, and 126. In addition, visual indicators (eg, lane marks, detected vehicles and / or their locations and / or paths, detected traffic lights, etc.) that are consistent across the images captured from each of the image capture devices 122, 124, and 126 ) Can be specified. The processing unit 110 may also filter out inconsistent information across the captured images (eg, vehicles changing lanes, lane models indicating vehicles too close to vehicle 200, etc.). Accordingly, the processing unit 110 may select information derived from two of the first, second, and third plurality of images based on the identification of the consistent and inconsistent information.

  [0189] Navigation responses may include, for example, turns, lane shifts, and acceleration changes. Processing unit 110 may generate one or more navigation responses based on the analysis performed in step 720 and the techniques described above in connection with FIG. The processing unit 110 may also use the data derived from the execution of the speed and acceleration module 406 to generate one or more navigation responses. In some embodiments, processing unit 110 may include a relative position, relative velocity, and / or relative position between vehicle 200 and an object detected in any of the first, second, and third plurality of images. One or more navigation responses may be generated based on the acceleration. Multiple navigation responses may occur simultaneously, sequentially, or in any combination thereof.

[0190] Reinforcement learning and trained navigation system
[0191] The following sections are partially autonomous (eg, one or more drivers assist a system or function), whether autonomous control of the vehicle is fully autonomous (autonomous vehicle). Autonomous driving will be discussed together with systems and methods for achieving autonomous control of vehicles. As shown in FIG. 8, the autonomous driving task can be divided into three main modules including a detection module 801, a driving policy module 803, and a control module 805. In some embodiments, modules 801, 803, and 805 can be stored in memory unit 140 and / or memory unit 150 of system 100 and / or modules 801, 803, and 805 (or a portion thereof). ) Can be stored separately from system 100 (eg, stored in a server accessible by system 100 by wireless transceiver 172). Further, any of the modules disclosed herein (eg, modules 801, 803, and 805) implement techniques associated with trained systems (such as neural or deep neural networks) or untrained systems. Can be.

  [0192] The detection module 801 that can be implemented using the processing unit 110 can process various tasks related to detecting navigational conditions in the environment of the host vehicle. Such tasks may depend on inputs from various sensors and sensing systems associated with the host vehicle. These inputs include images or image streams from one or more on-board cameras, GPS position information, accelerometer outputs, user feedback, user inputs to one or more user interface devices, radar, lidar, etc. May be included. Collect and analyze sensing data, which may include data from cameras and / or any other available sensors along with map information, and go to a "sensing state" that describes information extracted from scenes in the environment of the host vehicle. And can be systematically represented. The detection state includes, among any potential (possible) detection information, the target vehicle, lane marks, pedestrians, traffic lights, road geometry, lane shapes, obstacles, and other objects / vehicles. Detection information relating to the distance, relative speed, and relative acceleration of the vehicle. Supervised machine learning can be performed to create a detection status output based on the detection data provided to the detection module 801. The output of the sensing module may represent a sensed navigation “state” of the host vehicle, which may be sent to the driving policy module 803.

  [0193] Although the detection state can be determined based on image data received from one or more cameras or image sensors associated with the host vehicle, the detection state used for navigation can be any suitable sensor. Alternatively, it can be determined using a combination of sensors. In some embodiments, the detection state can be determined without using the captured image data. In fact, any of the navigation principles described herein may be applicable to sensing states determined based on captured image data as well as sensing states determined using other non-image based sensors. The detection state can also be determined by a source external to the host vehicle. For example, the sensing state may be based on information received from a source remote from the host vehicle (eg, shared from other vehicles, shared from a central server, or information related to the navigation state of the host vehicle). (Based on sensor information, processed status information, etc.) shared from any other source of information).

  [0194] The driving policy module 803, which is described in more detail below and can be implemented using the processing unit 110, determines one or more navigation actions to be performed by the host vehicle depending on the detected navigation state. To implement a desired operation policy. If there is no other agent (e.g., target vehicle or pedestrian) in the environment of the host vehicle, the sensing state input to driving policy module 803 can be processed in a relatively simple manner. This task becomes more complex if the sensing state requires negotiation with one or more other agents. The techniques used to generate the output of the driving policy module 803 may include reinforcement learning (discussed in more detail below). The output of the driving policy module 803 can include at least one navigation operation for the host vehicle, a desired acceleration (which can lead to an updated speed of the host vehicle), among other desired navigation operations. It may specifically include the desired yaw rate, the desired trajectory of the host vehicle.

  [0195] Based on the output from the driving policy module 803, the control module 805, which can also be implemented using the processing unit 110, controls one or more actuators or controlled devices associated with the host vehicle. Instructions can be determined. Such actuators and devices may include an accelerator, one or more steering controls, brakes, signal transmitters, displays, or any other actuator or device that may be controlled as part of a navigation operation associated with the host vehicle. Control theory aspects can be used to generate the output of the control module 805. To implement the desired navigational goals or requirements of the driving policy module 803, the control module 805 may be responsible for determining and outputting commands to controllable components of the host vehicle.

  [0196] Returning to the driving policy module 803, in some embodiments, the driving policy module 803 can be implemented using a trained system trained by reinforcement learning. In other embodiments, the driving policy module 803 can be implemented without machine learning methods by using a specified algorithm to "manually" address various scenarios that may occur during autonomous navigation. However, such an approach, while feasible, can result in driving policies that are too simple, and may lack the flexibility of a trained system based on machine learning. The trained system may be better prepared to handle complex navigation situations and can better determine if a taxi is parked or parked to take or drop passengers, Better determine whether pedestrians are crossing the road ahead of the host vehicle, better balance the unexpected behavior of other drivers with self-defense, and include target vehicles and / or pedestrians Better traverse congested roads, better determine when to interrupt certain navigation rules or augment other rules, and detect undetected but expected conditions (e.g., pedestrians are Etc.) can be better expected. A trained system based on reinforcement learning may be better equipped to cope with a continuous, high-dimensional state space as well as a continuous motion space.

[0197] Training the system using reinforcement learning may include learning a driving policy to map from detected states to navigation actions. The driving policy is a function π: S → A, where S is a set of states,

Is the motion space (eg, desired speed, acceleration, yaw command, etc.). State space is _{S = S S xS p, S} S is a detection state, _{S p} is the additional information for the state that was saved by the policy. Function at discrete time intervals, it is possible to observe the current state s _t ∈S at time t, desired operation by applying the policy a _{t =} π (s _t) can be obtained.

[0198] The system can be trained by exposing it to various navigation states, applying policies to the system, and rewarding (based on reward functions designed to reward the desired navigation behavior). . Based on the reward feedback, the system can "learn" the policy and become trained in creating the desired navigation behavior. For example, the learning system observes the current state s _t ∈S, and

The operation at _t A can be determined based on Based on the determined action (and the performance of that action), the environment moves to the next state _{st +} 1εS for observation by the learning system. For each action determined according to the observed state, the feedback to the learning system is a reward signal r ₁ , r ₂ ,. . . It is.

[0199] The goal of reinforcement learning (RL) is to find the policy π. At time t, in a state s _t, it is usually assumed when there is reward function r _t for measuring the quality of immediate performing operations a _t. However, by performing the operation a _t at time t, the impact on the environment, thus affecting the value of the future state. As a result, when deciding which action to take, one should not only consider current rewards, but also future rewards. In some cases, if the system determines that a lower-reward option makes more rewards in the future if the option is taken here, even if a particular action is associated with a lower reward than another option available, Even the system should perform that action. To formalize this, the policy π and the initial state s are

If the agent starts from the state s ₀ = s and obeys the policy π from there, the probability of the vector (r ₁ ,..., R _T ) is equal to the rewards r ₁ ,. . . , _RT . The value of the initial state s can be determined by the following equation.
[0200]

[0201] Instead of limiting the target period to T, the following formula can be defined for some fixed γ∈ (0, 1) by discounting future rewards.
[0202]

[0203] In any case, the best policy is
[0204]

[0205] The expected value is over the initial state s.

[0206] There are several possible schemes for training a driving policy system. For example, the system can use a mimicry approach (eg, behavior cloning) that learns from state / action pairs, where the action is selected by a good agent (eg, a human) depending on the particular observation state. Things. Suppose a human driver is observed. This observation, as a basis for training the operating policy system, many examples of (s _{t, a t)} form (s _t is a state, a _t is an operation of the human driver) Can be acquired, observed, and used. For example, it is possible to learn the policy [pi using supervised learning as π (s _t) ≒ a t is established. This approach has many potential advantages. First, there is no need to define a reward function. Second, learning is "supervised" and takes place off-line (no need to apply agents within the learning process). A disadvantage of this method is that various human drivers and even the same human driver are not deterministic in terms of their policy choices. _{Therefore, || π (s t) -a} t || it is often impossible to learn very small functions. In addition, even small errors can accumulate gradually to produce large errors.

[0207] Another technique that may be used is policy-based learning. Here, the policy is expressed in a parametric form and can be directly optimized using appropriate optimization techniques (eg, stochastic gradient descent). This technique is

Is to directly solve the problem given by Of course, there are many ways to solve this problem. One advantage of this approach is that it addresses the problem directly, resulting in good practical results in many cases. One potential disadvantage is that this approach often requires “on-policy” training, ie, learning π is an iterative process, and the less-than-perfect policy π _j is used to In order to construct the next policy π _j , one must interact with the environment while operating on π _j .

[0208] The system can also be trained by value-based learning (learning of Q function or V function). Suppose that a good approximation can be learned for the optimal value function V ^* . An optimal policy can be constructed (eg, by using the Bellman equation). Some versions of value-based learning can be performed offline (referred to as "off-policy" training). Some disadvantages of value-based approaches can be attributed to their strong dependence on Markov assumptions and the required approximation of complex functions (it is more difficult to approximate a value function than to directly approximate a policy) Can be).

[0209] Another technique may include model-based learning and planning (learning the probability of state transitions and solving an optimization problem that finds the optimal V). A combination of these techniques can also be used to train a learning system. In this approach, take the dynamics of the process, namely a (s _{t, a t),} it is possible to learn a function that results in a distribution over the next state s _{t + 1.} Once this function has been learned, the optimization problem can be solved to find a policy π whose value is optimal. This is called a "plan." One advantage of this approach is the learning portion "supervised" may be to be applied off-line by observing the triplet _{(s t, a t, s} t + 1). As with the "mimicry" approach, one disadvantage of this approach can be that small errors in the learning process can accumulate and result in a poorly functioning policy.

  [0210] Another approach to training the driving policy module 803 may include decomposing the driving policy function into semantically significant components. Doing so involves manually implementing some of the policies that can guarantee the security of the policy, as well as adaptability to many scenarios, a human-like balance between self-defense / aggression behavior, and Allows other parts of the policy to be implemented using reinforcement learning techniques that can allow human-like negotiations with other drivers. From a technical point of view, reinforcement learning methods combine several methodologies and can provide an easy-to-use training procedure where most of the training can be performed using recorded data or self-constructing simulators.

[0211] In some embodiments, training of the driving policy module 803 may utilize an "alternative" mechanism. To illustrate that, consider a simple scenario of a driving policy for a two-lane highway. In the direct RL method, the state

The first component of the policy π, π (s) that maps to is the desired acceleration command, and the second component of π (s) is the yaw rate. With the modified approach, the following policy can be constructed.

[0212] Automatic Cruise Control (ACC) policy, o _ACC : S → A; This policy always outputs a yaw rate of 0, and changes only the speed in order to implement smooth and accident-free driving.

[0213] ACC + Left policy, o _L : S → A; The vertical command of this policy is the same as the ACC command. Yaw rate is a simple implementation of centering a vehicle towards the center of the left lane, while ensuring safe lateral movement (eg, not moving left if there is a car on the left).

[0214] ACC + Right Policy, o _R : S → A; Same as o _L , but the vehicle may be centered towards the center of the right lane.

[0215] These policies can be called "options." Depending on these "choices", one can learn a policy π _o : S → O for choosing a choice, where O is the set of available choices. In one case, O = {o _ACC , o _L , o _R } holds. For all s

, The choice selector policy π _o defines the actual policy, π: S → A.

  In practice, the policy function can be decomposed into a choice graph 901 as shown in FIG. FIG. 10 shows another example of the option graph 1000. The choice graph may represent a hierarchical set of decisions organized as a directed acyclic graph (DAG). There is a special node called the root node 903 of the graph. This node has no input nodes. The decision process starts at the root node and traverses this graph until it reaches a “leaf” node that points to a node that has no output decision line. As shown in FIG. 9, the leaf nodes may include, for example, nodes 905, 907, and 909. Upon encountering a leaf node, the driving policy module 803 may output acceleration and steering commands associated with a desired navigation operation associated with the leaf node.

  [0217] For example, internal nodes, such as nodes 911, 913, 915, may provide an implementation of a policy that selects a child among its available options. The set of available children of an internal node includes all of the nodes associated with a particular internal node by a decision line. For example, the internal node 913 shown as “merge” in FIG. 9 has three child nodes 909, 915, and 917 (“stay”, “overtake on the right side”, and “overtake on the left side”) respectively connected to the node 913 by a decision line. Each).

  [0218] The flexibility of the decision-making system can be obtained by allowing nodes to adjust their positions within the hierarchy of the choice graph. For example, any of the nodes may be allowed to declare themselves "critical". Each node can implement a function "is critical" that outputs "true" if the node is within a critical section of its policy implementation. For example, a node responsible for takeover may declare itself critical during operation. This may impose constraints on the set of available children of node u, which are all children of node u and which are designated as critical from v to leaf nodes. It may include all nodes v for which there is a path through the nodes. Such an approach may, on the one hand, allow the declaration of the desired path on the graph at each time step, while, on the other hand, preserving the stability of the policy, especially while critical parts of the policy are being enforced. Can be.

  [0219] By defining a graph of alternatives, the problem of learning the driving policy π: S → A can be decomposed into a problem defining the policy of each node of the graph, and the policy of the internal nodes can be used. Should be selected from among the child nodes. For some of the nodes, individual policies can be implemented manually (eg, by an if-then algorithm that specifies a set of behaviors depending on the observed state), while for others, reinforcement learning Policies can be implemented using a trained system built by. The choice of a manual approach or a trained / learned approach may depend on the safety aspects associated with the task and its relative simplicity. The choice graph may be constructed in such a way that some of the nodes are easily implemented while other nodes may depend on the trained model. Such an approach can ensure safe operation of the system.

  [0220] The following discussion provides further details regarding the role of the options graph of FIG. As discussed above, the input to the driving policy module is, for example, a "sensing state" that outlines an environmental map obtained from available sensors. The output of the driving policy module 803 is a set of desires (optionally with a set of strict constraints) that define the trajectory as a solution to the optimization problem.

  [0221] As described above, the option graph represents a set of hierarchical decisions organized as a DAG. There is a special node called the "root" of the graph. The root node is the only node that has no input edges (eg, decision lines). The decision process starts at the root node and traverses the graph until it reaches a "leaf" node, a node that has no output edges. Each internal node should implement a policy that selects one child from among its available children. Every leaf node should implement a policy that defines a desired set (eg, a set of navigational targets for the host vehicle) based on the entire path from the root to the leaves. The set of desires, together with the set of strict constraints directly defined based on the sensing state, establishes an optimization problem whose solution is the trajectory of the vehicle. Strict constraints can be used to further enhance the safety of the system, and "desires" can be used to provide driving comfort and human-like driving behavior of the system. The trajectory provided as a solution to the optimization problem thus defines the commands to be given to the steering, brake and / or engine actuators to achieve the trajectory.

  Returning to FIG. 9, the option graph 901 relates to a two-lane highway including a merging lane (meaning that the third lane joins the right lane or the left lane of the highway at some point). Represents a graph of choices. The root node 903 first determines whether the host vehicle is in a simple road scenario or approaching a merge scenario. This is an example of a determination that can be made based on the detection state. The simple road node 911 includes three child nodes: a stay node 909, a left passing node 917, and a right passing node 915. Staying refers to a situation where the host vehicle wants to continue traveling in the same lane. The remaining nodes are leaf nodes (no output edge / line). Thus, the remaining nodes define a set of desires. The first desire defined by this node may include, for example, a desired lateral position as close as possible to the center of the current travel lane. There may also be a desire to navigate smoothly (eg, within a predefined or acceptable acceleration maximum). The staying node can also determine how the host vehicle reacts to other vehicles. For example, the remaining nodes may survey the detected target vehicles and assign each to a semantic meaning that may be translated into a component of the trajectory.

  [0223] Various semantic meanings can be assigned to target vehicles in the environment of the host vehicle. For example, in some embodiments, the semantic meaning may include any of the following indications: 1) Irrelevant: Indicates that the detected vehicle in the scene is not currently involved 2) Neighboring lane: Detect Indicates that the vehicle is in an adjacent lane and should maintain an appropriate offset with respect to the vehicle (the exact offset can be calculated by an optimization problem that constructs the trajectory given the desires and strict constraints) Can, and possibly depend on the vehicle, the remaining leaves of the options graph set the type of semantics of the target vehicle that defines the desire for the target vehicle) 3) Yield: The host vehicle (especially If the host vehicle determines that the target vehicle is likely to break into the lane of the host vehicle, try to yield to the detected target vehicle, for example by decelerating; 5) Follow-up: the host vehicle wants to follow this target vehicle to maintain smooth driving, 6) left / right side Overtaking: This means that the host vehicle wants to initiate a lane change to the left or right lane. The left passing node 917 and the right passing node 915 are internal nodes for which the desire has not yet been determined.

  [0224] The next node in the option graph 901 is a gap selection node 919. This node may be responsible for selecting the gap between two target vehicles in the particular target lane that the host vehicle wants to enter. By selecting a node in the form of IDj, for some value of j, the host vehicle is given a desire for a trajectory optimization problem, for example, a leaf that specifies that the host vehicle wants to take action to reach the selected gap. To reach. Such an operation may include first accelerating / braking within the current lane and proceeding to the target lane at a suitable time to enter the selected gap. If the gap selection node 919 cannot find a suitable gap, it returns to the center of the current lane and proceeds to a stop node 921 that defines the desire to cancel the overtake.

  [0225] Returning to merge node 913, as the host vehicle approaches the merge, the host vehicle has several options that may depend on the particular situation. For example, as shown in FIG. 11A, the host vehicle 1105 is moving along the two-lane road without detecting any other target vehicle in the main lane or the merged lane 1111 of the two-lane road. In this situation, when the operation policy module 803 reaches the merging node 913, it can select the node 909 to stay. That is, if the target vehicle is not detected as joining the road, it may be desirable to stay in the current lane.

  [0226] In FIG. 11B, this situation is slightly different. Here, the host vehicle 1105 detects one or a plurality of target vehicles 1107 entering the main road 1112 from the merging lane 1111. In this situation, when the driving policy module 803 faces the merging node 913, the driving policy module 803 may decide to initiate a left handing operation to avoid a merging situation.

  [0227] In FIG. 11C, the host vehicle 1105 encounters one or more target vehicles 1107 entering the main road 1112 from the merging lane 1111. The host vehicle 1105 also detects a target vehicle 1109 that moves in a lane adjacent to the host vehicle lane. The host vehicle also detects one or more target vehicles 1110 traveling in the same lane as the host vehicle 1105. In this situation, the driving policy module 803 may adjust the speed of the host vehicle 1105 to yield to the target vehicle 1107 and proceed ahead of the target vehicle 1115. This can be achieved, for example, by going to a gap selection node 919 that selects the gap between ID0 (vehicle 1107) and ID1 (vehicle 1115) as the appropriate confluence gap. In that case, the appropriate gap in the confluence situation sets the goal for the trajectory planner optimization problem.

  [0228] As discussed above, nodes in the graph of alternatives may declare themselves "critical", such declaration ensuring that the selected alternative passes through the critical nodes. obtain. Formally, each node can implement the function IsCritical. After performing a forward pass from the root to the leaf on the option graph and solving the trajectory planner optimization problem, a backward pass from the leaf to the root can be performed. Along this backward path, the IsCritical function of every node in the path can be called, and a list of all critical nodes can be saved. The driving policy module 803 may be required to select a path from the root node to a leaf node that passes through all critical nodes in the forward path corresponding to the next time frame.

  [0229] FIGS. 11A-11C can be used to illustrate the potential benefits of this approach. For example, in a situation where the overtaking operation is started and the driving policy module 803 reaches the leaf corresponding to IDk, when the host vehicle is in the middle of the overtaking operation, for example, it is not desirable to select the node 909 to stay. To avoid this jump property, IDj nodes can designate themselves as critical. During operation, the success of the trajectory planner can be monitored, and the function IsCritical returns a "true" value if the overtaking operation proceeds as intended. This approach may ensure that the overtaking operation continues in the next time frame (rather than jumping to another potentially inconsistent operation before completing the first selected operation). On the other hand, if the operation monitoring indicates that the selected operation is not proceeding as intended, or if the operation becomes unnecessary or impossible, the function IsCritical can return a "false" value. This may allow the gap selection node to select a different gap in the next time frame, or abort the overtaking operation altogether. This approach may, on the one hand, allow the declaration of the desired path on the graph of choices at each time step, while on the other hand promoting policy stability during a critical part of the execution. Can help.

  [0230] The strict constraints described in more detail below can be distinguished from the desires of navigation. For example, strict constraints can ensure safe driving by applying an additional filtering layer of planned navigation behavior. Rather than using a trained system built on reinforcement learning, the strict constraints involved that can be programmed and defined manually can be determined from the sensed state. However, in some embodiments, the trained system can learn applicable strict constraints that will be applied and followed. Such an approach may facilitate driving policy module 803 to reach a selected operation that is already compliant with the applicable strict constraints, so that it is later modified to comply with the applicable strict constraints. Can be reduced or eliminated. Nevertheless, as a redundant safety measure, even if the operation policy module 803 is trained to take into account predetermined strict restrictions, strict restrictions can be applied to the output of the operation policy module 803.

  [0231] There are many examples of potential strict constraints. For example, a strict constraint may be defined in relation to a guardrail at a road edge. Under no circumstances shall the host vehicle cross the guardrail. Such rules cause severe lateral constraints on the trajectory of the host vehicle. Another example of a stringent constraint may include road bumps (eg, speed control bumps), which may cause stringent constraints on driving speed before or during and across bumps. Strict constraints can be considered that safety should be paramount, and thus can be manually defined rather than relying exclusively on a trained system to learn the constraints during training.

  [0232] In contrast to strict constraints, the goal of desire may be to enable or achieve comfortable driving. As discussed above, one example of a desire may include the goal of positioning the host vehicle at a lateral position in a lane corresponding to the center of the lane of the host vehicle. Another desire may include the ID of the gap to enter. The host vehicle does not have to be strictly in the center of the lane; instead, the desire to be as close as possible to the center of the lane makes it easier to move to the center of the lane if the host vehicle deviates from the center of the lane Note that this can be guaranteed. The desire does not have to put safety first. In some embodiments, the desire may require negotiation with other drivers and pedestrians. Certain approaches to building desires may utilize a graph of choices, and policies implemented within at least some nodes of the graph may be based on reinforcement learning.

[0233] For the nodes of the graph of options 901 or 1000 implemented as nodes trained based on learning, the training process may include decomposing the problem into a supervised learning phase and a reinforcement learning phase. During the supervised learning phase,

As but _satisfied, from the _{(s t, a} t)

A differentiable mapping to can be learned. This may be similar to "model-based" reinforcement learning. But in the forward loop of the network,

With the actual value of _{st + 1} , thereby eliminating the problem of accumulating errors.

The role of prediction is to convey messages from the future to past actions. In this sense, the algorithm can be a combination of "model-based" reinforcement learning and "policy-based learning".

[0234] An important factor that may be provided in some scenarios is the differentiable path from future losses / rewards to return to operational decisions. In the structure of a graph of choices, the implementation of choices, including safety constraints, is usually not differentiable. To overcome this problem, the choice of children at the nodes of the policy being learned can be probabilistic. That is, a node can output a probability vector p, which assigns a probability to be used in selecting each of the children of a particular node. Assuming a node has k children, a ⁽¹⁾ ,. . . , A ^(k) be the operation of the path from each child to the leaf. Thus, the resulting prediction action is

Which may result in a differentiable path from the action to p. In practice, the action a can be chosen to be a ⁽ⁱ⁾ with respect to ip, where a and

Can be referred to as additional noise.

[0235] _s t, as given to _{a t,}

Can be used with the actual data to train. A simulator can be used to train a node's policy. Later, fine tuning of the policy can be realized using actual data. Two concepts can make the simulation more realistic. First, one can construct an initial policy using imitation and a "behavioral cloning" paradigm that uses large real-world datasets. In some cases, the resulting agent may be suitable. In other cases, the resulting agent forms at least a very good initial policy for other agents on the road. Second, training can be enhanced using self-play and using our unique policy. For example, given an initial implementation of other agents (vehicles / pedestrians) that may be encountered, a policy may be trained based on a simulator. Some of the other agents can be replaced with new policies and the process can be repeated. As a result, the policy can continue to improve as it should respond to a wider variety of other agents with different levels of sophistication.

  [0236] Further, in some embodiments, the system can implement a multi-agent approach. For example, the system may consider data from various sources and / or images captured from multiple angles. In addition, the consideration of events that are not directly related to the host vehicle, but may have an impact on the host vehicle, and that may result in unpredictable situations involving other vehicles are considerations. Because of the possibilities (eg, radar can “see through” inevitable events affecting leading vehicles and host vehicles, as well as anticipating the high likelihood of such events), some embodiments disclosed It can result in energy savings.

[0237] Trained system with imposed navigation constraints
[0238] In the context of autonomous driving, a key concern is how to ensure that the learned policies of the trained navigation network are secure. In some embodiments, the driving policy system can be trained using the constraints, so that the actions selected by the trained system may already take into account applicable safety constraints. In addition, some embodiments provide an additional layer of security by passing selected actions of the trained system through one or more stringent constraints dictated by a particular sensing scene in the environment of the host vehicle. can do. Such an approach may ensure that the actions performed by the host vehicle are limited to those that are verified to meet applicable safety constraints.

  [0239] At its core, the navigation system may include a learning algorithm based on a policy function that maps the observed state to one or more desired actions. In some implementations, the learning algorithm is a deep learning algorithm. The desired action may include at least one action that is expected to maximize an expected reward for the vehicle. In some cases, the actual action taken by the vehicle may correspond to one of the desired actions, but in other cases, the actual action taken may be the observed state, one or more desired actions. , And non-learning strict constraints imposed on the learning navigation engine (eg, safety constraints). These constraints may include non-driving areas surrounding various types of detected objects (e.g., target vehicles, pedestrians, roadside or on-road static objects, roadside or on-road moving objects, guardrails, etc.). . In some cases, the size of the area may vary based on the detected movement (eg, speed and / or direction) of the detected object. Other constraints include the maximum speed of travel of a pedestrian as it passes through the zone of influence, the maximum deceleration (to account for the spacing of the target vehicle behind the host vehicle), the mandatory pedestrian crossing or level crossing. It may include a stop or the like.

  [0240] Strict constraints used with systems trained by machine learning can provide some degree of safety in autonomous driving that can exceed the level of safety that can be obtained based on the output of only the trained system. For example, a machine learning system may be trained using a desired set of constraints as a training guideline, so that a trained system may perform actions that consist of applicable navigational constraints and satisfy those constraints, The selection can be made according to the detected navigation state. However, when selecting a navigation action, the trained system still has some flexibility, so the action selected by the trained system may not strictly follow the relevant navigation constraints There can be at least some situations. Therefore, using non-machine learning components that guarantee the strict application of relevant navigation constraints outside the learning / training framework to require that the selected action closely follow the relevant navigation constraints, The outputs of the trained system can be combined, compared, filtered, adjusted, modified, etc.

[0241] The following discussion illustrates the potential benefits (especially from a safety perspective) that can be obtained from combining trained systems and trained systems with algorithmic components that are outside the training / learning framework. Further details regarding As discussed above, the policy-based reinforcement learning objectives can be optimized by increasing the probability gradient. The purpose (for example, expected reward)

Can be defined as

[0242] In machine learning scenarios, objectives including expected values can be used. However, such purposes may not return operations that are strictly constrained by navigational constraints without being constrained by those constraints. For example, for a trajectory that represents a rare "turn" event to be avoided (for example, an accident)

Holds, and for the rest of the orbit,

Considering the reward function that holds, one goal of the learning system may be to learn to perform the overtaking operation. Usually, in an accident-free orbit,

Rewards a smooth pass without problems and penalizes staying in the lane, and thus in the range [-1,1], without completing the overtaking. sequence

If represents an accident, the reward -r should give a penalty high enough to prevent such occurrences. The question is what value of r should be in order to guarantee accident-free operation.

[0243]

Effect of the accident for are additive term -p ^r, p is noted that the probability mass trajectory with events of the accident. If this term is negligible, i.e., p << 1 / r, then the learning system may decide to overtake the overtaking operation more frequently than the more self-defense policy at the expense of not completing some overtaking operations successfully. To achieve success, the policy of causing an accident can be prioritized (or a generally careless driving policy can be employed). In other words, if the probability of an accident is less than or equal to p, r must be set such that r >> 1 / p. It may be desirable to make p extremely small (for example, about p = 10 ⁻⁹ ). Therefore, r should be large. In the policy gradient,

Can be estimated. The following lemma is for random variables

Is greater than r at r >> 1 / p

It shows that it becomes larger with. Thus, estimating the objective may be difficult, and estimating its gradient may be more difficult.

Lemma: Let π _o be a policy and p and r be scalars, so that at probability p,

Is obtained, and at the probability 1-p,

Is obtained. Therefore,
[0245]

The following approximation holds, and the final approximation corresponds to the case where r ≧ 1 / p.

[0247] This explanation is in the form

Indicates that functional objects may not be able to guarantee functional security without causing a distribution problem. Baseline subtraction methods to reduce variance may not provide sufficient treatment for the problem because the problem is:

Because the estimate shifts to an equally high variance of the baseline constant, which is equally subject to numerical instability. Further, if the probability of an accident is p, at least 1 / p sequences should be sampled on average before obtaining the event of the accident. this is,

Imply the lower end of the 1 / p sample of the sequence for the learning algorithm that seeks to minimize A solution to this problem can be found in the architectural designs described herein, rather than by numerical tuning techniques. The approach here is based on the idea that strict constraints should be put outside the learning framework. In other words, the policy function can be decomposed into a learnable part and a non-learnable part. Formally, the policy function is

Can be configured as

Maps the (agnostic) state space to a set of wishes (eg, desired navigational goals, etc.), while π ^(T) maps the wishes (to determine how a car should move Map) to orbit. function

Are the driving comfort, and whether to overtake any other vehicle, give way to any other vehicle, what is the desired position of the host vehicle in the host vehicle's lane, etc. Responsible for making strategic decisions. The mapping of detected navigational states to wishes is a policy that can be learned from experience by maximizing expected rewards.

It is.

Can be converted into a cost function over the driving trajectory. By finding a trajectory that minimizes the cost subject to strict constraints on functional security, a function π ^(T) that is not a learned function can be implemented. This disassembly can provide comfortable driving and at the same time guarantee functional safety.

[0248] The bi-overflow navigation situation shown in FIG. 11D gives an example that further illustrates these concepts. In the double polymerization flow, the vehicle reaches the merging area 1130 from both the left side and the right side. From each side, it can be determined whether a vehicle, such as vehicle 1133 or vehicle 1135, merges into the lane opposite merging area 1130. Running congested traffic successfully in congested traffic can require significant negotiation skills and experience, and heuristics by listing all possible trajectories that all agents in the scene can perform. It can be difficult to implement in an approach or brute force approach. In this example of a dual polymerization stream, a desired set D suitable for dual polymerization operation can be defined. D is the Cartesian product of the following set: D = [0, v _max ] × Lx {g, t, o}
Where [0, v _max ] is the desired target speed of the host vehicle, and L = {1,1.5,2,2.5,3,3.5,4} , The desired lateral position in lanes, where the integer indicates the center of the lane, the fraction indicates the boundary of the lane, and {g, t, o} is assigned to each of the other n vehicles. This is the classification label that is used. If the host vehicle should give way to another vehicle, it can be assigned a "g" to the other vehicle, and if the host vehicle should get way to another vehicle, it can be assigned a "t" to the other vehicle. Or if the host vehicle is to maintain an offset distance with respect to other vehicles, the other vehicle may be assigned an "o".

The following is a description of how the set of desires (v, l, c ₁ ,..., C _n ) ∈D can be converted to a cost function over the driving trajectory. The driving trajectory is (x ₁ , y ₁ ),. . . , (X _k , y _k ), where (x _i , y _i ) is the (lateral, vertical) position of the host vehicle (in self-centered units) at time τ · i. is there. In some experiments, τ = 0.1 seconds and k = 10. Of course, other values can be selected. The cost assigned to the trajectory may include the desired speed, lateral position, and a weighted sum of the individual costs assigned to the labels assigned to each of the other n vehicles.

[0250] Given the desired velocity v∈ [0, v _max ], the cost of the trajectory associated with the velocity

It is.

[0251] Given a desired lateral position l∈L, the cost associated with the desired lateral position

[0252] Here, dist (x, y, l) is the distance from the point (x, y) to the position 1 of the lane. With respect to the cost due to other vehicles, for any other vehicle, (x ′ ₁ , y ′ ₁ ),. . . , (X ′ _k , y ′ _k ) can represent other vehicles in self-centered units of the host vehicle, and i is (x _i , y _i ) and (x ′ _j , y ′ _j) ) May be the earliest point at which j exists such that the distance between them is small. If there is no such point, i can be set as i = ∞. If another vehicle is classified as "Yield", it may be desirable that τi> τj + 0.5, since at least 0.5 seconds after the other vehicle reaches the intersection of the tracks, the host vehicle Reaches the same point. A possible formula for converting the above constraint into cost is [τ (ji) +0.5] ₊ .

Similarly, if another car is classified as “take the road”, it may be desirable that τj> τi + 0.5, which is the cost [τ (i−j) +0.5]. _{+ Can} be converted. If another vehicle is classified as "offset," it may be desirable for i = ∞, which means that the trajectory of the host vehicle and the trajectory of the offset vehicle do not intersect. This condition can be translated into cost by penalizing the distance between the orbits.

[0254] Assigning a weight to each of these costs may give a single objective function π ^(T) for the trajectory planner. The cost to promote smooth driving can be added for the purpose. Strict constraints can be added for the purpose to guarantee the functional safety of the track. For _{example, (x} i, _{y i)} is forbidden to deviate from the road _{obtained, (x} i, _{y i)} is | i-j | any in the other case is small any track point of the vehicle (x It can be barred from approaching ( _x'j , _y'j ) for ' _j , _y'j ).

In summary, the policy π _θ can be decomposed into mapping from agnostic states to sets of wishes and mapping from wishes to real orbits. The latter mapping is not based on learning and can be implemented by solving an optimization problem whose cost depends on desire and whose strict constraints can guarantee the functional security of the policy.

[0256] The following commentary describes the mapping from agnostic states to sets of desires. As described above, systems that rely solely on reinforcement learning to comply with functional safety

You may face a large and cumbersome variance of The result of this is to use a policy gradient iteration to map the (agnostic) state space to a set of wishes, followed by a real trajectory that does not include a system trained based on machine learning. This can be avoided by decomposing the problem into mapping.

[0257] For various reasons, decision making can be further broken down into semantically significant components. For example, the size of D may be large and even continuous. In the dual polymerization flow scenario described above with respect to FIG. 11D, D = [0, v _max ] xLx {g, t, o} ⁿ ) holds. In addition, the gradient estimator is

May be included. In this equation, the variance can increase with the target period T. In some cases, the value of T can be approximately 250, which can be high enough to create significant variance. Assuming the sampling rate is in the range of 10 Hz and the merging area 1130 is 100 meters, preparation for merging may begin about 300 meters before the merging area. If the host vehicle moves at 16 meters / second (about 60 km / h), the value of T for the episode may be approximately 250.

Returning to the concept of the graph of options, FIG. 11E shows a graph of options that can represent the dual polymerization flow scenario shown in FIG. 11D. As discussed above, the graph of choices may represent a hierarchical set of decisions organized as a directed acyclic graph (DAG). There may be a special node in the graph called the "root" node 1140, and the root node is the only node that has no input edges (eg, decision lines). The decision process may start at the root node and traverse the graph until it reaches a “leaf” node, ie, a node that has no output edges. Each internal node can implement a policy function that selects one child from its available children. There can be a default mapping from the transverse set on the choices graph to the desired set D. In other words, the traversal of the choice on the graph can be automatically converted to a desire in D. Given a node v in the graph, the parameter vector θ _v may define a policy that selects children of v. If theta is a concatenation of all theta _v, using the policy defined by theta _v at each node v, by traversing the leaves from the root of the graph while selecting a child node,

Can be determined.

  [0259] In the graph 1139 of the dual polymerization flow option of FIG. 11E, whether the host vehicle is within the merging region (eg, region 1130 of FIG. 11D) or, alternatively, the host vehicle is approaching the merging region, and The root node 1140 can first determine if it needs to prepare for the resulting merger. In either case, the host vehicle may need to determine whether to change lanes (eg, to the left or right) or to stay in the current lane. If the host vehicle decides to change lanes, the host vehicle needs to continue and determine if the conditions are appropriate for performing a lane change operation (eg, at “go” node 1142). possible. If it is not possible to change lanes, the host vehicle may attempt to enter the desired lane (eg, at node 1144 as part of negotiations with vehicles in the desired lane) by aiming to be on the lane mark. You can try to "push" toward it. Alternatively, the host vehicle may choose to “stay” in the same lane (eg, at node 1146). This process can determine the lateral position of the host vehicle in a natural way. For example.

  [0260] This may allow the desired lateral position to be determined in a natural way. For example, if the host vehicle changes lane from lane 2 to lane 3, the "go" node may set the desired lateral position to 3, and the "stay" node may set the desired lateral position to 2 And the "push forward" node can set the desired lateral position to 2.5. The host vehicle can then decide whether to keep the "same" speed (node 1148), "accelerate" (node 1150), or "decelerate" (node 1152). Next, the host vehicle can enter a “chain” structure 1154 that examines other vehicles and sets their semantic meaning to values in the set {g, t, o}. This process may set desires for other vehicles. The parameters of all nodes in this chain can be shared (similar to a recurrent neural network).

  [0261] The potential benefit of an option is the interpretability of the result. Another potential benefit is that the decomposable structure of set D can be exploited, so that the policy at each node can be chosen from a small number of possibilities. In addition, the structure may allow the variance of the policy gradient estimator to be reduced.

  [0262] As discussed above, the length of an episode in a dual polymerization flow scenario may be approximately T = 250 steps. This value (or any other suitable value depending on the particular navigation scenario) may provide sufficient time to acknowledge the consequences of host vehicle operation (eg, the host vehicle changes lanes in preparation for a merge) If it decides to do so, the host vehicle will only benefit after completing the merge successfully.) On the other hand, the dynamics of driving require that the host vehicle make decisions at a sufficiently fast frequency (eg, 10 Hz in the above case).

  [0263] The graph of choices may allow the effective value of T to be reduced in at least two ways. First, given a high level decision, a reward can be set for a low level decision taking into account shorter episodes. For example, if the host vehicle has already selected the “lane change” and “go” nodes, then by watching a few seconds of episodes, the policy to assign semantic meaning to the vehicle may be learned (T Is 20 to 30 instead of 250). Second, for high-level decisions (such as changing lanes or staying in the same lane), the host vehicle may not need to make decisions every 0.1 seconds. Alternatively, the host vehicle may be able to make decisions less frequently (e.g., every second) or may implement an "end choice" function, where the slope is calculated only after each end of the choice. obtain. In each case, the effective value of T may be an order of magnitude less than its original value. Overall, the estimates at all nodes can depend on the value of T an order of magnitude smaller than the original 250 steps, which can immediately turn into smaller variances.

  [0264] As discussed above, strict constraints can promote safer driving and there can be several different types of constraints. For example, static strict constraints can be defined directly from the sensing state. These may include deceleration bumps, speed limits, road curvature, intersections, etc. in the host vehicle's environment that may imply one or more constraints on the speed, heading, acceleration, braking (deceleration), etc. of the vehicle. Static strict constraints can also include semantic free space, where the host vehicle is forbidden, for example, from going outside free space and navigating too close to physical barriers. You. Static strict constraints can also limit (eg, prohibit) operations that do not follow various aspects of the kinematics of the vehicle, for example, static strict constraints can cause the host vehicle to roll over Can be used to inhibit operations that can lead to slipping, or otherwise losing control.

[0265] Strict constraints can also be related to vehicles. For example, a constraint may be used that requires a vehicle to maintain a vertical distance of at least one meter to other vehicles and a lateral distance of at least 0.5 meters to other vehicles. Constraints can also be applied to prevent the host vehicle from maintaining a collision course with one or more other vehicles. For example, time τ may be a measure of time based on a particular scene. The predicted trajectory of the host vehicle and one or more other vehicles from the current time to time τ can be considered. If two orbits intersect,

May represent the time at which the vehicle i arrives at the intersection and the time at which it leaves the intersection. That is, each car needs a certain amount of time to reach a point when the first part of the car passes the intersection and the last part of the car passes the intersection. This time separates the arrival time from the departure time.

(I.e., the arrival time of vehicle 1 is less than the arrival time of vehicle 2), and it is desired to ensure that vehicle 1 has left the intersection before vehicle 2 arrives. Otherwise, a collision will occur. Therefore,

Strict constraints can be implemented as Further, to ensure that vehicle 1 and vehicle 2 do not hit each other by a minimum amount, additional safety margins are included by including a buffer time (eg, 0.5 seconds or another suitable value) in the constraint. Obtainable. The exact constraints related to the predicted intersection trajectory of the two vehicles are:

Can be expressed as

  [0266] The time τ of tracking the trajectory of the host vehicle and one or more other vehicles may be different. However, in an intersection scenario where speed may be slow, τ may be longer and τ may be defined such that the host vehicle enters and exits the intersection in less than τ seconds.

  [0267] Of course, applying strict constraints to the trajectories of vehicles requires that the trajectories of those vehicles be predicted. For the host vehicle, trajectory prediction can be relatively straightforward, as the host vehicle generally already knows and actually plans the intended trajectory at any given time. For other vehicles, predicting the trajectory of those vehicles may not be very straightforward. In other vehicles, the baseline calculation to determine the predicted trajectory is captured, for example, by one or more cameras and / or other sensors (radar, lidar, sound, etc.) mounted on the host vehicle. It may depend on the current speed and heading of other vehicles determined based on the analysis of the image stream.

  [0268] However, there may be some exceptions that simplify the problem or at least provide additional confidence in the predicted trajectory for another vehicle. For example, for a structured road with lane indications and rules to give way, the trajectory of the other vehicle may be based at least in part on the location of the other vehicle with respect to the lane and the applicable road Can be based on a rule that yields. Thus, in some situations, if there is a lane structure to be observed, it can be assumed that vehicles in adjacent lanes will adhere to lane boundaries. That is, unless the host vehicle has observed evidence (e.g., traffic lights, strong lateral movement, or movement across lane boundaries) indicating that a vehicle in the adjacent lane is breaking into the host vehicle's lane. It can be assumed that vehicles in that lane will stay in their lane.

  [0269] Other situations may also give hints about the expected trajectory of other vehicles. For example, on a stop sign, a traffic light, a roundabout, or the like where the host vehicle may have a priority traffic right, it can be assumed that another vehicle will protect the priority traffic right. Thus, as long as there is no evidence that a rule has been observed to be broken, other vehicles can be assumed to follow a trajectory that respects the priority right of the host vehicle.

  [0270] Strict constraints can also be applied for pedestrians in the environment of the host vehicle. For example, a buffer distance for a pedestrian can be established such that the host vehicle is barred from navigating any closer than a prescribed buffer distance for any observed pedestrian. The pedestrian's buffer distance can be any suitable distance. In some embodiments, the buffer distance may be at least 1 meter for an observed pedestrian.

  [0271] Similar to the vehicle situation, strict constraints can also be applied to the relative movement between the pedestrian and the host vehicle. For example, a pedestrian's trajectory (based on heading direction and speed) can be monitored relative to the predicted trajectory of the host vehicle. Given a particular pedestrian's trajectory, for all points p on the trajectory, t (p) may represent the time it takes for a pedestrian to reach point p. To keep the required buffer distance of at least one meter from the pedestrian, t (p) is (with sufficient time difference so that the host vehicle passes in front of the pedestrian by a distance of at least one meter). T (p) must be greater than the time the host vehicle reaches point p, or (eg, if the host vehicle brakes and yield to a pedestrian), the time the host vehicle reaches point p Must be below. Furthermore, in the latter example, the host vehicle arrives at point p at a time sufficiently slower than the pedestrian so that the host vehicle can pass behind the pedestrian and maintain the required buffer distance of at least one meter. May be required by strict constraints. Of course, there may be exceptions to the exact constraints on pedestrians. For example, if the host vehicle has priority right-of-way or is very slow and there is no observed evidence that pedestrians will refuse to give way to the host vehicle or navigate toward the host vehicle. , Strict pedestrian constraints can be relaxed (eg, to a narrower buffer of at least 0.75 meters or 0.50 meters).

  In some examples, when it is determined that all the conditions cannot be satisfied, the constraint can be relaxed. For example, in situations where the road is too narrow from both curves or a desired distance from the curve and the parked vehicle (e.g., 0.5 meters), one or more of the constraints may be relaxed if there are mitigating events. it can. For example, if there are no pedestrians (or other objects) on the sidewalk, you can go slowly 0.1 m from the curve. In some embodiments, constraints may be relaxed if doing so improves the user experience. For example, to avoid pits, the constraints may be relaxed to allow the vehicle to navigate closer to the edge of a lane, curve, or pedestrian than would normally be allowed. Further, when deciding which constraints to relax, in some embodiments, the one or more constraints that you decide to relax are those that are deemed to have the least adverse effect on safety. . For example, the constraints on how close a vehicle can move to a curve or concrete barrier can be relaxed before relaxing the constraint dealing with proximity to other vehicles. In some embodiments, the pedestrian constraint may be the last to be relaxed or may never be relaxed in some circumstances.

  FIG. 12 illustrates an example of a scene that may be captured and analyzed during navigation of a host vehicle. For example, the host vehicle may receive a plurality of images representing the environment of the host vehicle from cameras associated with the host vehicle (eg, at least one of image capture device 122, image capture device 124, and image capture device 126). Navigation system (eg, system 100). The scene shown in FIG. 12 is one example of an image that may be captured at time t from the environment of a host vehicle traveling in lane 1210 along predicted trajectory 1212. The navigation system includes at least one processing device (e.g., as described above) that is specially programmed to receive the plurality of images and analyze the images to determine motion in response to the scene. (Including either an EyeQ processor or other device). In particular, at least one processing device may implement the sensing module 801, the operating policy module 803, and the control module 805 shown in FIG. The detection module 801 can be responsible for collecting and outputting image information collected from the camera and providing that information to the driving policy module 803 in the form of the identified navigation state, wherein the driving policy module 803 is A trained navigation system that is trained by machine learning methods such as dozens and reinforcement learning can be configured. Based on the navigation state information provided by the detection module 801 to the driving policy module 803, the driving policy module 803 may respond to the host vehicle in response to the identified navigation state (eg, by implementing the above-described alternative graph approach). , A desired navigation operation to be performed can be generated.

  [0274] In some embodiments, at least one processing device can directly translate a desired navigation operation into a navigation command using, for example, the control module 805. However, in other embodiments, strict constraints are applied to test the desired navigation behavior provided by the driving policy module 803 for various predetermined navigation constraints that may be involved by the scene and the desired navigation behavior. be able to. For example, if the driving policy module 803 outputs a desired navigation action that causes the host vehicle to follow the track 1212, the navigation action may be subject to one or more strict constraints related to various aspects of the host vehicle's environment. Can be tested. For example, the captured image 1201 may determine curbs 1213, pedestrians 1215, target vehicles 1217, and static objects (eg, an overturned box) that are present in the scene. Each of these may be associated with one or more strict constraints. For example, the curb 1213 may be associated with a static constraint that prevents the host vehicle from navigating on or over the curb onto the sidewalk 1214. The curb 1213 defines a non-navigable area for the host vehicle, a predetermined distance apart (eg, 0.1 m, 0.25 m, 0.5 m, 1 m, etc.) from the curb and extending along the curb. It may also relate to a road barrier envelope that defines (eg, a buffer zone). Of course, static constraints may relate to other types of roadside boundaries (eg, guardrails, concrete columns, cones, pylons, any other type of roadside barrier).

  [0275] It should be noted that distance and ranging can be determined by any suitable method. For example, in some embodiments, range information may be provided by an onboard radar and / or lidar system. Alternatively or additionally, the distance information may be derived by analyzing one or more images captured from the environment of the host vehicle. For example, the number of pixels of the recognized object represented in the image can be determined and compared to the known field of view and focal length geometry of the image capture device to determine scale and distance. For example, by observing changes in scale between objects in multiple images over a known time interval, velocity and acceleration can be determined. This analysis may indicate how fast the object is away from or approaching the host vehicle, as well as the direction of travel toward or away from the host vehicle. By analyzing the change in position of the X coordinate of the object from one image to another over a known period of time, the traversing speed can be determined.

  [0276] Pedestrian 1215 may be associated with a pedestrian envelope that defines a buffer area 1216. In some cases, the strict constraints imposed may prohibit the host vehicle from navigating within a distance of 1 meter (in any direction relative to the pedestrian) from the pedestrian 1215. The pedestrian 1215 can also determine the location of the pedestrian affected area 1220. This area of influence may be associated with constraints that limit the speed of the host vehicle within the area of influence. The area of influence may extend from pedestrian 1215 by 5 meters, 10 meters, 20 meters, and so on. A different speed limit can be associated with each class of influence zone. For example, within an area of 1 meter to 5 meters from pedestrian 1215, the host vehicle may be at a first speed (e.g., 10 mph or 20 mph, etc.) that may be less than the speed limit in a pedestrian affected area extending 5 meters to 10 meters. ). Any grade for the various stages of the affected area can be used. In some embodiments, the first stage can be less than 1 meter to 5 meters and can only be 1 meter to 2 meters. In other embodiments, the first stage of the affected area may extend from 1 meter (boundary of the non-navigating area around the pedestrian) to a distance of at least 10 meters. Next, the second stage may extend from 10 meters to at least about 20 meters. The second phase may relate to a maximum travel speed of the host vehicle that exceeds a maximum travel speed associated with the first phase of the pedestrian impact zone.

  [0277] One or more static object constraints may also be involved depending on the detected scene in the environment of the host vehicle. For example, in image 1201, at least one processing device may detect a static object, such as box 1219, that is on the road. The detected static objects may include various objects such as trees, poles, road signs, at least one of the objects in the road. One or more predefined navigation constraints may be associated with the detected static object. For example, such a constraint may include a static object envelope, which defines a buffer area around the object within which navigation of the host vehicle may be prohibited. At least a portion of the buffer area may extend a predetermined distance from an edge of the detected static object. For example, in the scene represented by image 1201, a buffer area of at least 0.1 meters, 0.25 meters, 0.5 meters, or more may be associated with box 1219, so that the host vehicle has been detected. Pass the right or left side of the box by at least some distance (eg, the distance of the buffer zone) to avoid colliding with a static object.

  [0278] The predefined strict constraints may also include one or more target vehicle constraints. For example, a target vehicle 1217 may be detected in the image 1201. One or more strict constraints may be used to ensure that the host vehicle does not collide with the target vehicle 1217. In some cases, the target vehicle envelope may be related to the distance of a single buffer zone. For example, the buffer zone may be defined by a one meter distance surrounding the target vehicle in all directions. The buffer zone may define an area extending at least 1 meter from the target vehicle where host vehicles are prohibited from navigating therein.

  [0279] However, the envelope surrounding target vehicle 1217 need not be defined by a fixed buffer distance. In some cases, the predefined strict constraints associated with the target vehicle (or any other movable object detected in the environment of the host vehicle) may depend on the orientation of the host vehicle with respect to the detected target vehicle. For example, in some cases (e.g., when the host vehicle is traveling toward the target vehicle), the distance of the longitudinal buffer zone from the target vehicle to the front or rear of the host vehicle is at least one meter. obtain. For example, if the host vehicle is moving in the same or opposite direction as the target vehicle, so that the side of the host vehicle passes directly beside the side of the target vehicle (for example, from one of the target vehicle to the host vehicle) The distance of the lateral buffer zone (which spans the sides) can be at least 0.5 meters.

[0280] As described above, other constraints may be involved by detecting target vehicles or pedestrians in the environment of the host vehicle. For example, the predicted trajectory of the host vehicle and the target vehicle 1217 can be considered, and the strict constraint when the two trajectories intersect (eg, at intersection 1230) is:

, And the host vehicle is the vehicle 1 and the target vehicle 1217 is the vehicle 2. Similarly, the trajectory of pedestrian 1215 can be monitored (based on heading direction and speed) with respect to the predicted trajectory of the host vehicle. Given a particular pedestrian's trajectory, for all points p on the trajectory, t (p) represents the time it takes for the pedestrian to reach point p (ie, point 1231 in FIG. 12). To keep the required buffer distance of at least one meter from the pedestrian, t (p) is (with sufficient time difference so that the host vehicle passes in front of the pedestrian by a distance of at least one meter). T (p) must be greater than the time the host vehicle reaches point p, or (eg, if the host vehicle brakes and yield to a pedestrian), the time the host vehicle reaches point p Must be below. Further, in the latter example, the host vehicle reaches point p at a time sufficiently slower than the pedestrian so that the host vehicle can pass behind the pedestrian and maintain the required buffer distance of at least one meter. Is required by strict constraints.

  [0281] Other strict constraints can also be used. For example, in at least some cases, the maximum deceleration rate of the host vehicle may be used. This maximum deceleration rate can be determined based on a detected distance to a target vehicle that follows the host vehicle (e.g., using images collected from a rear-facing camera). Strict constraints may include a forced stop at a detected pedestrian crossing or level crossing or other applicable constraints.

  [0282] If analysis of a scene in the environment of the host vehicle indicates that one or more predefined navigation constraints may be involved, those constraints may be imposed on one or more planned navigation actions of the host vehicle. Can be imposed. For example, if the analysis of the scene results in the driving policy module 803 returning a desired navigation action, the desired navigation action can be tested against one or more of the constraints involved. If it is determined that the desired navigation action violates any aspect of the constraint involved (eg, when a predefined strict constraint requires that the host vehicle stay at least 1.0 meters from pedestrian 1215, the desired (Where the navigation operation carries the host vehicle within a distance of 0.7 meters of pedestrian 1215), at least one modification to the desired navigation operation may be made based on one or more predefined navigation constraints. Adjusting the desired navigation behavior in this manner can result in the actual navigation behavior of the host vehicle in compliance with constraints imposed by the particular scene detected in the environment of the host vehicle.

  [0283] After determining the actual navigation operation of the host vehicle, performing the navigation operation by causing at least one adjustment of the navigation actuator of the host vehicle in response to the determined actual navigation operation of the host vehicle. Can be. The navigation actuator may include at least one of a host vehicle steering mechanism, a brake, or an accelerator.

[0284] Prioritized constraints
[0285] As described above, various strict constraints can be used with the navigation system to ensure safe operation of the host vehicle. Constraints may include, among other things, the minimum safe driving distance for a pedestrian, target vehicle, road barrier, or detected object, the maximum speed of travel of the detected pedestrian when passing through the affected area, or the maximum rate of deceleration of the host vehicle. May be included. These constraints can be imposed by trained systems that are trained based on machine learning (supervised, enhanced, or a combination thereof), but may be subject to expected situations that occur within the scene of the host vehicle's environment. Untrained systems (using algorithms that deal directly) may also be useful.

  [0286] In any case, there may be a hierarchy of constraints. In other words, some navigation constraints may take precedence over other navigation constraints. Thus, if a situation arises in which a navigation operation that would satisfy all the constraints involved is not available, the navigation system can determine the available navigation operation that first fulfills the highest priority constraint. . For example, the system may cause the vehicle to first avoid a pedestrian, even if navigation to avoid the pedestrian would cause a collision with another vehicle or object detected on the road. In another example, the system may cause the vehicle to ride on a curb to avoid pedestrians.

  FIG. 13 shows a flowchart illustrating an algorithm for implementing a hierarchy of involved constraints determined based on an analysis of a scene in the environment of the host vehicle. For example, in step 1301, at least one processor associated with the navigation system (eg, an EyeQ processor, etc.) can receive a plurality of images representing the environment of the host vehicle from a camera mounted on the host vehicle. At step 1303, a navigation state associated with the host vehicle may be determined by analyzing one or more images representing a scene of the environment of the host vehicle. For example, the navigation state may be that the host vehicle is traveling along a two-lane road 1210 as shown in FIG. This may specifically indicate that a pedestrian 1215 is waiting to cross the road on which the host vehicle travels, and that an object 1219 is ahead of the host vehicle lane.

  [0288] At step 1305, one or more navigation constraints involved by the navigation state of the host vehicle may be determined. For example, at least one processing device may analyze a scene in the environment of the host vehicle represented by one or more captured images, and then be engaged by objects, vehicles, pedestrians, etc. recognized by image analysis of the captured images. One or more navigation constraints to be performed can be determined. In some embodiments, the at least one processing device can determine at least a first default navigation constraint and a second default navigation constraint that are involved by a navigation state, and the first default navigation constraint. May be different from the second default navigation constraint. For example, a first navigation constraint may relate to one or more target vehicles detected in the environment of the host vehicle, and a second navigation constraint may relate to a pedestrian detected in the environment of the host vehicle. Can be related to

  [0289] At step 1307, at least one processing device may determine a priority associated with the constraint identified at step 1305. In the described example, the second default navigation constraint relating to the pedestrian may have a higher priority than the first default navigation constraint relating to the target vehicle. Although the priority associated with the navigation constraint may be determined or assigned based on various factors, in some embodiments, the priority of the navigation constraint may be determined by its relative safety perspective. Can be related to importance. For example, in as many situations as possible, it may be important to follow or meet all implemented navigation constraints, but some constraints may be associated with stronger safety risks than others. , So you can assign a higher priority. For example, a navigation constraint that requires the host vehicle to be at least one meter away from the pedestrian may have a higher priority than a constraint that requires the host vehicle to be at least one meter away from the target vehicle. . This may be because a collision with a pedestrian may have more severe consequences than a collision with another vehicle. Similarly, maintaining a distance between the host vehicle and the target vehicle can mean that the host vehicle avoids boxes in the road, travels below a certain speed on deceleration bumps, or maximizes the occupants of the host vehicle. It may have a higher priority than constraints requiring exposure below the acceleration level.

  [0290] The driving policy module 803 is designed to maximize safety by satisfying the navigation constraints involved by a particular scene or navigation state, but satisfying all constraints involved by the situation. It may be physically impossible. In such a situation, as shown in step 1309, the priority of each constraint involved can be used to determine which of the constraints involved should be satisfied first. Continuing the above example, in a situation where it is not possible to satisfy both the constraint of the gap with the pedestrian and the constraint of the gap with the target vehicle and only one of the constraints can be met, the constraint of the gap with the pedestrian is A higher priority may result in the constraint being met before attempting to maintain a gap to the target vehicle. Thus, under normal circumstances, if both the first predefined navigation constraint and the second predefined navigation constraint can be met, as shown in step 1311, the at least one processing device identifies the host vehicle A first navigation action for the host vehicle that satisfies both the first predefined navigation constraint and the second predefined navigation constraint may be determined based on the navigation state. However, in other situations where not all of the constraints involved can be met, as shown in step 1313, if both the first and second default navigation constraints cannot be met, then at least One processing device satisfies a second default navigation constraint (ie, a higher priority constraint) based on the identified navigation state, but a first default navigation constraint (greater than the second navigation constraint). A second navigation action for a host vehicle that does not meet the lower priority.

  [0291] Next, at step 1315, the at least one processing device determines the first navigation operation or the determined first navigation operation for the host vehicle to perform the navigation operation for the determined host vehicle. An adjustment of at least one of the navigation actuators of the host vehicle can be caused in response to the second navigation operation. As in the previous example, the navigation actuator may include at least one of a steering mechanism, a brake, or an accelerator.

[0292] Relaxation of constraints
[0293] As discussed above, navigation constraints can be imposed for security. Constraints may include, among other things, the minimum safe driving distance for a pedestrian, target vehicle, road barrier, or detected object, the maximum speed of travel of the detected pedestrian when passing through the affected area, or the maximum rate of deceleration of the host vehicle. May be included. These constraints can be imposed in a learning navigation system or a non-learning navigation system. In certain situations, these constraints can be relaxed. For example, if the host vehicle slows or stops near a pedestrian and travels slowly to convey the intent to pass by the pedestrian, the response of the pedestrian can be detected from the acquired image. If the pedestrian's reaction is to stay still or stop moving (and / or if eye contact with the pedestrian is detected), the intention of the navigation system to pass by the pedestrian is Can be understood. In such a situation, the system relaxes one or more predefined constraints and enforces less stringent constraints (eg, within 0.5 meters of a pedestrian rather than within the more stringent one meter boundary). (Allowing the vehicle to navigate within range).

  FIG. 14 shows a flowchart for implementing control of a host vehicle based on relaxing one or more navigation constraints. In step 1401, at least one processing device may receive a plurality of images representing an environment of a host vehicle from a camera associated with the host vehicle. Analyzing the image at step 1403 may allow for identifying a navigation condition associated with the host vehicle. At step 1405, at least one processor may determine a navigation constraint associated with the navigation state of the host vehicle. The navigation constraints may include a first predefined navigation constraint that is involved by at least one aspect of the navigation state. In step 1407, analyzing the plurality of images may determine the presence of at least one navigation constraint mitigation factor.

  [0295] Navigation constraint mitigation factors may include any suitable indicator that one or more navigation constraints may be interrupted, changed, or otherwise relaxed in at least one aspect. In some embodiments, the at least one navigation constraint mitigation factor may include a determination (based on image analysis) that the pedestrian's eyes are looking in the direction of the host vehicle. In that case, it can be assumed that the pedestrian recognizes the host vehicle more safely. As a result, the reliability that the pedestrian is not involved in an unexpected operation that causes the pedestrian to move in the route of the host vehicle may be further increased. Other constraint relaxation factors can also be used. For example, the at least one navigation constraint mitigation factor is a pedestrian determined to be stationary (eg, a person estimated to be unlikely to enter the path of the host vehicle) or determined to be decelerating in movement. May be included. When the host vehicle stops, it is determined that the vehicle is not moving, and the navigation constraint mitigation factor such as a pedestrian who resumes moving after that can include a more complicated operation. In such situations, pedestrians may be considered to understand that the host vehicle has priority access, and pedestrians attempting to stop may indicate their intention to yield to the host vehicle. Can be. Other situations where one or more constraints may be relaxed are curb types (e.g., low curbs or steep slopes may allow for relaxed distance constraints), pedestrians or other objects on the sidewalk. Lacking, including a vehicle without its own engine running, and having a reduced distance, or in situations where the pedestrian is away from the area in which the host vehicle is traveling and / or Including situations where

  [0296] Having identified the existence of a navigation constraint mitigation factor (eg, at step 1407), a second navigation constraint can be determined or created in response to detecting the constraint mitigation factor. This second navigation constraint may be different from the first navigation constraint and may include at least one property that is relaxed as compared to the first navigation constraint. The second navigation constraint may include a newly created constraint based on the first constraint, wherein the newly created constraint includes at least one modification that relaxes the first constraint in at least one point. Including. Alternatively, the second constraint may constitute a predetermined constraint that is less strict than the first navigation constraint in at least one respect. In some embodiments, this second constraint may be reserved for use only in situations where constraint mitigation factors are identified in the environment of the host vehicle. Whether the second constraint is newly generated or is selected from a predetermined set of constraints that is fully or partially available, it is more stringent (which can be applied without the detection of an associated navigation constraint mitigation factor). Applying the second navigation constraint instead of the first navigation constraint may be referred to as constraint relaxation and may be implemented at step 1409.

  If at least one constraint mitigation factor is detected in step 1407 and at least one constraint is relaxed in step 1409, a navigation operation for the host vehicle can be determined in step 1411. A navigation operation for the host vehicle may be based on the identified navigation state and may satisfy a second navigation constraint. At step 1413, the navigation operation may be performed by causing at least one adjustment of a navigation actuator of the host vehicle in response to the determined navigation operation.

  [0298] As discussed above, using navigational constraints and relaxed navigational constraints may result in a navigation system that is trained (eg, by machine learning) or an untrained navigation system (eg, a particular navigation state). (A system programmed to respond with a predetermined operation in response to a predetermined operation). When using a trained navigation system, the availability of relaxed navigation constraints for a particular navigation situation may represent a mode switch from a trained system response to an untrained system response. For example, the trained navigation network can determine an original navigation action for the host vehicle based on the first navigation constraint. However, the operation performed by the vehicle may be different from the navigation operation satisfying the first navigation constraint. Rather, the action taken may satisfy a more relaxed second navigation constraint (eg, in response to detecting a particular condition in the host vehicle's environment, such as the presence of a navigation constraint mitigation factor). ) It can be an action determined by an untrained system.

  [0299] There are many examples of navigation constraints that can be relaxed in response to detecting constraint relaxation factors in the environment of the host vehicle. For example, if the predefined navigation constraint includes a buffer zone associated with the detected pedestrian, and at least a portion of the buffer zone extends a predetermined distance from the detected pedestrian, the (newly determined, predetermined Relaxed navigation constraints (recalled from memory from the set or generated as relaxed versions of existing constraints) may include different or modified buffer areas. For example, a different or modified buffer zone may have a shorter distance to the pedestrian than the original or unmodified buffer zone to the detected pedestrian. As a result, if an appropriate constraint mitigation factor is detected within the environment of the host vehicle, the host vehicle may be allowed to navigate closer to the detected pedestrian, taking into account the relaxed constraints.

  [0300] The relaxed characteristics of the navigation constraint may include a reduced width of the buffer area associated with at least one pedestrian as described above. However, the mitigated property may also include the reduced width of the buffer area associated with the target vehicle, the detected object, a roadside barrier, or any other object detected in the environment of the host vehicle.

  [0301] The at least one relaxed property may also include other types of modifications in the navigation constraint property. For example, the mitigated property may include a speed increase associated with at least one predefined navigation constraint. The mitigated property may also include an increase in the maximum allowable deceleration / acceleration associated with the at least one predefined navigation constraint.

  [0302] As described above, constraints can be relaxed in certain situations, but navigation constraints can be enhanced in other situations. For example, in some situations, the navigation system may determine that the conditions justify increasing the set of normal navigation constraints. Such augmentation may include adding new constraints to the set of predefined constraints or adjusting one or more aspects of the predefined constraints. This addition or adjustment may result in more discreet navigation for a given set of constraints applicable under normal driving conditions. Conditions that may justify increased constraints may include sensor failures or adverse environmental conditions (rain, snow, fog, or other conditions associated with reduced visibility or reduced vehicle traction).

  [0303] FIG. 15 shows a flowchart for implementing control of a host vehicle based on enhancing one or more navigation constraints. In step 1501, at least one processing device may receive a plurality of images representing an environment of a host vehicle from a camera associated with the host vehicle. Analyzing the image at step 1503 may enable identifying navigational conditions associated with the host vehicle. At step 1505, at least one processor may determine a navigation constraint associated with the navigation state of the host vehicle. The navigation constraints may include a first predefined navigation constraint that is involved by at least one aspect of the navigation state. In step 1507, analyzing the plurality of images may determine the presence of at least one navigation constraint enhancer.

  [0304] The navigation constraints involved may include any of the navigation constraints discussed above (eg, with respect to FIG. 12) or any other suitable navigation constraints. Navigation constraint enhancement factors may include any indicator that one or more navigation constraints may be supplemented / enhanced in at least one aspect. The supplementing or augmenting of the navigation constraints can be done on a set-by-set basis (eg, by adding a new navigation constraint to a given set of constraints) or can be done on a constraint-by-constraint basis (eg, modified constraints). Modifying a particular constraint such that is more restrictive than the original, or adding a new constraint corresponding to the predetermined constraint, wherein the new constraint is at least in one aspect more than the corresponding constraint. Is restrictive). Additionally or alternatively, supplementing or augmenting navigation constraints may refer to selecting from a predefined set of constraints based on hierarchy. For example, an enhanced set of constraints may be provided for selection based on whether a navigation enhancement factor is detected within or for the host vehicle. Under normal conditions where no enhancement factor is detected, the navigation constraints involved can be derived from the constraints applicable to normal conditions. On the other hand, if one or more constraint enhancement factors are detected, the constraints involved can be generated for the one or more enhancement factors or can be derived from predetermined enhanced constraints. . An enhanced constraint may be more restrictive in at least one aspect than a corresponding constraint that is applicable under normal conditions.

  [0305] In some embodiments, the at least one navigation constraint augmenter is detecting (eg, based on image analysis) the presence of ice, snow, or water on a road surface in the environment of the host vehicle. May be included. This determination may include, for example, areas of higher reflectivity than expected on dry roads (eg, indicating ice or water on the road), white areas on the road indicating snow, and vertical Shadows on the road consistent with the presence of grooves (eg, tire marks in snow), water droplets or small particles of ice / snow on the windshield of the host vehicle, or water or ice / snow on the road surface It may be based on detecting any other suitable indicator.

  [0306] The at least one navigation constraint augmenting factor may also include detecting small particles on an exterior surface of the host vehicle windshield. Such small grains can impair the image quality of one or more image capture devices associated with the host vehicle. Although described with respect to the windshield of the host vehicle as being associated with a camera mounted behind the windshield of the host vehicle, other surfaces associated with the host vehicle (eg, a camera lens or lens cover, a headlight lens) Detecting small particles on the rear window, taillight lens, or any other surface of the host vehicle that is visible (or detected by a sensor) to the image capture device also indicates that there are navigation constraint enhancement factors. obtain.

  [0307] The navigation constraint enhancement factor can also be detected as a characteristic of one or more image acquisition devices. For example, a detected decrease in image quality of one or more images captured by an image capture device (eg, a camera) associated with the host vehicle may also constitute a navigation constraint enhancement factor. The loss of image quality may be associated with a hardware failure or partial hardware failure associated with the image capture device or an assembly associated with the image capture device. Such degradation in image quality may be caused by environmental conditions. For example, the presence of smoke, fog, rain, snow, and the like in the air surrounding the host vehicle may contribute to a reduction in image quality for roads, pedestrians, target vehicles, and the like that may be in the environment of the host vehicle.

  [0308] Navigation constraint enhancement factors may also relate to other aspects of the host vehicle. For example, in some situations, navigation constraint enhancement factors may include a detected or partial failure of a system or sensor associated with the host vehicle. Such augmenting factors may, for example, affect the host vehicle's ability to navigate to navigation constraints associated with the host vehicle's navigation state, such as speed sensors, GPS receivers, accelerometers, cameras, associated with the host vehicle. It may include detecting a failure or partial failure of a radar, rider, brake, tire, or any other system.

  [0309] If it is determined that a navigation constraint enhancement factor is present (eg, at step 1507), a second navigation constraint may be determined or created in response to detecting the constraint enhancement factor. This second navigation constraint may be different from the first navigation constraint and may include at least one property that is enhanced with respect to the first navigation constraint. Detecting a constraint enhancement factor within or associated with the host vehicle's environment indicates that at least one navigation capability of the host vehicle may be reduced compared to normal operating conditions. To obtain, the second navigation constraint may be more restrictive than the first navigation constraint. Such loss of performance may include reduced road traction (e.g., ice, snow or water on the road, reduced tire pressure, etc.), impaired visibility (e.g., rain, snow, Dust, smoke, fog, etc.), impaired detection capabilities (eg, sensor failure or partial failure, sensor performance degradation, etc.) or the ability of the host vehicle to navigate in response to detected navigational conditions. Any other reduction may be included.

  [0310] If at least one constraint enhancement factor is detected in step 1507 and if at least one constraint is enhanced in step 1509, a navigation operation for the host vehicle can be determined in step 1511. Navigation actions for the host vehicle may be based on the identified navigation state and may satisfy a second navigation (ie, enhanced) constraint. In step 1513, the navigation operation may be performed by causing at least one adjustment of a navigation actuator of the host vehicle in response to the determined navigation operation.

  [0311] As discussed above, using navigation constraints and augmented navigation constraints may result in a trained navigation system (e.g., by machine learning) or an untrained navigation system (e.g., to a particular navigation state). (A system programmed to respond with a predetermined action accordingly). When using a trained navigation system, the availability of increased navigation constraints for a particular navigation situation may represent a mode switch from a trained system response to an untrained system response. For example, the trained navigation network can determine an original navigation action for the host vehicle based on the first navigation constraint. However, the operation performed by the vehicle may be different from the navigation operation satisfying the first navigation constraint. Rather, the action taken may satisfy the augmented second navigation constraint (eg, in response to detecting a particular condition in the host vehicle's environment, such as the presence of a navigation constraint augmenter). It may be an action determined by an untrained system.

  [0312] There are many examples of navigation constraints that can be generated, supplemented, or enhanced in response to detecting a constraint enhancement factor in the environment of the host vehicle. For example, if the predefined navigation constraint includes a buffer zone associated with a detected pedestrian, object, vehicle, etc., and at least a portion of the buffer zone extends a predetermined distance from the detected pedestrian / object / vehicle, An enhanced navigation constraint (newly determined, recalled from memory from a predetermined set, or generated as an enhanced version of an existing constraint) may include a different or modified buffer area. For example, a different or modified buffer area may have a longer distance to the pedestrian / object / vehicle than the original or unmodified buffer area to the detected pedestrian / object / vehicle. As a result, if an appropriate constraint enhancement factor is detected in or in relation to the host vehicle, the enhanced constraints are taken into account and the detected pedestrian / object / vehicle is navigated further. The host vehicle may be forced to do so.

  [0313] The at least one enhanced property may also include other types of modifications in the navigation constraint property. For example, the enhanced property may include a slowdown associated with at least one predefined navigation constraint. The enhanced characteristics may also include a decrease in the maximum allowable deceleration / acceleration associated with at least one predefined navigation constraint.

[0314] Navigation based on long-term plan
[0315] In some embodiments, the disclosed navigation system can not only respond to detected navigational conditions in the environment of the host vehicle, but also determine one or more navigational actions based on a long-term plan. it can. For example, the system may consider the potential impact of one or more navigation actions available as options for navigating with respect to the detected navigation state on future navigation states. Examining the effect of available actions on future states may allow the navigation system to determine navigation actions based not only on the currently detected navigation state, but also on a long-term plan. Navigation using long-term planning techniques may be particularly applicable where one or more reward functions are used by the navigation system as a technique for selecting a navigation action from the available options. Potential rewards can be analyzed for available navigation actions that can be taken in response to the detected current navigation state of the host vehicle. However, further, potential rewards can be analyzed in relation to actions that can be taken in response to future navigation states that are expected to result from available actions for the current navigation state. As a result, even though the selected navigation action may not result in the highest reward of the available actions that can be performed depending on the current navigation state, the disclosed navigation system is partially In the case (1), the navigation operation may be selected according to the detected navigation state. This may, among other things, trigger one or more potential navigation actions that will reward the selected action or, in some cases, higher than any of the actions available for the current navigation state. May be applicable if the system determines that the selected action can result in the navigation state of. This principle can be more simply expressed as performing less favorable operations now, as rewards will result in higher options in the future. Thus, the disclosed navigation system capable of long-term planning can select a sub-optimal short-term action if the long-term prediction indicates that a short-term loss of reward can result in an increase in long-term reward.

  [0316] Generally, autonomous driving applications may include a series of planning problems from which the navigation system can determine immediate action to optimize long-term objectives. For example, if a vehicle encounters a situation of merging at a roundabout, the navigation system may determine an immediate acceleration or braking command to begin navigation into the roundabout. The immediate action for the navigational condition detected at the roundabout may include an acceleration command or a brake command depending on the detected condition, but the long-term purpose is to successfully merge and the long-term The net effect is the success / failure of the merge. The planning problem can be addressed by decomposing the problem into two phases. First, supervised learning can be applied to predict the near future based on the present (assuming the predictors are differentiable for the current representation). Second, the complete trajectory of the agent can be modeled using a regression neural network, and unexplained factors are modeled as (additional) input nodes. This may allow using supervised learning methods and direct optimization on regression neural networks to find solutions to long-term programming problems. Such an approach may also allow for robust policy learning by incorporating hostile elements to the environment.

  [0317] Two of the most fundamental elements of an autonomous driving system are sensing and planning. Sensing addresses finding a compact representation of the current state of the environment, while planning addresses determining which actions to take in order to optimize future objectives. Supervised machine learning is useful for solving detection problems. A machine learning algorithm framework, especially a reinforcement learning (RL) framework as described above, can also be used for the planning part.

[0318] RL may be performed by a series of consecutive rounds. In Round t, planner (agent or operating policy module 803 as an alias) may observe the status s _t ∈S representing the agent and the environment. The planner should then determine the action at _t A. After performing the action, the agent receives an immediate reward

And is moved to a new state _st + 1. As an example, the host vehicle may include an adaptive cruise control (ACC) system in which the vehicle autonomously accelerates / brakes to maintain a smooth drive while maintaining a sufficient distance to the preceding vehicle. Should be implemented. The state is

Where _xt is the distance to the preceding vehicle and _vt is the speed of the host vehicle relative to the speed of the preceding vehicle. motion

It is (host vehicle is decelerated in a case where a t _<0 is established) an acceleration command. Reward, (reflecting the smoothness of the operation) | can be a function that depends on and (host vehicle reflects to keep a safe distance from the preceding vehicle) s _{t |} a _t. The planner's goal is to maximize the cumulative reward (possibly up to a discounted period of time or future rewards). To do so, the planner can utilize a policy π: S → A that maps states to actions.

[0319] supervised learning (SL) can be regarded as a special case of RL, in the supervised learning is _{s t} is a sampling from the distribution S, the reward _{function, r t = -l (a t} , y t ) format can have a, l is the loss function, learning side, the value of y _t is noisy) value (in the case of optimal operation to perform when the acknowledge state s _t Observe. There may be some differences between the general RL model and the particular case of SL, and those differences can make the general RL problem more difficult.

In some SL situations, the operation (or prediction) performed by the learning side may not have an effect on the environment. In other _words, it is independent and _{s t + 1} and _{a t.} This can have two important implications. First, in SL, samples (s ₁ , y ₁ ),. . . , (S m, _{y m)} can be collected in advance, by doing so, it is possible to start the search for the policy (or predictor) with excellent precision for the first time sample. In contrast, at the RL, the state _{st + 1} typically depends on the action taken (or even the previous state), and the action taken therefore depends on the policy used to generate the action. This ties the data generation process to the policy learning process. Second, since the work with SL does not affect the environment, the contribution of the selection of a _t on the performance of π is local. Specifically, a _t only affects the value of the immediate compensation. In contrast, in RL, the actions performed in round t may have a long-term effect on reward values in future rounds.

[0321] In SL, the shape of the reward _{r t = -l (a t,} y t) is the knowledge of along with the "right" solution _{y t,} to provide a complete knowledge of the rewards for all choices may be of _{a t} it can be, it may allow the calculation of the derivative of the reward on a _t. In contrast, at the RL, the "one-shot" value of the reward may be all that can be observed for a particular choice of action taken. This can be referred to as "bandit" feedback. In RL-based systems, if only "bandit" feedback is available, the system may not always know if the action taken was the best action to take, so this is a long term This is one of the most important reasons why "search" is needed as part of a successful navigation plan.

[0322] Many RL algorithms rely at least in part on a mathematically explicit model of the Markov Decision Process (MDP). Markov assumption is that the distribution of s _{t + 1} is determined completely s _t and a _t as given. This results in a closed form of the cumulative reward for a given policy with respect to a stationary distribution over the states of the MDP. The stationary distribution of the policy can be represented as a solution to a linear programming problem. This 1) can be referred to as a policy search optimization for the main problem, and 2) results in two algorithms groups that optimization for the dual problem that the variable is called value function V ^[pi. The value function determines the expected cumulative reward when the MDP starts in the initial state s, from which the action is selected according to π. The quantities involved are the state-action value function ^Qπ (s, a), which assumes a start from state s, an action a that is immediately selected, and an action selected therefrom according to π. Determine the cumulative reward. This Q function can cause characterization of the optimal policy (using the Bellman equation). Specifically, this Q function can indicate that the optimal policy is a deterministic function from S to A (in fact, the optimal policy is the "greedy" policy for the optimal Q function). Can be characterized as).

[0323] One potential advantage of the MDP model is that it allows the future to be combined with the present using a Q-function. For example, given that the host vehicle is currently in state s, the value of Q ^π (s, a) may indicate the effect of performing operation a for the future. Thus, the Q function provides a local measure of the quality of action a, which can make the RL problem similar to the SL scenario.

[0324] Many RL algorithms approximate the V or Q function in some way. Value iteration algorithms, such as the Q-learning algorithm, can take advantage of the fact that the V and Q functions of the optimal policy can be fixed points of some operator obtained from the Bellman equation. The actor-critic policy iterative algorithm aims to learn the policy in an iterative way, and at iteration t, "critic"

, And based on this estimate, “actor” improves the policy.

  [0325] Despite the mathematical simplicity of MDP and the convenience of switching to the Q-function representation, this approach may have some limitations. For example, in some cases, the concept of an approximation of the behavioral state of Markov may be all that can be found. Further, state transitions may depend not only on the actions of the agent, but also on the actions of other players in the environment. For example, in the ACC example above, the dynamics of an autonomous vehicle may be Markov-type, but the next state may depend on the behavior of the driver of another vehicle that is not necessarily Markov-type. One possible solution to this problem is to use a partially observed MDP, where the partially observed MDP has a Markov state but can be seen according to the hidden state It is assumed to be a distributed observation.

  [0326] A more direct approach may consider game-theoretic generalization of MDP (eg, a stochastic game framework). In fact, the algorithms for MDP can be generalized to multi-agent games (eg, minimax Q learning or Nash Q learning). Other approaches may include explicit modeling of other players and vanishing regret learning algorithms. Learning in a multi-agent configuration can be more complicated than learning in a single-agent configuration.

  [0327] A second limitation of the Q-function representation may arise from departures from tabular settings. Tabular settings are where the number of states and operations is small, so that Q can be represented as a table of | S | rows and | A | columns. However, if the natural representation of S and A includes a Euclidean space and the state space and the operation space are discretized, the number of states / actions can be exponential in terms of magnitude. In such a case, it may not be practical to adopt a table format setting. Alternatively, the Q function may be approximated by some function from a parametric hypothesis class (eg, a particular architecture neural network). For example, a deep Q network (DQN) learning algorithm can be used. In DQN, the state space may be continuous, but the operating space may remain a small, discrete set. Techniques for accommodating a continuous motion space are possible, but they may depend on approximating the Q function. In any case, the Q function can be complex and sensitive to noise, and thus can be difficult to learn.

  [0328] A different approach may be to address the RL problem using a recurrent neural network (RNN). In some cases, RNNs can be combined with robustness to hostile environments from multi-agent game concepts and game theory. Furthermore, this approach may not explicitly rely on Markov assumptions.

[0329] Hereinafter, a method for navigation based on a plan based on prediction will be described in more detail. In this method, the state space S is

And the state space A is

Can be assumed to be a subset of This can be a natural expression in many applications. As noted above, there may be two main differences between RL and SL, the differences being: (1) Past behavior affects future rewards, so future The fact that information may need to be retransmitted in the past, and (2) the "bandit" nature of rewards can obscure the dependency between (state, action) and rewards, which It can complicate the process.

[0330] As a first step in this approach, it can be observed that there is an interesting issue where the bandit nature of rewards is not important. For example, reward values for ACC applications (discussed in more detail below) may be differentiable with respect to current state and operation. In fact, even if the reward is given in a "bandit" style,

Differentiable function such that

Can be a relatively simple SL problem (eg, a one-dimensional regression problem). Therefore, the first step in this approach is to make a differentiable function for s and a

Or the instance vector is

And a differentiable function that minimizes at least some regression loss over the sample with the target scalar being r _t

Using a regression learning algorithm to learn. In some situations, elements of the search can be used to create a training set.

[0331] Similar concepts can be used to address the connection between the past and the future. For example,

Function such that

Suppose that can be learned. Learning such a function can be characterized as an SL problem.

Can be considered as a predictor for the near future. Next, the policy that maps from S to A can be described using the parametric function π _θ : S → A. Representing π _θ as a neural network can enable the representation of episodes of running agents T rounds using a recurrent neural network (RNN), and the next state is

Is defined as here,

Can be defined by the environment and can represent unpredictable aspects in the near future. the s t _{+ 1} depends in differentiable manner to s _t and a _t may allow connection between the past operation and future compensation values. The parameter vector of the policy function π _θ can be learned by backpropagation on the resulting RNN. It should be noted that there is no need to impose a clear probabilistic assumptions v _t. Specifically, there is no need for Markov-related requirements. Alternatively, a regression network may be used to propagate "sufficient" information between the past and the future. Intuitively,

May describe the foreseeable part of the near future, while v _t may represent unpredictable aspects that may be caused by the behavior of other players in the environment. The learning system should learn a policy that is robust to the behavior of other players. If || v _t || is large, the connection between past actions and future reward values may be too noisy to learn meaningful policies. Clearly expressing the dynamics of the system in a transparent manner can allow past knowledge to be more easily incorporated. For example, past knowledge

Can be simplified.

[0332] As discussed above, the learning system can benefit from robustness to hostile environments, such as the environment of a host vehicle, which can include other drivers that can operate in unexpected ways. v In the model does not impose stochastic assumed for _t, v _t can be considered the environment selected by hostile way. In some cases, it is possible to restrict the μ _t, otherwise the enemy is difficult to plan a problem, or impossible to be able to even. One natural constraint may be to require || μ _t || to be bounded by the constraint.

[0333] Robustness to hostile environments can be useful in autonomous driving applications. Choosing μ _{t in a} hostile manner can even speed up the learning process, as it can concentrate the learning system towards a robust optimal policy. A simple game can be used to illustrate this concept. The state is

And the operation is

, And the loss function of _{immediate, 0.1 | a t | + [} | s t | -2] a _{+, [x] + = max} {x, 0} is, ReLU (normalized linear units) Function It is. The next state _is a _{s t + 1 = s t +} a t + v t, v t ∈ [-0.5,0.5] is selected for the environment hostile way. Here, it is possible to describe the optimal policy as a network of two layers with _{ReLU: a t = - [s} t -1.5] + + [- s t -1.5] +. It should be appreciated that when | s _t | ∈ (1.5,2], the optimal operation may have an immediate loss greater than operation a = 0, so the system plans for the future. it can be, derivative of loss for good .a _t without depending only on the loss of immediate is 0.1sign _{(a t),} differentiated for _{s t is, 1 [| s t |>} 2] sign in the context of should be recognized that a _{(s t) .s t ∈ (} 1.5,2], hostile selection of _{v t is} to set v t = 0.5, therefore _{a t} > 1.5-s _t whenever the can has a non-zero loss on round t + 1. in such cases, the derivative of the loss can be reversed directly propagated to a _t. Therefore hostile selection of v _t is, a resulting navigation system nonzero backpropagation message if the selection of _t is sub-optimal Such a relationship may lead to the expectation that the current action will provide a higher optimal reward in the future (even if the action results in sub-optimal rewards or even losses). , The navigation system may facilitate selecting the current action.

[0334] Such an approach can be applied to virtually all possible navigation situations. Hereinafter, one example, that is, a method applied to adaptive cruise control (ACC) will be described. In the ACC problem, the host vehicle may attempt to maintain a sufficient distance to the target vehicle ahead (eg, 1.5 seconds to the target vehicle). Another goal may be to drive as smoothly as possible while maintaining the desired gap. A model representing this situation can be defined as follows. The state space is

And the operating space is

It is. The first coordinate of the state is the speed of the target vehicle, the second coordinate is the speed of the host vehicle, and the last coordinate is the distance between the host vehicle and the target vehicle (e.g., Along with the target vehicle position minus the target position). Operation to be performed by the host vehicle is accelerated, it can be represented by a _t. The quantity τ can indicate the time difference between successive rounds. τ can be set to any suitable amount, but in one example, τ can be 0.1 seconds. The position _st is

And the (unknown) acceleration of the target vehicle

Can be indicated by

[0335] The complete dynamics of the system can be described by the following equation:

[0336] This can be described as the sum of two vectors.

[0337] The first vector is a predictable part, and the second vector is an unpredictable part. The reward for the round t is determined as follows.

[0338] The first term may result in a penalty for non-zero acceleration, thus promoting smooth driving. The second term is the desired distance between the distance to the target car x _t

And this is defined as the maximum between a distance of 1 meter and a braking distance of 1.5 seconds. In some cases, this ratio can be exactly one, but as long as this ratio is in the range of [0.7, 1.3], the policy can do without any penalty, Some slack, a property that may be important in realizing proper driving, can be noticed to the host vehicle in navigation.

  [0339] Implementing the techniques outlined above, the navigation system of the host vehicle may select an action according to the observed state (eg, by the action of the driving policy module 803 in the processing unit 110 of the navigation system). Can be. The actions selected are not based on an analysis of only the rewards associated with available response actions with respect to the perceived navigation state, but rather with respect to future states, potential actions in response to future states, and potential actions It can also be based on reviewing and analyzing rewards to be paid.

  FIG. 16 shows an algorithm method for navigation based on detection and a long-term plan. For example, in step 1601, at least one processing device 110 of a navigation system for a host vehicle may receive a plurality of images. These images can capture a scene representing the environment of the host vehicle and can be provided by any of the image capture devices described above (eg, cameras, sensors, etc.). Analyzing one or more of these images at step 1603 may allow at least one processing device 110 to identify a current navigation state associated with the host vehicle (as described above).

  In steps 1605, 1607, and 1609, various potential navigation actions can be determined according to the detected navigation state. (E.g., to complete the merge, to smoothly follow the preceding vehicle, to overtake the target vehicle, to avoid objects on the road, to slow down due to detected stop signs, to interrupt These potential navigation actions (e.g., available from the first navigation action) to avoid an upcoming target vehicle or to complete any other navigation action that may help the system's navigation goals. ) Can be determined based on the sensing state and the long-term goal of the navigation system.

  [0342] For each potential navigation action determined, the system can determine an expected reward. Expected rewards may be determined according to any of the techniques described above and may include an analysis of certain potential behaviors for one or more reward functions. Expected rewards 1606, 1608, and 1610 may be determined for each of the potential navigation actions determined at steps 1605, 1607, and 1609, respectively (eg, first, second, and Nth). .

  [0343] In some cases, the host vehicle's navigation system may determine the potential action available based on the value associated with the expected reward 1606, 1608, and 1610 (or any other type of indicator of the expected reward). You can choose from. For example, in some situations, the action that results in the highest expected reward may be selected.

  [0344] In particular, in other cases where the navigation system is involved in long-term planning to determine the navigation behavior for the host vehicle, the system may not select the potential behavior that results in the highest expected reward. Rather, the system may look into the future and analyze whether there may be an opportunity to later realize a higher reward if a low-reward action is selected depending on the current navigation state. For example, a future state can be determined for any or all of the potential actions determined in steps 1605, 1607, and 1609. Each future state determined in steps 1613, 1615, and 1617 becomes the current navigation state modified by each potential action (eg, the potential actions determined in steps 1605, 1607, and 1609). It may represent a future navigation state that is expected to occur based on it.

  [0345] For each of the future states predicted in steps 1613, 1615, and 1617, determine one or more future actions (as available navigation options depending on the determined future state). And can be evaluated. In steps 1619, 1621, and 1623, for example, an expected reward value or any other type of indicator associated with one or more of the future actions is created (eg, based on one or more reward functions). can do. The expected reward associated with one or more future actions is assessed by comparing the value of a reward function associated with each future action or by comparing any other indicator associated with the expected reward. can do.

  [0346] In step 1625, the navigation system for the host vehicle may determine the current navigation state (e.g., in steps 1605, 1607, and 1609) and not based solely on the potential actions identified (e.g., Based on the expected reward, which is also determined as a result of the potential future action available depending on the predicted future state (determined in steps 1613, 1615 and 1617). To select a navigation action for the host vehicle. The selection at step 1625 may be based on an analysis of choices and rewards performed at steps 1619, 1621, and 1623.

  [0347] The selection of a navigation action at step 1625 may be based solely on comparing expected rewards associated with future action options. In this case, the navigation system may select an action for the current state based solely on comparing expected rewards resulting from the action for a potential future navigation state. For example, the system can select a potential action identified in step 1605, 1607, or 1609 that is associated with the highest future reward value determined by the analysis in steps 1619, 1621, and 1623. .

  [0348] The selection of a navigation action at step 1625 may be based solely on comparing current action options (as described above). In this situation, the navigation system may select the potential action identified in step 1605, 1607, or 1609 that is associated with the highest expected reward 1606, 1608, or 1610. This selection can be made with little or no consideration of future expected rewards for available navigation actions depending on future navigation conditions or expected future navigation conditions.

  [0349] On the other hand, in some cases, selecting a navigation action in step 1625 may be based on comparing expected rewards associated with both future action options and current action options. This may in fact be one of the principles of navigation based on long-term planning. Analyze the expected reward for future actions to achieve a potentially higher reward for subsequent navigation actions that are expected to be available depending on future navigation states, and It can be determined whether any can justify selecting an operation with a lower reward accordingly. As an example, the value of expected reward 1606 or other indicator may indicate the highest expected reward among rewards 1606, 1608, and 1610. On the other hand, expected reward 1608 may indicate the lowest expected reward among rewards 1606, 1608, and 1610. Rather than simply selecting the potential action determined in step 1605 (ie, the action that produces the highest expected reward 1606), the future state, potential future Analysis of behavior and future rewards can be used. In one example, the reward identified in step 1621 (in response to at least one future action for the future state determined in step 1615 based on the second potential action determined in step 1607) is the expected reward 1606 It may be determined that there is a higher possibility. Based on this comparison, select the second potential action determined in step 1607 instead of the first potential action determined in step 1605, even though expected reward 1606 is higher than expected reward 1608. be able to. In one example, the potential navigation action determined in step 1605 can include merging before the detected target vehicle, while the potential navigation action determined in step 1607 can include This may include joining. The expected reward 1606 to merge before the target vehicle may be higher than the expected reward 1608 associated with merging behind the target vehicle, but merging behind the target vehicle may be higher than the expected reward 1606, 1608. Or it is determined that an action option that provides a higher potential reward than other rewards based on actions available in response to the current detected navigation state may result in a possible future state. There is.

  [0350] Making a selection among the potential actions in step 1625 may include selecting any of the expected rewards (or any other metric or indicator of a benefit associated with one potential action over another potential action). Based on an appropriate comparison of In some cases, as described above, it is expected that the second potential action will result in at least one future action associated with the expected reward higher than the reward associated with the first potential action. In that case, the second potential operation can be selected over the first potential operation. In other cases, more complex comparisons can be used. For example, a reward associated with a choice of action in response to a predicted future state can be compared to a plurality of expected rewards associated with a potential action determined.

  [0351] In some scenarios, at least one of the future actions provides a higher reward than any of the expected rewards (eg, expected rewards 1606, 1608, 1610, etc.) as a result of the potential action for the current state. If expected, actions and expected rewards based on predicted future states may affect the selection of potential actions for the current state. In some cases, as a guide to select potential actions for the current navigation state (eg, expected rewards associated with potential actions for the current state being detected as well as potential rewards for potential future navigation states). Future action options that yield the highest expected reward (of the expected rewards associated with the most future action options) can be used. That is, after identifying a future action option that yields the highest expected reward (or a reward that exceeds a predetermined threshold, etc.), the potential for future states associated with the identified future action that yields the highest expected reward An action can be selected at step 1625.

  [0352] In other cases, a selection of available actions can be made based on the determined difference between the expected rewards. For example, if the difference between the expected reward 1606 and the expected reward 1606 related to the future operation determined in Step 1621 exceeds the difference between the expected reward 1608 and the expected reward 1606 (assuming a difference of + sign), Step 1607 May be selected. In another example, the difference between the expected reward associated with the future action determined in step 1621 and the expected reward associated with the future action determined in step 1619 is the difference between the expected reward 1608 and the expected reward 1606. If so, a second potential action determined in step 1607 can be selected.

  [0353] Several examples have been described for making a selection among potential actions for the current navigation state. However, any other suitable comparison technique or criterion can be used to select actions available by long-term planning based on an analysis of actions and rewards over the expected future state. In addition, FIG. 16 shows two layers in the long-term plan analysis (eg, a first layer that considers the rewards arising from potential actions for the current state, and options for future actions depending on the predicted future state). (A second tier that considers the rewards arising from), but analyzes based on more tiers may also be possible. For example, rather than base long-term planning analysis on one or two layers, when choosing among the potential actions available depending on the current navigational state, three layers of analysis, four layers Or more layers can be used.

  [0354] After making a selection among the potential actions in response to the detected navigation state, in step 1627, at least one processor communicates with at least one of the navigation actuators of the host vehicle in response to the selected potential navigation action. One adjustment can be triggered. The navigation actuator may include any suitable device for controlling at least one aspect of the host vehicle. For example, the navigation actuator may include at least one of a steering mechanism, a brake, or an accelerator.

[0355] Navigation based on estimated aggression of others
[0356] The target vehicle can be monitored by analyzing the acquired image stream to determine an indicator of driving aggression. Although aggression is described herein as a qualitative or quantitative parameter, it does not use other characteristics, ie, perceived level of attention (potential driver impairment, distraction-cell phone, dozing, etc.). obtain. In some cases, the target vehicle may be considered to have a self-defense posture, and in some cases, the target vehicle may be considered to have a more aggressive posture. A navigation action can be selected or determined based on the aggression indicator. For example, in some cases, the relative speed, relative acceleration, increase in relative acceleration, follow-up distance, etc. with respect to the host vehicle can be tracked to determine whether the target vehicle is aggressive or self-defense. . If the target vehicle is determined to have a level of attack that exceeds the threshold, for example, the host vehicle may lean to yield to the target vehicle. Determining the level of attack of the target vehicle based on the determined behavior of the target vehicle with respect to one or more obstacles in the route or near the target vehicle (e.g., preceding vehicles, roadside obstacles, traffic lights, etc.) You can also.

[0357] As an introduction to this concept, an example of an experiment relating to the merging of a host vehicle in a roundabout is described, where the navigation goal is to exit through a roundabout. This situation may begin with the host vehicle reaching the entrance of the roundabout and may end with reaching the exit of the roundabout (eg, a second exit). Success is measured based on whether the host vehicle always keeps a safe distance from all other vehicles, whether the host vehicle exits the route as quickly as possible, and whether the host vehicle follows a policy of smooth acceleration can do. In this discussion, _NT target vehicles may be randomly placed on the roundabout. To model confusion between hostile and typical behaviors, the target vehicle can be modeled with an "aggressive" driving policy with probability p, so that the host vehicle tries to join before the target vehicle When attempting, the offensive target vehicle accelerates. With a probability 1-p, the target vehicle can be modeled by a "self-defense" driving policy, so that the target vehicle slows down and joins the host vehicle. In this experiment, p = 0.5, and information regarding other driver types may not be provided to the navigation system of the host vehicle. Other driver types can be randomly selected at the beginning of the episode.

  [0358] The navigation state can be expressed as the speed and position of the host vehicle (agent) and the position, speed and acceleration of the target vehicle. To distinguish between aggressive and self-defending drivers based on their current state, it may be important to keep track of the target acceleration. All target vehicles can move on a one-dimensional curve that outlines the path of the roundabout. The host vehicle can move on its own one-dimensional curve that intersects the curve of the target vehicle at the junction, which is the origin of both curves. In order to model reasonable driving, the absolute value of the acceleration of all vehicles can be capped by a constant. Driving in the opposite direction is not allowed, so the speed can also pass through the ReLU. Note that by not allowing driving in the opposite direction, agents cannot repent of their past actions and may require long-term planning.

As described above, the next state _{st + 1} is a predictable part

When, it can be decomposed into the sum of the unpredictable part v _t. Expression

, While may represent the position and velocity of the dynamics of the (differentiable manner can be clearly defined in the) vehicle, v _t may represent an acceleration of the target vehicle.

It can be demonstrated and can be represented as a combination of ReLU function on affine transformation, and thus is differentiable with respect to s _t and a _t. Vector v _t may implement obtained determined by the simulator in a way that can not be differentiated, aggressive behavior and self-defense behavior of other targets in the part of the target. Two frames from such a simulator are shown in FIGS. 17A and 17B. In this experimental example, it was learned that the host vehicle 1701 decelerated when reaching the entrance of the roundabout. The host vehicle 1701 may yield to an aggressive vehicle (eg, vehicle 1703 and vehicle 1705) and proceed safely when merging in front of self-defending vehicles (eg, vehicles 1706, 1708, and 1710). I learned. In the example shown in FIGS. 17A and 17B, the type of the target vehicle is not given to the navigation system of the host vehicle 1701. Rather, whether a particular vehicle is determined to be aggressive or self-defense is determined, for example, by estimation based on the observed position and acceleration of the target vehicle. In FIG. 17A, based on the position, speed, and / or relative acceleration, host vehicle 1701 can determine that vehicle 1703 has an aggressive tendency, and thus host vehicle 1701 is positioned ahead of target vehicle 1703. Instead of trying to merge, the vehicle can stop and wait for the target vehicle 1703 to pass. However, in FIG. 17B, the target vehicle 1710 moving behind the vehicle 1703 shows a self-defense tendency (again based on the observed position, speed, and / or relative acceleration of the vehicle 1710). The vehicle 1701 has recognized and thus has completed a successful merge before the target vehicle 1710 and behind the target vehicle 1703.

  FIG. 18 is a flowchart illustrating an example of an algorithm for navigating the host vehicle based on the predicted aggression of another vehicle. In the example of FIG. 18, a level of aggression associated with at least one target vehicle can be estimated based on the observed behavior of the target vehicle with respect to objects in the environment of the target vehicle. For example, in step 1801, at least one processing device (eg, processing device 110) of the host vehicle navigation system can receive a plurality of images representing the environment of the host vehicle from a camera associated with the host vehicle. At step 1803, analyzing one or more of the received images may enable at least one processor to identify a target vehicle (eg, vehicle 1703) within the environment of host vehicle 1701. In step 1805, analyzing one or more of the received images may enable at least one processing device to identify at least one obstacle for the target vehicle in the environment of the host vehicle. The object may be debris on a road, stop signal / signal, pedestrian, another vehicle (eg, a vehicle moving in front of a target vehicle or a parked vehicle), a box on a road, a road barrier, a curve, or a host. It may include any other type of object that may be encountered in a vehicle environment. In step 1807, analyzing one or more of the received images may include at least one processing device determining at least one navigation characteristic of the target vehicle with respect to at least one identified obstacle for the target vehicle. May be possible.

  [0361] Various navigation characteristics may be used to estimate the detected level of aggression of the target vehicle to determine an appropriate navigation response to the target vehicle. For example, such navigation characteristics include relative acceleration between the target vehicle and at least one identified obstacle, distance of the target vehicle from the obstacle (eg, following distance of the target vehicle behind another vehicle). And / or the relative speed between the target vehicle and the obstacle, and the like.

  [0362] In some embodiments, the navigation characteristics of the target vehicle can be determined based on outputs from sensors (eg, radar, speed sensors, GPS, etc.) associated with the host vehicle. However, in some cases, navigation characteristics of the target vehicle may be determined based in part or entirely on analyzing images of the environment of the host vehicle. Recognizing a target vehicle in the environment of a host vehicle using, for example, the image analysis techniques described in U.S. Patent No. 9,168,868, by way of example and incorporated herein by reference. it can. Monitoring the position of the target vehicle in the captured image over a period of time and / or the position in the captured image of one or more features (eg, tail lights, headlights, bumpers, wheels, etc.) associated with the target vehicle. Monitoring the relative distance, speed, and / or acceleration between the target vehicle and the host vehicle or between the target vehicle and one or more other objects in the host vehicle's environment. It may be possible to ask.

  [0363] The level of aggression of the identified target vehicle can be estimated from any suitable observed navigation characteristic of the target vehicle, or any combination of the observed navigation characteristics. For example, a determination of aggression can be made based on any observed characteristics and one or more predetermined threshold levels or any other suitable qualitative or quantitative analysis. In some embodiments, a target vehicle may be considered aggressive if it is observed that the target vehicle follows a host vehicle or another vehicle at a distance less than a predetermined aggressive distance threshold. . On the other hand, a target vehicle that is observed to follow a host vehicle or another vehicle at a distance that exceeds a predetermined aggressive distance threshold can be considered self-defense. The predetermined aggressive distance threshold need not be the same as the predetermined self-defense distance threshold. In addition, either or both the predetermined aggressive distance threshold and the predetermined self-defense distance threshold may include a range of values rather than explicit boundary values. Furthermore, neither the predetermined aggressive distance threshold nor the predetermined self-defense distance threshold need be fixed. Rather, these values or ranges may shift over time, and different thresholds / threshold ranges may be applied based on observed characteristics of the target vehicle. For example, the applied threshold may depend on one or more other characteristics of the target vehicle. The higher relative velocity and / or acceleration observed may justify the application of a larger threshold / threshold range. Conversely, lower relative speeds and / or accelerations, including zero relative speeds and / or accelerations, may justify the application of smaller distance thresholds / threshold ranges in making offensive / self-defense estimates.

  [0364] The aggressive / self-defense estimate may also be based on thresholds of relative speed and / or relative acceleration. A target vehicle may be considered aggressive if its observed relative speed and / or its relative acceleration with respect to another vehicle is above a predetermined level or range. A target vehicle may be considered self-defense if its observed relative speed with respect to another vehicle and / or its relative acceleration falls below a predetermined level or range.

  [0365] Although the aggressive / self-defective determination may be made based solely on any observed navigation characteristics, the determination may also depend on any combination of the observed characteristics. For example, as mentioned above, in some cases, a target vehicle may be considered aggressive based solely on being observed to follow another vehicle at a distance below a certain threshold or range. . However, in other cases, following another vehicle less than a predetermined distance (which may be the same or different from a threshold applied when the determination is based solely on distance), and with a relative value greater than a predetermined value or range If it has speed and / or relative acceleration, the target vehicle can be considered aggressive. Similarly, a target vehicle can be considered self-defense based solely on observing following another vehicle at a distance that exceeds a certain threshold or range. However, in other cases, following another vehicle over a predetermined distance (which may be the same as or different from the threshold applied when the determination is based solely on distance), and If it has a relative speed and / or a relative acceleration, the target vehicle can be considered self-defense. The system 100 may determine if the vehicle has an acceleration or deceleration of 0.5 G (eg, jerk 5 m / s3), if the vehicle has a lateral acceleration of 0.5 G on a lane change or on a curve, If either is done by another vehicle, the vehicle changes lanes and gives way to another vehicle beyond 0.3G deceleration or 3m / s3 jerk, and / or vehicle stops If two lanes are changed without incident, it can be determined that they are aggressive / self-defense.

  [0366] It is to be understood that reference to an amount above a range may indicate that the amount is above or included in all the values associated with the range. Similarly, reference to an amount below a range may indicate that the amount is below or included in all the values associated with the range. In addition, although the examples described for making offensive / self-defense estimates have been described with respect to distance, relative acceleration, and relative speed, any other suitable amount may be used. For example, the time to collision or any indirect indicator of target vehicle distance, acceleration, and / or speed at which the calculations may be used may be used. While the above example focuses on target vehicles for other vehicles, aggressive / self-defense estimates may be based on targets for any other type of obstacle (eg, pedestrians, road barriers, traffic lights, rubble, etc.). It should also be noted that this can be done by observing the navigation characteristics of the vehicle.

  [0367] Returning to the example shown in FIGS. 17A and 17B, when the host vehicle 1701 approaches the roundabout, the navigation system including its at least one processing device receives an image stream from a camera associated with the host vehicle. be able to. Any of the target vehicles 1703, 1705, 1706, 1708, and 1710 can be identified based on one or more analysis of the received images. Further, the navigation system can analyze one or more navigation characteristics of the identified target vehicle. The navigation system may recognize that the gap between target vehicles 1703 and 1705 represents a first opportunity for a potential merge into the roundabout. The navigation system may analyze the target vehicle 1703 to determine an aggression indicator associated with the target vehicle 1703. If the target vehicle 1703 is considered aggressive, the host vehicle's navigation system may decide to yield to the vehicle 1703 instead of merging in front of the vehicle 1703. On the other hand, if the target vehicle 1703 is considered self-defense, the navigation system of the host vehicle may attempt to complete the merging operation in front of the vehicle 1703.

  [0368] When the host vehicle 1701 reaches the roundabout, at least one processing device of the navigation system can analyze the captured images to determine a navigation characteristic associated with the target vehicle 1703. For example, based on the image, it may be determined that vehicle 1703 is following vehicle 1705 at a distance that provides sufficient clearance for host vehicle 1701 to enter safely. In fact, it may be determined that the vehicle 1703 is following the vehicle 1705 at a distance exceeding the aggressive distance threshold, and therefore, based on this information, the navigation system of the host vehicle may defend the target vehicle 1703 by itself. You can lean on identifying yourself as a target. However, in some situations, multiple navigation characteristics of the target vehicle may be analyzed in making an offensive / self-defense decision as discussed above. Expanding the analysis, the host vehicle's navigation system indicates that the target vehicle 1703 follows the target vehicle 1705 at a non-aggressive distance, but the vehicle 1703 exceeds one or more thresholds associated with aggressive behavior It may be determined that the vehicle has a relative speed and / or relative acceleration with respect to the vehicle 1705. In fact, the host vehicle 1701 can determine that the target vehicle 1703 is accelerating with respect to the vehicle 1705, and that the gap between the vehicles 1703 and 1705 is narrowing. Based on a further analysis of the relative speed, acceleration, and distance (and even the speed at which the gap between vehicles 1703 and 1705 is narrowing), host vehicle 1701 determines that target vehicle 1703 is acting aggressively. Can be determined. Thus, although there may be enough clearance for the host vehicle to safely navigate, host vehicle 1701 expects that merging in front of target vehicle 1703 will result in a vehicle aggressively navigating directly behind the host vehicle. can do. Further, if the host vehicle 1701 merges in front of the vehicle 1703, the target vehicle 1703 may continue to accelerate toward the host vehicle 1701, or may have a non-zero relative velocity based on behavior observed by image analysis or other sensor outputs. At the host vehicle 1701. Such a situation may be undesirable from a safety point of view and may also cause discomfort to the passengers of the host vehicle. For that reason, as shown in FIG. 17B, the host vehicle 1701 yields to the vehicle 1703, and the vehicle 1710 is deemed self-defense behind the vehicle 1703 and based on one or more analyzes of its navigation characteristics. You can decide to join the roundabout before.

  [0369] Returning to FIG. 18, in step 1809, at least one processing device of the host vehicle navigation system determines a target vehicle for the host vehicle based on the at least one identified navigation characteristic for the identified obstacle. A navigation action (eg, merging before the vehicle 1710 and behind the vehicle 1703) can be determined. To perform the navigation operation at (step 1811), the at least one processing device can cause at least one adjustment of a navigation actuator of the host vehicle in response to the determined navigation operation. For example, brakes can be applied to yield to vehicle 1703 in FIG. 17A, along with steering the wheels of the host vehicle to cause the host vehicle to enter the roundabout behind vehicle 1703 as shown in FIG. 17B. You can step on the accelerator.

  [0370] As described in the above examples, the navigation of the host vehicle may be based on the navigation characteristics of the target vehicle with respect to another vehicle or object. In addition, navigation of the host vehicle may be based solely on the navigation characteristics of the target vehicle without specific reference to another vehicle or object. For example, in step 1807 of FIG. 18, analyzing a plurality of images captured from the environment of the host vehicle determines at least one navigation characteristic of the identified target vehicle indicating a level of aggression associated with the target vehicle. May be able to The navigation characteristics may include speed, acceleration, etc. that need not be referenced to another object or target vehicle to make an aggressive / self-defense decision. For example, an observed acceleration and / or velocity associated with a target vehicle that is above a predetermined threshold or included in or exceeds a range may indicate aggressive behavior. Conversely, observed accelerations and / or velocities associated with the target vehicle that fall below a predetermined threshold or fall within or exceed a certain range may exhibit self-defense behavior.

  [0371] Of course, in some examples, the observed navigation characteristics (eg, position, distance, acceleration, etc.) can be referenced to the host vehicle to make an aggressive / self-defense determination. . For example, the observed navigation characteristics of the target vehicle, indicating the level of aggression associated with the target vehicle, include an increase in the relative acceleration between the target vehicle and the host vehicle, a following distance of the target vehicle behind the host vehicle, It may include the relative speed between the target vehicle and the host vehicle.

[0372] Navigation based on accident liability constraints
[0373] As described in the above section, planned navigation actions can be tested against predetermined constraints to ensure compliance with certain rules. In some embodiments, this concept can be extended to consider potential accident liability. As discussed below, the main goal of autonomous navigation is security. Absolute safety may not be possible (e.g., at least because certain host vehicles under autonomous control cannot control other vehicles around them and can only control their own movements). Using potential accident liability as a consideration in autonomous navigation and as a constraint on planned operations means that any operation in which a particular autonomous vehicle is considered dangerous (eg, potential accident liability is hosted It may help to ensure that no action is taken (which may be attributed to the vehicle). A desired level of accident avoidance (eg, less than 10 ⁻⁹ per driving hour) is achieved if the host vehicle only performs actions that are safe and that do not cause a host vehicle liability or liability accident. be able to.

  [0374] The challenge posed by the latest approaches to autonomous driving is a lack of security (or at least not being able to provide the desired level of safety) and a lack of scalability. Consider the issues that guarantee multi-agent safe driving. Because society is unlikely to tolerate machine-induced traffic deaths, an acceptable level of safety is paramount to the acceptance of autonomous vehicles. The goal may be to have no accident at all, but it is not possible because it is possible to envisage situations where multiple agents are generally involved in the accident and the accident only occurs due to the negligence of other agents. There is. For example, as shown in FIG. 19, a host vehicle 1901 is traveling on a multi-lane highway, and the host vehicle 1901 can control its own operation with respect to target vehicles 1903, 1905, 1907, and 1909. Inability to control the operation of the target vehicle surrounding itself. As a result, for example, when the vehicle 1905 suddenly breaks into the lane of the host vehicle on the collision course with the host vehicle, the host vehicle 1901 may not be able to avoid an accident with at least one of the target vehicles. To address this difficulty, the typical response of an autonomous vehicle expert is to use a data-driven approach in which safety verification becomes more stringent as data over more mileage is collected. is there.

[0375] However, to understand the problematic nature of data-driven approaches to safety, the fatality rate caused by accidents per hour (human) driving is known to be ^10-6. Consider first that To allow society to allow machines to replace humans in driving tasks, it is reasonable to assume that mortality should be reduced to three orders of magnitude, a probability of 10 ⁻⁹ per hour. This estimate is similar to the assumed fatality of airbags and aviation standards. For example, ^10-9 is the probability that a wing will spontaneously deviate from an aircraft in the air. However, attempting to guarantee security using a data driven approach that provides additional reliability in proportion to the cumulative mileage is not practical. The amount of data required to ensure lethality operation per 10 ^-9 1 hour is proportional to the reverse (i.e. data of 10 ⁹ hours), it is approximately 30 billion miles around. In addition, multi-agent systems can interact with their environment and have a realistic simulator available that emulates real human driving with all its richness and complexity, such as reckless driving Otherwise (but the problem of verifying a simulator is more difficult than creating a secure autonomous vehicle agent), it is likely that it cannot be verified off-line. Any change to the planning and control software requires the collection of new data of the same size, which is clearly unmanageable and impractical. Further, developing a system with data always faces a lack of interpretability and intelligibility of the actions performed, and if an autonomous vehicle (AV) causes an accident leading to accidental death, the reason must be understood. No. As a result, while a model-based approach for safety is needed, existing "functional safety" and ASIL requirements in the automotive industry are not designed to address a multi-agent environment.

  [0376] A second major challenge in developing a safe driving model for autonomous vehicles is the need for scalability. The premise underlying AV is more than "creating a better world", but rather based on the assumption that mobility without a driver can be maintained at a lower cost than with a driver. This premise is always tied to the concept of scalability in the sense of supporting mass production of AVs (in millions) and more importantly of supporting the trivial incremental costs of being able to operate in new cities. . Therefore, the cost of computation and detection is important, and when AV is mass produced, the cost of verification and the ability to drive "anywhere" rather than a few selected cities is also necessary to sustain the business. Requirements.

  [0377] The problems with recent approaches are: "i) the required" computational density ", (ii) the method by which the high-definition map is defined and created, and (iii) the required specifications of the sensor. It's a brute force approach. The brute force approach is contrary to scalability, infinite on-board calculations are ubiquitous, the cost of building and maintaining HD maps is trivial and scalable, and novel and ultra-high sensors have been developed to match the grade of the car. Transfer the burden to the future that will be commercialized at trivial cost. In the future where any of the above will be realized, indeed it makes sense, but obtaining all of the above influences is likely to be a low probability event. Thus, providing a formal model that is socially acceptable and scalable in terms of supporting millions of vehicles driving anywhere in the developed world, integrating safety and scalability within AV programs There is a need to.

  [0378] The disclosed embodiments can provide target safety levels (and even exceed safety targets) and scale to systems that include millions (or more) autonomous vehicles. Represent a solution that can also be. The safety, the introduction of a model called "Responsible sensitive safety" (RSS), this model is to formalize the concept are possible and can be explained the construction of "accident negligence", the operation of the robotic agents " Incorporate a sense of responsibility. The definition of RSS is agnostic by the way it is implemented, which is a key feature to facilitate the goal of creating a convincing global security model. RSS is motivated by the idea that agents play an asymmetric role in an accident (as seen in FIG. 19), where typically only one of the agents is responsible for the accident, and Take responsible. The RSS model also includes a formal treatment of “careful driving” under limited sensing conditions where not all agents are always visible (eg, due to occlusion). One of the main goals of the RSS model is to ensure that agents do not have "negligence" on themselves or cause accidents for which they are responsible. The model may only be useful with an RSS-compliant efficient policy (e.g., a function that maps "sensing state" to action). For example, actions that appear to be harmless at the moment can lead to catastrophe in the distant future ("butterfly effect"). RSS may help to establish a set of local constraints on the short-term future that can guarantee (or at least virtually guarantee) that future accidents will not occur as a result of host vehicle operation.

[0379] Another contribution focuses on the introduction of "semantic" languages consisting of units, measurements, and working spaces, and specifications on how they are incorporated into AV planning, detection, and operation. Will be expanded to. In order to know what the semantics are, in this context, consider how a person taking a driving course is instructed to think about a "driving policy". These instructions are not geometrical a ( "to 13.7 meters traveling at the current speed, then accelerated at a rate of 0.8 m / s ^2" does not take the form). Rather, these instructions are of a semantic nature ("run ahead of the car" or "pass the car on the left"). The typical language of the human driving policy relates to longitudinal and lateral targets rather than geometric units of acceleration vectors. Formal semantic languages are linked to the computational complexity of plans that do not grow exponentially with time and the number of agents, linked to the way safety and comfort interact, and the computation of detection is It can be useful in several ways, linked to the methods defined, and to the specifications of how they interact within the modality of the sensor and the fusion methodology. (Based on semantic language) fusion methodologies, while performing only off-line verification over the data set operating data of about 10 ⁵ hours, RSS model realized mortality 1 the required per operating hour ^10-9 Can be guaranteed.

[0380] For example, in the setting of the reinforcement learning, plan no matter period used for, Q function on semantic space the number of the track to be examined at any given time is limited by the 10 ⁴ (e.g. , A function that evaluates the long-term quality of performing the action aεA when the agent is in the state sεS. Given such a Q function, the natural choice of action is the highest quality, π (s) = argmax _a Q (s, a)). The signal-to-noise ratio in this space can be high, allowing an effective machine learning method to successfully model the Q-function. In the case of detection calculations, semantics may make it possible to distinguish between errors affecting safety and those affecting driving comfort. Defines a PAC model for detection (probabilistic and approximately correct (PAC)) tied to the Q function, borrows PAC learning terms from Valiant, and is RSS compliant but allows for optimization of driving comfort And how to incorporate measurement errors into the plan. As other standard measures of error, such as errors relative to the global coordinate system, may not conform to the PAC detection model, the semantic language can be important to the success of this model's property aspects. In addition, semantic languages can be constructed using low-bandwidth sensing data, and thus can be an important enabler for defining HD maps that can be constructed by crowdsourcing and support scalability. .

  [0381] In summary, the disclosed embodiments may include a formal model that spans a key component of AV, namely detection, planning, and operation. This model may help to ensure that there is no incident of responsibility on the AV side from a planning perspective. Furthermore, with the PAC detection model, even with detection errors, the described fusion methodology may require only a very reasonable scale of the offline data set to comply with the described safety model. Furthermore, this model can link security and scalability by means of a semantic language, thereby providing a complete methodology for secure and scalable AV. Finally, it should be noted that developing an approved safety model adopted by the industry and regulatory agencies may be a necessary condition for the success of AV.

  [0382] The RSS model can generally follow classical detection-plan-motion robot control methods. The sensing system may be responsible for understanding the current state of the host vehicle's environment. The planning portion, which can be referred to as a "driving policy" and can be implemented as a trained system (e.g., a neural network) or a set of hard-coded instructions by a combination, is a selection of available options to achieve a driving goal. From a point of view, it may be responsible for determining what is the next best move (eg, how to move from the left lane to the right lane to exit the highway). The operating part is responsible for performing the plan (e.g., an actuator and one or more actuators for steering, accelerating, and / or braking the vehicle to perform a selected navigation operation). Controller system). The embodiments described below mainly focus on the sensing part and the planning part.

  [0383] An accident may be triggered by a detection error or a planning error. Planning is a multi-agent attempt, as there are other road users (humans and machines) that respond to the operation of the AV. The RSS model described is designed to address security, especially for planning parts. This can be called multi-agent security. In a statistical approach, the estimation of the probability of a planning error may be performed "on-line". That is, each time the software is updated, it must drive billions of miles with the new version to provide an acceptable level of estimation of the frequency of planning errors. This is obviously not feasible. As an alternative, the RSS model may provide 100% assurance (or virtually 100% assurance) that the planning module will not make a mistake where the AV is negligible (the concept of "negligence" is formally defined) . The RSS model may also provide an efficient means for its verification that does not rely on online testing.

[0384] Since the detection can be independent of the vehicle's operation, and thus the probability of severe detection errors can be verified using "off-line" data, errors in the detection system are more Can be easy. However, even it is difficult that the collecting off-line data of the operation of more than ¹⁰⁹ hours. As part of the description of the disclosed sensing system, a fusion approach is described that can be verified using a significantly smaller amount of data.

  [0385] The described RSS system may be scalable to millions of vehicles. For example, the described semantic driving policies and applied safety constraints may be consistent with sensing and mapping requirements that can scale to millions of vehicles even in today's technology.

[0386] Because of the underlying components of such systems, there are security definitions that are the minimum standards that an AV system may need to adhere to. In the following technical lemma, statistical methods for the verification of AV systems are not feasible, even for verifying simple assertions such as "the system causes N accidents per hour". It is shown. This implies that only model-based security definitions are a viable tool for verifying AV systems. Lemma 1 Let X be a probability space, and let A be an event that satisfies Pr (A) = p ₁ <0.1. From X,

sample iid sample,

And Therefore, Pr (Z = 0) ≧ e− ² holds.
Proof We use the inequality 1-x ≧ e ^−2x (provided for completeness in Appendix A.1) to obtain

System 1 AV system AV ₁ is assumed small but accidents with insufficient probability _{p 1.} Any deterministic verification procedure given 1 / p ₁ samples do not distinguish between different AV system AV ₀ is not at all cause AV ₁ and accidents with a certain probability.

[0387] To obtain a view across typical values of such probabilities, assume that an accident probability of ^10-9 per hour is desired, and that a particular AV system provides only a probability of ^10-8 . Even if the system is to give the driver of 10 ⁸ hours, there is a certain probability that can not be shown the verification process that the system is dangerous.

  [0388] Finally, it should be noted that this challenge relates to disabling a single particular dangerous AV system. The complete solution cannot be considered as a single system because new versions, bug fixes, and updates are required. Each change creates a new system from the verifier's point of view, even a single line of code change. Thus, a statistically validated solution must do it online over a new sample after any small modification or change to address the shift in the distribution of states observed and reached by the new system. Obtaining such a huge number of samples repeatedly and systematically, again with a certain probability of not being able to verify the system, is not feasible.

  [0389] Furthermore, any statistical claims must be formalized for measurement. Claiming statistical properties over the number of accidents the system causes is significantly weaker than claiming that the system operates in a safe manner. To say that, we need to formally define what safety is.

[0390] Absolute security is impossible
[0391] The action a performed by the car c can be considered absolutely safe if it is possible at a certain point in the future that no accident will occur after that action. For example, observing a simple driving scenario as shown in FIG. 19 shows that it is impossible to realize absolute safety. From the viewpoint of the vehicle 1901, there is no operation that can guarantee that the surrounding vehicles do not collide with itself. It is also impossible to solve the problem by prohibiting autonomous vehicles from being in such situations. All highways with more than two lanes will cause this problem at some point, and forbidding this scenario would require staying in the garage. These implications seem seemingly disappointing. Nothing is absolutely secure. However, as is evident from the fact that human drivers do not adhere to the absolute safety requirements, such requirements for obtaining the absolute safety defined above can be too stringent. Rather, humans act according to the concept of security, which relies on responsibility.

[0392] Responsibility sensitive security
[0393] An important aspect missing from the concept of absolute safety is the asymmetry of most accidents, that is, it is usually one of the drivers who is responsible for the collision and therefore should be negligent. is there. In the example of FIG. 19, for example, when the left car 1909 suddenly collides, the center car 1901 has no fault. To formalize the fact that it considers the central car 1901 not responsible, the behavior of the AV 1901 staying in its lane can be considered safe. In order to do that, we explain the formal concept of "accident negligence" or accident liability, which can serve as a premise for safe driving techniques.

[0394] As an example, 2 Tsunokuruma running at the same speed in tandem along a straight road c _f, consider the simple case of c _{r. Suppose} that the previous car, cf, suddenly braked on an obstacle that appeared on the road and was able to avoid it. Unfortunately, c _r did not maintain a sufficient distance from the c _f impinges on the rear of c _f unable to react in time. Negligence is obvious that in the c _r, keeping a safe distance from the front of the car, is that although unexpected advance in preparation for a reasonable brakes, is the responsibility of the back of the car.

  [0395] Next, a wider range of scenario groups, that is, driving on a multi-lane road where vehicles can freely change lanes, interrupting the route of another vehicle, driving at various speeds, and the like will be examined. To simplify the following explanation, a straight road on a plane whose horizontal and vertical axes are the x-axis and the y-axis, respectively, is assumed. This can be achieved under mild conditions by defining a homomorphism between the actual curved road and the straight road. In addition, we consider discrete space-time. The definition may help distinguish between two sets of intuitively different cases, those that are simple without significant lateral manipulation and those that are more complex with lateral movement.

Definition 1 (Corridor of Car) The corridor of car c has the range [cx _{, left} , cx _{, right} ] × [± ∞], where _cx , _left , cx _{, right.} Is the position of the leftmost corner and the position of the rightmost corner of c.

Definition 2 (Interruption) At time t−1, car c ₁ (car 2003 in FIGS. 20A and 20B) does not cross the corridor of car c ₀ (car 2001 in FIGS. 20A and 20B), and at time t when crossing, the car _{c 1} interrupts the corridor of the car _{c 0} at time t.

  [0398] A further distinction can be made between the front / rear of the corridor. The term "direction of interruption" may refer to moving in the direction of the associated corridor boundary. These definitions can define cases involving lateral movement. In a simple case where such a case does not occur, such as a simple case where one car follows another, a safe vertical distance is determined.

Definition 3 (Safe Longitudinal Distance) The vertical distance 2101 (FIG. 21) between a car _cr (vehicle 2103) and another car c _f (vehicle 2105) in the corridor ahead of _cr is is safe with respect to the reaction time p, any brake commands _a carried out by _{c f, | a | <a} max, the _{brake, if c r} is added to the maximum braking to a complete stop from time p, _{c r} is, c _f and not conflict.

[0400] The following Lemma _2, the speed of c r, _{c f,} reaction times [rho, and the maximum acceleration _{a max,} calculating the d according to _brake. Both ρ and a _{max, break} are constants and should be determined to some reasonable value by rules.

[0401] The vehicle behind the c _f in Lemma on second longitudinal axis and c _r. Let the maximum brake command and the maximum accelerator command be a _{max, break} , a _{max, accel,} and _let the reaction time of _{cr be} ρ. The longitudinal velocity of the vehicle and υ _r, υ _f, to their lengths _l f, and _{l r.} defined as _{υ p, max = υ r +} ρ · a max, accel,

Is determined. L ₌ a _{(l r + l f) /} 2. Therefore, the minimum safe longitudinal distance of _cr is:

Proof The distance at the time point t is _dt . To prevent accidents, _dt > L must be obtained for all t. To construct the d _min, it is necessary to find a lower bound that is the most demanding need for d _0. Obviously, _{d 0} should be at least L. As long as the two cars T ≧ [rho seconds does not stop, the speed of the preceding car, upsilon _f -Ta _max, whereas a _brake, the speed of _{c r is, υ ρ, max - (T} -ρ) a _The upper limit is set by _{max and accel} . So the distance between these cars after T seconds is

Sets a lower limit.

[0403] T _r is c _r is a time to stop completely (0 speed), T _f is to be noted that the other vehicle is time to stop completely. _{a max, brake (T r -T} f) = υ ρ, max -υ f + ρa max, brake is established, therefore, when it is _{T r} ≦ _{T f,} it is sufficient to require that a _d 0> L Note that If _Tr > _Tf , the following equation holds.

Requesting d _Tr > L and organizing the terms completes the proof.

[0404] Finally, a comparison operator that allows comparison with some concepts of "margin" is defined. When comparing lengths and velocities, etc., very similar quantities need to be recognized as "equal".
Definition 4 (mu Comparative) two numbers a, mu comparison of b in the case of a> b + mu is a> _mu b, in the case of a <b-mu is a <μ _b, | a- When b | ≦ μ, a = _μb .

The following comparisons (argmin, argmax, etc.) are μ comparisons for some appropriate μs. Assume that an accident has occurred between cars c ₁ and c ₂ . Determine the relevant moments that need to be investigated in order to consider who is responsible for the fault of the accident. This is a point in time before the accident, which was intuitively an "irrecoverable point", after which nothing can be done to prevent the accident.

[0406] Definition 5 (time of negligence)
There is an intersection between one of the cars and the corridor of the other car,
・ The vertical distance was not safe,
It is the earliest point before the accident.

[0407] There are clearly such moments at the moment of the accident because both conditions apply. The points of negligence can be divided into two distinct categories:
• An interruption also occurs, ie it is the first moment of the intersection of one car with the corridor of another car, which is at a dangerous distance.
No interruptions, i.e. a safe longitudinal distance has already been crossed with the corridor, and that distance has become dangerous at the time of negligence.

Definition 6 (μ-Lossing by Lateral Speed) It is assumed that an interrupt occurs between the vehicles c ₁ and c ₂ . If the lateral velocity relative to the direction of the interrupt is faster mu than the lateral speed of c _2, c ₁ is mu-Lose Then it said laterally rate.

  [0409] It should be noted that the direction of velocity is important. For example, a speed of -1,1 (both cars colliding with each other) is tied, but if the speed is 1,1 + .mu. / 2, a car that has a positive direction relative to the other car is negligible. is there. Intuitively, this definition allows one to neglect a car that crashes very quickly in the lateral direction to another car.

[0410] Definition 7 (lateral position by (μ _1, μ ₂₎ - Win) Assume an interrupt occurs between the car _c 1, _{c 2.} Interrupt its lateral position relative to the center (the closest center to an interrupt to the associated corridor) lanes (in absolute value) mu less than _1, if than that of c ₂ mu ₂ small, c ₁ is in lateral position (Μ ₁ , μ ₂ ) —You can say you win.

[0411] Intuitively, very close to the center of the lane (mu ₁₎ and other much more than cars (mu ₂ minute only) is close, does not inflict negligence the car.

[0412] Definition 8 (Negligence) The negligence or liability of the accident between cars c ₁ and c ₂ is a function of the state at the time of the negligence and is defined as follows:
If the fault was not at the time of the interruption, the fault is in the car behind.
· If the fault point is also a time interrupt, for one of the vehicles, without loss of generality of the part for c ₁ default the following two terms μ condition, i.e.,
• Loss in lateral speed. • Unless winning in lateral position is true, fault is in both cars.

  [0413] In this case, c1 is forgiven. In other words, when a dangerous interrupt occurs, both cars are negligible except when one of the cars is not (significantly) laterally fast and is (significantly) close to the center of the lane. This captures the desired behavior of maintaining a safe distance when following a car, and interrupting only at a safe distance when simply interrupting the corridor of a car traveling in its own lane. As discussed further below, an automatic controller-based system for following the above safety guidelines should not cause excessive self-defense driving.

[0414] Dealing with limited detection
[0415] After considering the highway example, the second example then addresses the limited detection problem. A very common human response when accused of an accident belongs to the "but not visible" category. That is often true. Humans, sometimes due to unconscious decisions concentrating on other parts of the road, sometimes inadvertently, and sometimes due to physical constraints (it is impossible to see a pedestrian behind a parked vehicle) Has limited detection capabilities. Of these human constraints, advanced automatic detection systems are only affected by the latter, and the computer is never inattentive, and the 360 ° view of the road indicates that the computer exceeds human detection capabilities. cause. Returning to the “but not visible” example, the appropriate answer is “should have been more careful”. To formalize what is conservative with limited detection, consider the scenario shown in FIG. Car 2201 (c ₀ ) is leaving the parking lot and trying to merge on a (possibly) crowded road, but because its view is blocked by building 2203, it is not possible to see if there is a car on the road Can not. Assume that this is a narrow road in a city with a speed limit of 30 km / h. The human driver's action is to slowly merge on the road and gain more vision until the detection limit is no longer limited. A significant moment should be defined, which is the first time the occluded object is first revealed, and after the object is revealed, it is addressed like any other detectable object.

  [0416] Definition 9 (Exposure Time) The exposure time of an object is the first time at which it can be seen.

[0417] Definition 10 (negligence unfair rate) in the exposed time or after car _{c 1} (car 2205) is traveling at a speed υ> υ _{limit, c 0} is assumed that it was not traveling at that speed . In this case, negligence is only _{c 1.} c ₁ can be said to bear the negligence by undue speed.

[0418] This extension allows the c ₀ is out safely from the parking lot. Using our earlier definition of responsibility sensitive security with a dynamic υ _limit definition (using reasonable margins in addition to road conditions and speed limits), in the worst case illustrated, What is needed is an interruption while assuming c ₁ does not exceed upsilon _limit is only possible to check whether a safe longitudinal distance. Intuitively, this is, running away from the slower and shield, thereby slowly widened their field of view, safely encourage the c ₀ to allow later to be joined on the road.

[0419] Extending the definition of accident liability to this limited detection base case, an extended group could address similar cases. What are the objects that can be obscured (potentially fast cars can't hide between two cars parked in close proximity, but pedestrians can) worst of whether the operation of the case looks like the target can be done (υ _limit of the pedestrian is much smaller than the car of υ _limit) is simple assumptions about, implies constraints on the operation, in the worst case You need to be prepared and have the ability to react to sudden exposure times. A more complex example in a city scenario can be obtained from a pedestrian scenario, possibly hidden by a parked vehicle. Accidental negligence for accidents with pedestrians can be determined.

[0420] Definition 11 (Negligence of accident with pedestrian) Negligence of accident with pedestrian is the following three:
A pedestrian collides with the side of the vehicle, and the lateral speed of the vehicle is less than μ with respect to the direction of the collision,
That the speed of the pedestrian at or after the exposure exceeds υ _limit ;
Fault is always on the car unless one of the things that the car has stopped completely does not apply.

  [0421] Irregularly, if the car does not collide with the pedestrian faster than μ, but the pedestrian collides with the side of the car, or the car is stopped, or the pedestrian is not necessarily in the collision direction There is no fault in the car only if you are running superhumanly fast in any direction.

  [0422] Although the described system may not guarantee absolute security, it can result in scenarios where very few, if any, accidents occur between autonomous vehicles. For example, if all cars (and other road users) can be verified without any problem that the accident as a result of the action performed is negligible, the accident can be eliminated. By definition, there is at least one liable car for every accident. Thus, if no car takes action that could be responsible for the resulting accident (according to the RSS model above), then the accident should not occur at all, and an unmanageable and impractical statistical method can be used. Provides the kind of absolute security or near absolute security required.

  [0423] The structure of all roads is not simple. Some, such as intersections and roundabouts, include more complex situations with various priority road rules. All hidden objects should not be considered cars or pedestrians, but bicycles and motorbikes, any legal road users. The principles introduced in this section can be extended to these additional cases.

[0424] Efficiently verified conditions for liability sensitive security
[0425] This section discusses aspects of implementing RSS. To begin with, it should be noted that the actions taken here can have a butterfly effect which triggers a series of events for an accident, for example after a 10-minute run. A "brute force" approach to examining all possible future results is not only impractical, but also likely impossible. To overcome this problem, the above definitions of responsibility-sensitive security will now be described, along with a computationally efficient way to verify them.

[0426] Computationally feasible security verification.
[0427] The main mathematical tool for computationally feasible verification is "induction". To prove an assertion by induction, we start by proving the assertion on a simple case, and then each step of induction extends the proof to a more complex case. To indicate this induction tool may be any useful safety verification, again consider the simple example of a car c _r to add run another car c _f (Figure 21). The following constraints can be applied to the _cr policy. Even if c _f in addition to deceleration -a _max, the resulting distance between the c _r and c _f at the next time step (defined in definition 3 and Lemma 2) at least a safe longitudinal distance So that the policy can select any acceleration command at each time step t. If such action is not present, _{c r} shall apply deceleration _{-a max.} The following lemma, using induction, to prove that never cause accidents and c _f any policy according to the above constraints.

[0428] Under the assumptions given in Lemma 3 Definition 3, when following constraints policy c _r is indicated above, c _r is never an accident with c _f.

[0429] Proof The proof is based on induction. With respect to the basis of induction, we start from the first state where the distance between two vehicles is safe (according to Lemma 2). The steps of induction are as follows. Consider the distance between the c _r and c _f at a certain point in time t. (C _f be performed by a maximum deceleration) no problem if there is operation result in safe distance. If all the operations cannot guarantee a safe distance, the maximum time during which the operation is not the maximum deceleration is set as t ′ <t. By inductive assumption, at time t '+ 1, we were at a safe distance from which we made the maximum deceleration. Thus, by definition of the safe distance, there is no collision from time t 'to the present, which concludes the proof.

[0430] The above example demonstrates a more general concept, namely that there is some emergency action taken by _cr in extreme cases to return _cr to a "safe state". It should be noted that the constraints on the policy described above depend only on one future time step and can therefore be verified in a computationally efficient manner.

  [0431] In order to generalize the concept of sufficient local characteristics for RSS, a default emergency policy (DEP) is first defined, and it is defined as a component to define local characteristics of action execution called "conservative". use. Next, it is shown that taking only prudent commands is sufficient for RSS.

Definition 12 (Default Emergency Policy) The default emergency policy (DEP) is to apply maximum braking force and maximum heading change towards zero degree heading for lane. Changes in the maximum braking force and heading are derived from the physical parameters of the vehicle (and possibly also from weather and road conditions). Definition 13 (secure state) State s is secure if performing a DEP starting from state s does not cause an accident with negligence of oneself. If it results in a safe condition, as in the simple case of a car following another car, the command is defined to be prudent. Suppose definition 14 to be in (careful command) state _{s 0.} If the next state s ₁ for the set A of possible commands other vehicle is likely to carry out here is safe, it commands a is prudent. The above definition depends on the worst case command in set A that other vehicles can make. Build set A based on reasonable upper bounds for maximum braking / acceleration and lateral movement.

  [0433] The following theory, again by induction, proves that there is no accident in which oneself is negligible if only a careful command is issued. Theorem 1 Assume that c is in a safe state at time 0, that c issues only deliberate commands for all time steps, and that there are no deliberate commands at any time step, then that c applies DEP. Therefore, c never causes an accident in which he has negligence. By proof induction. The basis of induction is derived from the steps from defining secure states and defining prudent commands.

[0434] One benefit of this approach is that it may not be necessary to look into an infinite future, because you can quickly return to a secure state and continue safely from there. Furthermore, given that re-planning at t + 1, and thus the DEP can be performed as needed, instead of the possible long-term plan that can be in mind (the plan can be changed at t + 1), Only the command given at time t should be examined. Then, when validated at runtime by this transparent model, it is possible to incorporate the learning components into the system. Finally, this local validation implies a complete future RSS, which is our desired goal. The hindrance to implementation is that the definition of prudence includes all trajectories that another agent can execute up to t _break , which is a huge space even at moderate t _breaks . To address this problem, we will then develop a conservative and thus an efficiently computable method for validating RSS in a scalable way.

[0435] Efficient prudent verification
[0436] The first view is that certain vehicles that can execute commands from set A that cause an accident where we perform negligence while performing DEP.

When and only if the condition is not secure. Therefore,

The procedure can be performed sequentially for scenes where there is a single target vehicle, and in the general case each of the other vehicles in the scene.

[0437] When considering a single target vehicle, an accident with negligence in c occurs, all in group A

Indicated by

Action a is not conservative when and only if there is a series of commands for As already proved, at time 0,

If it is true that is in the front corridor of c, there is a simple way to check the prudence of a,

Applying maximum braking for one time step (we do a) only needs to verify that the resulting longitudinal distance remains safe. The following lemma gives sufficient conditions for prudence in more complicated cases where lateral manipulation should also be considered.

Lemma 4 At time T = 0,

Is not in the front corridor of c. In all T∈ _{(0, t brake],} if there is no danger interruption in negligence in c, a is prudent.

[0439] Proof: a is not cautious, that is, causes an accident in which c has negligence

Suppose that There must be an interruption time T before the accident. First assume that T> 0. If this interrupt were at a safe longitudinal distance, there would be no accidents in which we were negligible because DEP was executed with a deceleration of -a _max and based on the definition of safe longitudinal distance (here Where the reaction time ρ exceeds the time resolution of the step). If the interrupt was not secure, then, according to the lemma theorem, negligence is not in c, and so is accidental negligence.

[0440] Finally, if T <0, then at the moment of the interruption,

Was in the corridor behind c. By induction, c had only made safe interrupts in the past, and therefore either the interrupt was safe or c was negligible. In each case, c is not the fault of the current accident.

[0441] In light of Lemma 4, there remains a problem of checking whether there is a dangerous interrupt in which c is negligible. An efficient algorithm for examining the possibility of a dangerous interrupt at time t is shown below. To verify the full trajectory, discretize the time interval [0, t _break ] and (with a slightly larger value of p in the definition of safe distance to ensure that discretization is not compromised) Apply algorithm to time intervals.

Minimum diagonal length of the rectangle that bounds

And For each time point t∈ _{[0, t brake],} defines a _{c length} (t) to be "spun" longitudinal c at time t,

And c _width [t] is determined in a similar manner,

Holds.

  [0442] The following theorem proving the correctness of the above algorithm.

  Theorem 2 When Algorithm 1 returns “unrealizable”, there is no dangerous interruption in which the own vehicle is negligible at time t. To prove this theorem, we use the following important lemma that proves the validity of the two components of Algorithm 1. Start with the vertical feasibility.

  Lemma 5 Under the notation of Algorithm 1, if the procedure for checking the feasibility in the vertical direction by returning “unrealizable” can be concluded, there is a dangerous interrupt in which the vehicle is at fault at time t. I can't get it.

[0445] Proof. Ignoring the lateral aspects of the interrupt operation, c

Examine the mere possibility that the vertical distance between is dangerous. position

At time t

It is clear that the locations that can be obtained by the demarcation are bounded.

Is satisfied, it can be obtained that any obtainable dangerous distance in the vertical direction is ≧ L.

And using the acceleration command bounded by a _{y, min} , a _{y, max} ,

,

It is assumed by the inconsistency rule that the dangerous vertical position and speed shown by are obtained.

By the definition of

And therefore the inter-vehicle distance is greater, ie,

Holds.

Speed, (y [t], υ y [t]) for relative safe longitudinally by the definition of the longitudinal risk, resulting

Is

Must be smaller than However, to achieve slower speeds, throughout the time window [0, t],

Must use an average acceleration of less than a _{y, min} , so that the vertical danger is inconsistent with that obtained using commands bounded by a _{y, min} , a _{y, max} . Is evident.

In the case, the proof is completed by considering the symmetric arguments.

[0446] Next, the feasibility of the horizontal direction.
[0447] Lemma 6 Under the notation of algorithm 1, if the procedure for examining the feasibility in the lateral direction by returning "unrealizable" can be concluded, there is a dangerous interruption in which the vehicle is at fault at time t. I can't get it.

Proof First, assume that x [t] = 0,

, And v _x [t], it is clear that simple changes in coordinates and consideration of relative velocity do not lose generality. Furthermore, with similar arguments,

Can be easily extended to the case where

[0449] In our case, the position of the car involved in the interruption (x [t], x [t] -W) depends on any lateral position that affects negligence (μ1, μ2). Note that it implies a winning characteristic. To laterally speed μ- “tie” (c may be (μ1, μ2) at lateral position-if victory is not won, this may be sufficient to inflict negligence on c), or To win μ- (c in lateral position (μ1, μ2) —if win, this is necessary to inflict negligence on c), v _x [ According to our assumptions on t], at time t:

The maximum lateral speed that can be used is obtained to be zero. Last position
Using the lateral acceleration bounded by a _{x, max} with
Starts with
It is left to check if there is an operation ending with. In words, an interrupt that ends at the desired lateral velocity of zero.

[0450] Algorithm definition

I want to be recalled. Assume that t _top <0. This is the maximum lateral acceleration,

When using

Implies that the time needed to reach zero lateral velocity is less than t. This implies that there is no operation that can be performed to reach the desired speed without delay, and thus there is no problematic operation. Thus, if this procedure returns "impossible" due to t _top <0, there is no real possibility of a dangerous interrupt in which c is actually negligible.

Consider the case where t _top > 0. Here, the procedure returns “impossible” due to x _max <−W.

And within the time range [0, t], parameterized by a

Consider a group of lateral velocity profiles for In the same way as used in the algorithm,

Is determined for each a. All of

Note that t _top (a)> 0 holds for Then, all time t'0 [0, t] the velocity profile _{u a} related determined as follows.

[0452] First of all, _{u a} constraint

It can be seen that satisfying. Second, it is possible to calculate the distance traveled while using u _a, it is because this distance corresponds to the integral of the piecewise linear function. To reach

X _max defined in the algorithm is exactly:

Note that Third, it can be seen that the travel distance monotonically increases with a and is not bounded. Therefore, any desired final position

about,

Is present. Specifically, when x = x [t] -W, such a value exists and is indicated by a _cut .

[0453] <because it is _{x [t] -W, a cut} > x max which defines within the algorithm should be recognized _{that a x, max} is obtained. This is insufficient to show that no valid operation can result in a position ≧ x [t] −W, which is only implied for members of the U group. Then, even outside of U, all velocity profiles that obtain a final position of x [t] -W prove that at least a _cut acceleration values must be used on the road, such acceleration values being Override the speed profile and thus complete the certification.

[0454] Some velocity profiles u have boundary constraints

Suppose that Furthermore, if the velocity profile u has some final position

Suppose we get Thus, the following equation is obtained.

[0455] For all τ,

Is assumed to hold. In particular,

Holds. From the mean value theorem,

There are to become such _{ζ0 [0, t top (a} cut)], a x, acceleration greater than _max (i.e. u ', differential speed) implies infeasible of u on the grounds that the use of .

[0456] Next,

Does not apply to all τ. In that case,

Is established,

There must be a point where If such a point τ _large is in [0, t _top (a _cut )], too large an acceleration is used, using the average value theorem in the same way as above, and {0 [0, τ _large ] Get. If this point exists only within [t _top (a _cut ), t], a similar argument gives the point ζ0 [τ _{large, t} ] at which an acceleration value less than −ax _{, max} is used, and Conclude. With the above lemma, the proof of Theorem 2 is immediate.

[0457] Safety verification-shielding
[0458] In a similar manner to dealing with observed objects, the extension of prudence to hidden objects is theorem 1 with the proviso that implicitly implies that there is no accident that we are negligible. It can be defined using a similar theorem.

  Definition 15 (Conservativeness against Hidden Objects) If the time of exposure of the object is t + 1, the default emergency policy (DEP) is ordered at t + 1, and there is no accident with our fault, the command given at time t is Be careful with hidden objects.

  Lemma 7: When only a careful command is issued for a hidden object and a non-hidden object, there is no accident in which oneself is negligible.

  [0461] Proof It is assumed that an accident in which oneself is negligible has occurred at time t in a state where the exposure time is t'≤t. By virtue of the prudence assumption, the command given at t'-1 made it possible to order a DEP at t 'without being liable to fault. We have not explicitly ordered DEP at time t 'because there was an accident in which we were negligent. However, since t ', we were safe against non-hidden objects, so the command we gave was safe, and there was no accident in which we were negligent.

  [0462] Again, we provide an efficient way to confirm vigilance against worst case assumptions about hidden objects, enabling feasible and scalable RSS.

[0463] Driving policy
[0464] The driving policy is based on the detected state (description of the world surrounding the human) and the driving command (for example, the command determines where the car should be and at what speed one second from now, (The next second is the lateral and vertical acceleration.) The driving command is communicated to the controller, which aims to actually move the car to the desired position / speed.

  [0465] The previous section described the formal safety model and the proposed constraints on the commands issued by the driving policy to ensure safety. Security constraints are designed for extreme cases. In general, they don't even want to need these constraints and want to build a driving policy that gives them a comfortable ride. The focus of this section is on how to build efficient driving policies, specifically those that require computational resources that can scale to millions of vehicles. Currently, this description does not address the issue of how to obtain the detection state, and assumes an ideal detection state that faithfully represents the world around humans without any restrictions. The effect of inaccurate detection on the operating policy will be discussed in a later section.

[0466] In the language of reinforcement learning (RL) discussed in the above section, the problem that determines the driving policy is clarified. At each iteration of the RL, the agent observes the state shown by describing s _t the world, it should choose to operate indicated by a _t on the basis of the policy function π that maps states operation. The results of their operations and their other factors that control does not reach the (operation or the like of the other agent), the state of the world is changed to s _t +1. (State, operation) sequence

Indicated by All policies trigger a probability function over a (state, action) sequence. This probability function may be affected by actions performed by the agent, but also depends on the environment (specifically, how other agents behave). Denote by P _{π the} probability over the (state, action) sequence caused by _π . The quality of the policy is

Is defined as

Is the sequence

Is a reward function that measures how good is. In most cases,

Is

Where ρ (s, a) is an instantaneous reward function that measures the immediate quality of being in state s and performing operation a. For simplicity, we proceed with this simpler case.

[0467] To clarify the problem of operation policy in the above RL languages, roads, as well as the position of the other road users not the vehicle only, speed, and some representation of the acceleration and s _t. Horizontal and vertical acceleration command and a _t. Next state s _{t + 1} depends on whether a _t as well as other agents which behave. Moment compensation ρ (s _{t, a t),} the difference in relative position / velocity / acceleration with respect to other cars, and our speed and desired speed, whether follow a desired route, our acceleration comfort whether such May depend on

[0468] One challenge in deciding which action the policy should perform at time t comes from having to estimate the long-term effect of this action on the reward. For example, in connection with the operation policy, operation performed at time t appears to be a good operation the current (i.e. reward value [rho (s _{t, a t)} is good) it may be, but an accident after 5 seconds (Ie, the reward value after 5 seconds becomes catastrophic). Therefore, when the agent is in state s, it is necessary to estimate the long-term quality of performing operation a. This is often referred to as the Q function, ie, Q (s, a) should reflect the long-term quality of performing operation a at time s. Given such a Q function, the natural choice of operation is to choose the highest quality, π (s) = argmax _a Q (s, a).

[0469] The immediate problem is how to determine Q and how to evaluate Q efficiently. s _t +1 _{is, (s} t, _{a t)} some deterministic function, i.e. _{s t + 1 = f (s} t, a t) is first subjected to simplification of (totally unrealistic) assumption that the I want to. If a person familiar with the Markov decision process (MDP), that it stronger than Markov assumption this assumption MDP (i.e. _(s t, is _s t +1 to _{a t)} as given are independent with past and Conditions ) Will be recognized. As mentioned in [5], even Markov assumptions are not sufficient for multi-agent scenarios such as driving, and therefore relax the assumptions later.

[0470] Under the simplifying this assumption, given a st, sequence determination of T step _{(a t, ..., a t} + T) the time for each t,. . . , The future of the state on the _{T (s t +1, ...,} s t + T +1) as well as the reward value can be accurately calculated. All these reward values, for example, their sum

, Can be summarized into a single number by defining Q (s, a) as follows:

  [0471] That is, Q (s, a) is me is in state s, the best future may wish when performing the operation a immediately.

[0472] How the Q can be calculated will be described. The first idea is that the set of possible actions A is a finite set

, And simply traversing all motion sequences in the discretized set. Then, a discrete operation sequence

The runtime is governed by the number of

Represents 10 lateral accelerations and 10 longitudinal accelerations, a possibility of 100 ^T is obtained, which is not feasible even at small T values. There are heuristics (eg, coarse-grained search) to speed up the search, but this brute force approach requires tremendous computational power.

  [0473] Parameter T is referred to as "period of Plan" often controls the natural trade-off between computation quality of time and evaluation, the present invention for the current operation as T is large While their evaluation is better (to clarify its effects further into the future), on the other hand, a large T increases the computation time exponentially. To understand why a large T value may be needed, consider the scenario where you are 200 meters before the highway exit and should exit from that exit. If the period of interest is long enough, the cumulative reward indicates whether the exit lane has been reached at some point τ between t and t + T. On the other hand, in a short time horizon, performing the right immediate action does not tell whether it will eventually lead itself to the exit lane.

[0474] Another approach is

Perform an offline calculation to build an approximation of Q as shown in, and during the online execution of the policy, without explicitly rolling out the future,

To use as an approximation to Q. One way to construct such an approximation is to discretize both the operating and state domains. These discretized pairs are

,

Indicated by Offline calculations are

Can be evaluated for the value of Q (s, a). Then all

About

To

Is determined to be Q (s, a). Furthermore, based on Bellman's pioneering research [2, 3], based on dynamic programming procedures (value iteration algorithm, etc.),

Q (s, a) can be calculated for each, and under our assumption, the total runtime is

It is about. The main problem with this approach is that in any reasonable approximation,

Is very large (due to the number of dimensions). In fact, the sensing state should represent six parameters for all other relevant vehicles in the scene: vertical position, horizontal position, speed and acceleration. Even if each dimension is discretized to only 10 values (very coarse discretization), there are 6 dimensions, so 10 ⁶ states are needed to describe a single car, and k To describe a car, a state of ^106k is required. this is,

Imposes impractical memory requirements for storing the value of Q for all (s, a) in

  [0475] One approach to address this number of dimensions of the problem, coming from the restricted function group such linear function or deep neural network covering characteristics determined manually (called hypothesis class often) To limit Q so that For example, consider a deep neural network that approximates Q in connection with playing in Atari games. This provides a resource efficient solution, provided that the family of functions approximating Q can be evaluated efficiently. However, there are some disadvantages of this approach. First, it is not known whether the selected family of functions includes a good approximation to the desired Q function. Second, even if such a function exists, it is not known whether existing algorithms can learn it efficiently. So far, there have been few success stories about learning Q-functions for complex multi-agent problems such as those encountered in driving. There are several theoretical reasons why this task is difficult. As stated with respect to the Markov assumption, there is a problem with the underlying existing methods. However, as explained below, a more serious problem is the very small signal-to-noise ratio due to the temporal resolution of the decision.

[0476] Consider a simple scenario where the vehicle needs to change lanes to get out of the highway exit with 200 meters remaining and there is currently no road. The best decision is to start changing lanes. The decision can be made every 0.1 seconds, so at the current time t, the best value of Q (s _t , a) should be for the action a, which corresponds to a slight lateral acceleration to the right. is there. Consider an operation a ′ corresponding to zero lateral acceleration. Here, changing the lane or changing after 0.1 second has only a very small difference, so the values of Q (s _t , a) and Q (s _t , a ′) are almost the same. . In other words, the advantage of choosing a in preference to a 'is very small. On the other hand, we use function approximation for Q, because of the noise in measuring the state s _t, the approximation of the present inventors with respect to the Q value is likely to involve noise. This causes a very low signal-to-noise ratio, which leads to very slow learning, especially with stochastic learning algorithms frequently used in approximation classes of neural networks. However, as noted, the problem is not specific to a particular function approximation class, but rather to the definition of the Q function.

  [0477] In summary, the available techniques can be broadly divided into two groups. The first group, by discretizing the or sensing state region searching over many operating sequence, a brute force approach which involves keeping the enormous table in memory. This approach can provide a very accurate approximation of Q, but requires infinite resources in terms of computation time or memory. The second group is a resource-efficient approach, which searches for a short sequence of actions or applies a functional approximation to Q. In each case, one is faced with obtaining a low-precision approximation of Q that can cause poor quality decisions.

  [0478] The approach described herein is to construct a Q-function that is resource efficient and accurate, that deviates from geometrical behavior, and that fits into the semantic behavioral space described in the next subsection. Including.

[0479] Semantic method
[0480] As a basis for the disclosed semantic approach, consider a young man who has just obtained a driver's license. The young man's father sits next to and gives the young man directions for a "driving policy". These instructions are not geometrical a ( "to 13.7 meters traveling at the current speed, then accelerated at a rate of 0.8 m / s ^2" does not take the form). Rather, these instructions are of a semantic nature ("follow the preceding car" or "pass quickly the car on the left"). Formalize the semantic language for such instructions and use it as a semantic working space. Next, a Q function over the semantic operation space is defined. Semantic behavior indicates that it can have a very long time span, which allows to estimate Q (s, a) without planning many future semantic behaviors. The total number of semantic operations is still small. This makes it possible to obtain an accurate estimate of the Q-function while still being resource efficient. In addition, as will be shown, learning techniques are combined to further improve the quality function without encountering small signal-to-noise ratios due to significant differences between various semantic operations.

[0481] Next, a semantic operation space is defined. The main idea is to determine the horizontal and vertical goals and the level of aggression to achieve those goals. The lateral target is a desired position in the lane's coordinate system (eg, "the target is to be in the center of lane number 2"). The vertical targets are of three types. The first is the relative position and speed with respect to other vehicles (eg, "the goal is to be behind vehicle number 3 at the same speed as vehicle number 3 and at a distance of 2 seconds from vehicle number 3"). . The second is a speed target (for example, "run at a speed obtained by multiplying the allowable speed of this road by 110%"). Third is a speed constraint at a particular location (eg, "0 speed at a stop line" when reaching an intersection or "up to 60 kmh at a particular location on a curve" when passing through a sharp curve. speed"). In a third option, a "velocity profile" (a number of discrete points on the route and the desired velocity at each of those points) can be applied instead. A reasonable number of lateral targets is bounded by 16 = 4 × 4 (four locations in up to four lanes of interest). A reasonable number of vertical targets of the first kind is bounded by 8 × 2 × 3 = 48 (eight related cars in front or behind, and three related distances). A reasonable number of absolute velocity targets is 10, and a reasonable upper bound on the number of velocity constraints is 2. Implementing a given lateral or longitudinal goal requires the addition of acceleration followed by deceleration (or vice versa). The willingness to achieve the goal is the maximum acceleration / deceleration (in absolute value) to achieve the goal. Using kinematic calculations with the goal and aggressiveness defined, a closed equation for implementing the goal is obtained. The only part left is to determine the combination between the horizontal and vertical goals (eg, "start with the horizontal goal and just start applying the vertical goal halfway"). That is. A set of 5 mixing times and 3 aggressive levels seems to be more than enough. Overall, its size was obtained semantic operation space is about 10 ^4.

  [0482] It is worth mentioning that the variable time required to satisfy these semantic actions is not the same as the frequency of the decision-making process. In order to be responsive to the dynamic world, our implementations should frequently make decisions every 100 ms. In contrast, each such determination is based on constructing a trajectory that satisfies some semantic motion with a much longer period of interest (eg, 10 seconds). Use a longer time horizon to help better evaluate short-term orbit prefixes. In the next subsection, we discuss the evaluation of semantic actions, but before that we argue that semantic actions cause sufficient search space.

  [0483] As discussed above, the semantic operating space is a subset of all possible geometric curves whose size is exponentially smaller (at T) than listing all possible geometric curves. cause. The first immediate question is whether this set of short term prefixes in the smaller search space contains all the geometric commands we want to use. This is actually sufficient in the following sense. If there are no other agents on the road, there is no reason to make any changes except to set a vertical goal and / or an absolute acceleration command and / or a speed constraint for a particular location. If the road includes other agents, it may be desirable to negotiate priority traffic with other agents. In this case, it is sufficient to set a vertical target for other agents. The exact implementation of these goals can change in the long term, but the short term prefix does not change much. Thus, a very good coverage of the relevant short-term geometric commands is obtained.

[0484] Construction of evaluation function for semantic behavior
[0485] defining a set of semantic operation indicated by A ^S. That the current is in the state s is a given, there is a need for a method to choose the best a ^S 0A ^S. To address this issue, we follow a similar approach to the alternative mechanism in [6]. The basic idea is to consider a ^S as a meta-action (or alternative). For each selection of meta operations, geometric trajectory representing the implementation of meta-operation _{^{a S (s 1, a 1}} ) ,. . . , (S _T , a _T ). To do it, it is necessary to know course other agents to our operations how react, now for some of the known deterministic function _{f s t + 1 = f (} s t, We continue to use the (unrealistic) assumption that a _t ) holds. As a good approximation of the quality of performing the semantic operation a ^S when in state s ¹ ,

Can be used.

  [0486] This approach can result in a strong driving policy. However, in some situations, a more sophisticated quality function may be needed. For example, suppose you have to go to an exit lane and are following a slower track before the exit lane. One semantic option is to keep running behind the truck slowly. Another semantic option is to overtake the truck in hopes of being able to return to the exit lane later and be on time. Since the quality measurement described above does not take into account what happens after overtaking the truck, choose the second semantic action even if there is enough time to overtake and return to the exit lane Never. Machine learning may facilitate building better assessments of semantic behavior that take into account more than immediate semantic behavior. As discussed above, learning the Q-function over immediate geometric operation is problematic due to the low signal-to-noise ratio (lack of advantage). This is not a problem when considering semantic operations, because there is a large difference between performing various semantic operations, and the semantic target period (how many ) Is very small (possibly less than three in many cases).

  [0487] Another potential benefit of applying machine learning is for generalization, that is, a full assessment of all roads by manual investigation of road characteristics that may involve some trial and error. it is possible to set the function. But is it possible to automatically generalize all roads? Here, in order to generalize an unknown road, the machine learning method discussed above can be trained based on a wide variety of road types. A semantic operation space according to the disclosed embodiments may allow for potential benefits, i.e., semantic operations include information about long periods of interest, and thus their quality while remaining resource efficient. Very accurate evaluation can be obtained.

[0488] Dynamics of other agents
[0489] So far, s t _{+ 1} has been using the assumption that it is deterministic function of s _t and a _t. This assumption is not entirely realistic, as we emphasized earlier, because our actions affect the behavior of other road users. Take into account some reactions of other agents to their actions (for example, assuming a safe interruption, the car behind you adjusts its speed so that it does not hit you from behind) It is not realistic to assume that we model all of the dynamics of other agents. The solution to this problem is to reapply decision making at a high frequency, thereby constantly adapting our policies to parts of the environment that go beyond our modeling. In a sense, this can be thought of as a Markovization of the world at every step.

[0490] Detection
[0491] This section describes the detection state, which is a description of scene-related information and forms the input to the driving policy module. In general, the sensing states include static and dynamic objects. A static object is information about lanes, physical boundaries of roads, restrictions on speed, restrictions on priority right of way, and shields (eg, fences that block related parts of merging roads). Dynamic objects include vehicles (eg, bounding box, speed, acceleration), pedestrians (bounding box, speed, acceleration), traffic lights, dynamic road divisions (eg, construction area cones), temporary traffic signs And police action, as well as other obstacles on the road (eg, animals, mattresses dropped from trucks, etc.).

  [0492] It is impossible to expect to obtain an accurate detection state s with any appropriate sensor setting. Instead, there are sensing systems that consider the raw sensor and mapping data, denoted x0X, and take x to create the appropriate sensing state.

Definition 16 (Detection System) [0493] Let S denote the area in the detection state, and X the area of the raw sensor and mapping data. Detection system function

It is.

[0494]

It is important to understand at what point should be accepted as a reasonable approximation to s. The ultimate way to answer this question is generally by examining the implications of this approximation for the performance of our driving policy, specifically for safety. In accordance with our safety-comfort distinction, we again distinguish between detection errors that lead to dangerous behavior and detection errors that affect aspects of riding comfort. Before giving details, the types of errors that the detection system can make may include:
• False negative: The detection system misses the object. • False positive: The detection system indicates the object is "not really". Inaccurate measurement: The detection system correctly detects the object, but incorrectly identifies its position or speed. Estimate Inaccurate semantics: The sensing system correctly detects the object, but misinterprets its semantic meaning, eg, traffic light color

[0495] Comfort
[0496] Recall that for a semantic operation a, we used Q (s, a) to indicate the evaluation of a, given that the current sensing state is s. Our policy chooses the action π (s) = argmax _a Q (s, a). Instead of s,

, The semantic behavior selected is

become.

Holds,

Should be accepted as a good approximation to s. However, for the true state s

Is almost optimal for some parameter ε, ie,

As long as

There is no problem even if you choose. in this case,

Are εaccurate with respect to Q, we say. Inevitably, it cannot be expected that the sensing system will always be εaccurate. Thus, it is also recognized that the sensing system fails with some small probability δ. In this case (Valiant's PAC learning terms)

We say that is Probably (wp of at least 1-δ), Approximately (up to ε), Correct, or PAC for short.

[0497] Several (ε, δ) pairs can be used to evaluate various aspects of the system. For example, three thresholds ε ₁ <ε ₂ <ε ₃ can be selected to represent a mild mistake, a medium mistake, and a decisive mistake, and different values of δ can be set for each of them. This results in the following definition:

[0498] Definition 17 (PAC Detection System) (accuracy, reliability) of the set of pairs of _{((ε 1, δ 1)} , ..., (ε k, δ k)) and the detection state area is S , X the raw sensor and mapping data area and D the distribution over X × S. Assume that the operation space is A, the quality function is Q: S × A → |, and π: S → A holds that π (s) ∈argmax _a Q (s, a). For all i∈ ｛1,. . . , K},

Detection system

Is probabilistic and approximately correct for the above parameters (PAC).

Here, the definition depends on the distribution D over X × S. It is important to emphasize that this distribution is constructed not by following our specific policy of autonomous vehicles, but by recording data of many human drivers. The former seems more appropriate, but the latter requires online verification which makes the development of a detection system impractical. Since the impact of any reasonable policy on D is small, a sufficient distribution can be built by applying simple data augmentation techniques, and then off-line verification can be performed on every major update of the detection system. This definition is

Provides sufficient but not necessary conditions for a comfortable ride. It is not necessary because it ignores the important fact that short-term incorrect decisions have little effect on ride comfort. For example, suppose there is a vehicle 100 meters away and that vehicle is slower than the host vehicle. The best decision here is to start accelerating weakly. If the detection system misses the vehicle but detects it the next time (after 100 milliseconds), the difference between the two rides is not significant. To simplify the presentation, we ignored this problem and called for stronger conditions. Adapting to the multi-frame PAC definition is conceptually simple, but more technical.

  Next, a design principle obtained from the above PAC definition will be derived. Recall that we described some types of detection errors. For false negative, false positive, and incorrect semantic types of errors, the error is for an unrelated object (eg, a left-turn signal when traveling straight ahead) or is captured by the δ portion of the definition. Or either. Therefore, frequently focus on the nature of the error of "inaccurate measurement" to occur.

  [0501] Somewhat surprisingly, a common approach to measuring the accuracy of a sensing system with self-accuracy (ie, by measuring the accuracy of the position of all objects relative to the host vehicle) guarantees a PAC sensing system. To indicate that it is not enough. Then, another approach to assuring a PAC detection system is proposed and how to obtain it efficiently is shown. Let's start with some additional definitions.

[0502] For each object о in the scene, p (о),

Is the position of о in the coordinate system of the host vehicle by s,

And Note that the distance between о and the host vehicle is || p ||.

Is the additional error

It is. о and the distance between the host vehicle

Is the additional error divided by || p (о) ||

It is.

  [0503] It is not practical to require small additional errors for distant objects. In fact, assume that о is a vehicle at a distance of 150 meters from the host vehicle and ε is of medium size, for example ε = 0.1. To obtain additional accuracy, this means that the position of the vehicle should be known to an accuracy of 10 cm. This is impractical for affordable sensors. On the other hand, it is necessary to estimate up to 10% equivalent position on the accuracy of 15m in relative accuracy. This is feasible (as described below).

[0504] For all о∈О, p (о) and

If the (relative) error between is up to ε, the detection system

Locates the set of objects О in an εego-accurate manner. The following example demonstrates that εego-accurate detection conditions for all reasonable Qs do not guarantee a PAC detection system. In fact, consider a scenario in which the host vehicle is traveling at a speed of 30 m / s, and there is a stopped vehicle 150 meters away. If the vehicle is in its own lane and there is no option to change lanes without delay, it must start decelerating immediately at a rate of at least 3 m / s ² (otherwise the stop will not be in time or later You have to slow down harder). On the other hand, if the vehicle is on the shoulder, there is no need to apply strong deceleration. p (о) is one of these cases, while

Is another case, assuming that there is a 5 meter difference between these two locations. Therefore,

Is as follows.

  That is, the detection system may be εego-accurate for rather small ε values (errors less than 3.5%), and it is necessary to apply a strong brake on any reasonable Q function The value of Q is completely different because a situation is confused with a situation where it is not necessary to apply the brakes hard.

[0506] The above example shows that ego-accuracy does not guarantee that our detection system is a PAC. Q depends on whether the PAC sensing system has enough other properties. A group of Q functions having a simple positioning characteristic that guarantees a PAC detection system will be described. The problem with εego-accuracy is that it can lead to semantic errors, and in the example above,

Failed to assign the vehicle to the correct lane even when ε <3.5% and εego-accurate. In order to solve this problem, a semantic unit related to the horizontal position is used.

Definition 18 (meaning unit) The center of the lane is a simple natural curve, that is, the differentiable injection mapping l: [a, b] → | ³ , and all a ≦ t ₁ <t ₂ ≦ About b, length

Is equal to t ₂ −t ₁ . The width of the lane is a function w: [a, b] → | ₊ . Point x∈ onto curve ^| projection ^3, the closest point on the curve x, i.e. _{t x = argmin t∈ [a,} b] || l (t) the point about -x || l _{(t x} ). Semantic longitudinal position of the x relative to the lane, is _{t x,} semantic lateral position of x with respect to the lane _is l (t x) / w ( t x). Define the semantic velocity and acceleration as the first and second derivatives above.

[0508] Similar to geometric units, using relative error for semantic longitudinal distance, while the true distance is p (о),

About some object

If we cause the semantic longitudinal distance of

(The maximum in the denominator is the case where the objects have approximately the same vertical distance (eg, addressing a neighboring car on another lane). This can lead to the following definitions:

Definition 19 (Error in Semantic Unit) It is assumed that the lane is 1 and the semantic vertical distance of the host vehicle to the lane is 0. A point

And the semantic horizontal and vertical distances to that point relative to the lane are p _lat (x), p _lon (x). Approximate measurements

,

And for x

The distance between p and p is defined as

  [0510] The distance between the lateral speed and the vertical speed is determined in the same manner. With the above definitions, we are ready to define the characteristics of the Q for the PAC sensing system and the corresponding sufficient conditions.

[0511] Definition 20 (Semantically-Lipschitz Q) All a, s,

about,

Holds, the Q function is L-semantically-Lipschitz,

, P is s with respect to the object о,

Is a measurement caused by

  [0512] As a direct system, the following is obtained.

[0513] Lemma 8 Q is L-semantically-Lipschitz and the detection system

Is at least 1-δ probability,

If you generate a semantic measure so that

Is a PAC sensing system with parameters 0, δ.

[0514] Safety
[0515] This section discusses the possibility of detecting errors that can cause unwanted behavior. As mentioned earlier, the policy is secure in a verifiable manner, in the sense that it does not cause an accident in the host AV. Such accidents can still be caused by hardware failures (eg, failure of all sensors or punctures on the highway), software failures (significant bugs in some of the modules), or misdetections. . Our goal is that the probability of such an event be very small (10 ⁻⁹ per hour of such an accident). The average number of hours (2016) spent by drivers in the United States on the road to assess the value of this value is less than 300 hours. So, hopefully, one must live 3.3 million years to encounter an accident arising from one of these types of events.

[0516] First of all, what is the detection error related to safety? Recall that our policy selects _a value of a that maximizes Q (s, a), i.e., π (s) = argmax _a Q (s, a), step by step. Ensure security by setting Q (s, a) =-＝ for all inconservative actions a (see definition 14). Thus, a first type of safety critical detection error is when our detection system leads to choosing a dangerous operation. Formally,

Decisions by

age,

Holds,

Can cause serious safety mistakes. The second type of safety critical detection mistake is that all actions

In accordance with s, a standard emergency policy (e.g. hard braking) must be applied, while according to s there is a safe operation, ie max _a Q (s, a)>-∞ is there. This is dangerous if our speed is fast and there is a car behind. Such mistakes are called ghosts that are serious for safety.

[0517] Safety-critical mistakes are usually caused by false negatives, while safety-critical ghosts are caused by false positives. While such mistakes can be caused by significantly inaccurate measurements, in most cases our comfort goals are far from the boundaries of the definition of safety. Therefore, it is unlikely that a modest measurement error will cause a serious safety error. How can we guarantee that the probability of a safety-critical mistake is very small, for example less than 10 ^-9 per hour? From Lemma 1, it is necessary to examine the system based on the long operating than 10 ⁹ hours without performing further assumptions. This is unrealistic (or at least extremely difficult), which is equivalent to recording 3.3 million car operations over a year. Furthermore, building a system that achieves such high accuracy is a major challenge. The solution to both system design and verification challenges is to utilize several subsystems, each of which is independently designed, relies on different technologies, and increases individual accuracy These systems are merged in a way that ensures that

_{[0518] s 1, s 2} , s 3 3 sub system shown in (extension to four or more is easy) assumed to build. Each subsystem should receive a and output safety / danger. Operations that the majority of subsystems (two in this case) accept as safe are considered safe. If no operation is deemed safe by at least two subsystems, a default emergency policy applies. The performance of this fusion scheme is analyzed based on the following definitions as follows.

[0519] Definition 21 (One side c-approximate independent)

Holds, the two Bernoulli random variables r ₁ and r ₂ are called one side c-approximate independent.

[0520] For i0 {1,2,3},

,

Indicates a Bernoulli random variable indicating whether the subsystem i makes a security-critical error / ghost. ^{Similarly, e m,} e ^g shows the safety serious mistake / ghost of the fusion system. Random variable for any pair i ≠ j

,

Is one sided c-approximate independent and the same is

,

We make use of the assumption that holds true. Before explaining why this assumption is reasonable, let me first analyze its implications. probability of e ^m is,

Can be bounded by

[0521] Therefore, all subsystems

If you have

Holds. The exact same differentiation applies to safety-critical ghost mistakes. Therefore, by applying the binding limit, we conclude as follows.

[0522] Corollary 2 For any pair i ≠ j, random variables

,

Is one sided c-approximate independent and the same is

,

Suppose that also applies. Further, for all i,

as well as

Is assumed to hold. Therefore, the following equation is established.

[0523] This system allows the use of significantly smaller datasets to validate the detection system. For example, if one wants to achieve a ^10-9 security-critical error probability, instead of taking about 10 ⁹ examples, take about 10 ⁵ examples and test each system separately.

  [0524] There may be sensor pairs that result in uncorrelated errors. For example, in contrast to cameras that are affected by bad weather conditions but are less likely to be affected by metal objects, radar works well in bad weather conditions but is caused by irrelevant metal objects. May not work. At first glance, the camera and lidar have a common source of error, for example, both are affected by foggy weather, heavy rain or snow. However, the type of error differs between a camera and a lidar, the camera may miss an object due to bad weather, and the lidar may detect a ghost by reflection from particles in the air. Because of the distinction between the two types of errors, the approximation independence is likely to continue to apply.

  [0525] Our definition of a safety-critical ghost requires that at least two sensors make every operation dangerous. It is unlikely to occur even in difficult conditions (eg dense fog). The reason is that in such situations, systems (e.g., lidars) that are subject to difficult conditions can declare high speed and lateral operation to be a dangerous operation, which is very self-defense. This is because the operation is commanded. As a result, the host AV travels very slowly, in which case, even if an emergency stop is required, it is not dangerous because the travel is slow. Thus, our definition results in a driving style adapted to road conditions.

[0526] Building a scalable detection system
[0527] The requirements of the sensing system in terms of both comfort and safety have been described. Next, a method for constructing a detection system that is scalable and satisfies those requirements will be described. There are three main components of the detection system. The first component is a 360-degree coverage of the camera-based scene long distance. The three main advantages of the camera are (1) high resolution, (2) texture, and (3) price. Lower prices allow for scalable systems. Textures make it possible to understand the semantics of the scene, including lane marks, traffic lights, pedestrian intentions, and more. High resolution allows long distance detection. Further, detecting lane marks and objects in the same area allows for excellent semantic lateral accuracy. Two major disadvantages of the camera are (1) the information is 2D and the depth distance is difficult to estimate, and (2) sensitivity to lighting conditions (low sun or bad weather). The following two components of the system are used to overcome these difficulties.

  [0528] The second component of the system, to determine the precise location along the target trajectory based on the (e.g., in an image) the position of the recognized landmarks identified in the environment (host vehicle A semantic high-definition mapping technique called REM (Road Experience Management), which includes navigation based on target trajectories that are predefined and stored for road segments along with capabilities. A common geometric approach to creating a map is to record the 3D point cloud (obtained by the lidar) in the map creation process, and then match existing lidar points to points in the map. Gives the location on the map. There are several disadvantages to this approach. First, this approach requires a large amount of memory per kilometer of mapping data because many points need to be stored. This forces expensive communication infrastructure. Second, the map is updated very infrequently because not all vehicles may have lidar sensors. Road changes (construction areas and hazards) can occur, and the “time to reflect reality” of lidar-based mapping solutions is long. In contrast, REM is, follow a method based on semantics. The concept is to utilize a large number of vehicles with cameras and software to detect semantically significant objects (lane marks, curves, poles, traffic lights, etc.) in the scene. Today, many new vehicles are equipped with an ADAS system that can be used to create maps using crowd sources. Since the processing is performed on the vehicle side, only a small amount of semantic data should be transmitted to the cloud. This allows the map to be updated very frequently in a scalable way. In addition, autonomous vehicles can receive small size mapping data on existing communication platforms (cellular networks). Finally, a very accurate localization on the map can be obtained based on the camera without the need for expensive riders.

  [0529] REM can be used for several purposes. First of all, REM is, (it is possible to make a plan on how to exit the highway in advance) to give the tip of the forward-looking statements about the static nature of the road. Second, REM provides all other accurate sources of static information, which, together with camera detection, result in a robust view of static parts of the world. Third, REM solves the problem of lifting image plane 2D information into a 3D world as follows. The map describes all of the lanes as curves in the 3D world. Locating the vehicle on the map allows all objects on the road to be lifted from the image plane to its 3D position trivially. This results in a positioning system that follows the precision in semantic units. A third component of the system can be a complementary radar and lidar system. These systems can serve two purposes. First, these systems can provide a very high level of accuracy to augment safety. Second, these systems can provide direct measurements of speed and distance, which further improves ride comfort.

  [0530] The following sections include technical lemmas for RSS systems and some implementation considerations.

Lemma 9: For all x0 [0, 0.1], 1−x> e− ^2x applies.
Proof. Let f (x) = 1−x−e− ^2x . Our goal is to show that ≧ 0 for x0 [0,0.1]. Note that f (0) = 0, so it is sufficient to obtain f (x) ≧ 0 within the above range. Clearly, f ′ (x) = − 1 + 2e− ^2x holds. Obviously, it is sufficient to verify that f '(0) = 1 and monotonically decreasing, so that f'(0.1)> 0, which can be numerically easily done. (F ′ (0.1) ≒ 0.637).

[0532] Efficient verification of caution-hidden objects
[0533] As with the non-hidden object, it can be checked whether giving the current command and then giving DEP is RSS. To do so, assuming that the exposure time is 1, _{unroll the} future until t _break , and then command a DEP that is by definition sufficient for discretion. When assuming the worst of cases related to hidden object, find out whether the accident of all of t'0 [0, t _brake] for Yes negligence can occur. Use some of the worst case operation and safe distance rules. Find points of interest using a shield-based approach, i.e. calculate the worst case for each shield. This is a decisive efficiency-driven approach, for example, a pedestrian can hide in many positions behind the car and perform many operations, but only a single worst case location and There are operations that pedestrians can perform.

[0534] Next, consider a more complicated case of a hidden pedestrian. Consider the shielded area behind the parked vehicle. The closest point in the occluded area and the front / side of our car c can be found, for example, by their geometric properties (triangles, rectangles). Formally, the occluded region can be viewed as a combination of a small number of convex regions of simple shape, each of which can be treated separately. In addition, pedestrians may be able to hit the front of the car from the occluded area (subject to v _limit ) if and only if they can hit the front of the car using the shortest possible path I understand. Use that the maximum distance a pedestrian can travel is v _limit · t ′ to get a simple confirmation of the possibility of hitting the front. It is pointed out that we are responsible for side impacts if and only if our lateral velocity exceeds μ in the impact direction and if the path is shorter than vlimit · t ′. Disclosed is an algorithm for verifying caution against hidden pedestrians, shown here in free pseudo code. The crucial part of determining whether there is a potential accident of negligence with a pedestrian hiding by a vehicle is done in a simple manner as described above.

[0535] Simulator verification problem
[0536] As discussed above, the security of multi-agents can be difficult to verify statistically, because such verification should be done in an "on-line" manner. By constructing a driving environment simulator, some people may claim that their driving policy can be verified in the "lab". However, verifying that a simulator faithfully represents reality is as difficult as verifying the policy itself. To understand why this is the case, applying the driving policy π to the simulator,

The probability of an accident in the real world is p,

Assume that the simulator has been verified in the sense that (0 must be less than ^10-9 ). Next, the operation policy is replaced to be π ′. Suppose that with a probability of 10 ⁻⁸ π ′ confuses a human driver and performs strange actions leading to an accident. This odd behavior may not have been modeled in the simulator (rather, it is more likely), consistent with the simulator's great capabilities in estimating the performance of the original policy π. . This proves that even if the simulator indicates that it reflects the reality of the driving policy π, it does not guarantee that it reflects the reality of another driving policy.

[0537] Lane based coordinate system
[0538] One simplification of assumptions that can be made within the definition of RSS is that the road is made up of adjacent straight lanes of constant width. The distinction between the horizontal and vertical axes and the ordering of the vertical positions can play an important role in RSS. Furthermore, the definition of those directions is explicitly based on the shape of the lane. Converting from a (global) location on a plane to a lane-based coordinate system reduces the problem to the original "constant width straight lane" case.

[0539] The center of the lane is a smooth directed curve r on a plane, and r ⁽¹⁾ ,. . . , R ^(k) are assumed to be all straight lines or arcs. Note that the smoothness of the curve implies that no pair of consecutive fragments can be linear. Formally, the curve maps the “longitudinal” parameter Y 0 [Y _min , Y _max ] ⊂ | in a plane, ie, the curve is a function of the form r: [Y _min , Y _max ] → | ² is there. Define a continuous lane width function w: [Y _min , Y _max ] → | ₊ that maps the vertical position Y to a positive lane width value. For each Y, a unit normal vector to the curve at the position Y indicated by r┴ (Y) can be determined from the smoothness of r. A subset of points on a plane in a lane is naturally defined as follows:
R = {r (Y) + αw (Y) r┴ (Y) | Y0 [Y min, Y max], α0 [± 1/2]}

[0540] Informally, our goal is to construct a transformation φ of R such that for two cars on the lane, their "logical ordering" is maintained, that is, on the curve. In the case where _cr is “behind” c _f , φ (c _r ) _y <φ (c _f ) _y holds. c _l is _{φ (c l) x <φ} (c r) x is satisfied in the case of the "left" of _{c r} on the curve. Similar to RSS, the y-axis is associated with the “vertical” axis and the x-axis is associated with the “horizontal” axis.

To determine φ, use the assumption that for all i, if r ⁽ⁱ⁾ is an arc of radius ρ, then the width of the lane through r ⁽ⁱ⁾ is ≦ ρ / 2. . Note that this assumption applies to any realistic road. This assumption assumes that for all (x ', y') 0R, a unique pair Y such that (x ', y') = r (Y ') + α'w (Y') r┴ (Y ') It clearly implies that '0 [Y _min , Y _max ], α'0 [± 1/2] exists. Then, φ: R → | ² can be defined such that φ (x ′, y ′) = (Y ′, α ′), and (Y ′, α ′) is (x ′, y ′). ) = R (Y ′) + α′w (Y ′) r┴ (Y ′).

[0542] This definition captures the concept of "lateral operation" in the lane coordinate system. For example, consider a widened lane where a car travels just above one of the lane boundaries (see FIG. 23). The spread of lane 2301 means that vehicle 2303 moves away from the center of the lane and thus has a lateral velocity relative to the lane. However, this does not mean that the vehicle 2303 performs a lateral operation. The definition of φ (x ′, y ′) _x = α ′, that is, the lateral distance to the center of the lane in w (Y ′) is the fixed lateral position where the lane boundary is ± １ ／. Implies having. Therefore, a car that does not leave one of the lane boundaries is not considered to make any lateral movement. Finally, it can be seen that φ is homomorphic. The term lane-based coordinate system is used when discussing φ (R) = [Y _min , Y _max ] x [± １ ／]. In this way, a reduction from the general lane geometry to a linear ordinate / abscissa system has been obtained.

[0543] Extending RSS to general road structure
[0544] This section describes a complete definition of RSS that applies to all road structures. This section deals with the definition of RSS, rather than how efficiently assuring that the policy complies with RSS. To capture any situation where there are multiple lane geometries, such as intersections, the concept of route priority is introduced next.

  [0545] The second generalization deals with a two-way road where there may be two cars traveling in opposite directions. In this case, the already established definition of RSS remains valid with minor generalization of "safe distance" to oncoming traffic. Managed intersections (using traffic lights to direct traffic flow) can be fully addressed by route priority and the concept of two-way roads. Unstructured roads (eg, parking lots) that do not have a clear definition of a route may also be addressed with RSS. RSS is still valid in this case, and the only modifications required are methods for defining virtual routes and for assigning each car to (possibly several) routes.

[0546] Route priority
[0547] To address the scenario where multiple different road geometries overlapping within a particular area are in one scene, the concept of route priority will now be introduced. The examples shown in FIGS. 24A-D include roundabouts, intersections, and merging to highways. A method has been described for converting general lane geometry to lane-based, with consistent meanings for the vertical and horizontal axes. Next, a scenario in which there are multiple routes with different road geometries is addressed. As a result, when two vehicles reach the overlapping area, both vehicles will interrupt the front corridor of the other vehicle. This phenomenon cannot occur if the two routes have the same geometry (as in the case of two adjacent highway lanes). Broadly speaking, the principle of route priority is that if routes r ₁ , r ₂ overlap and r ₁ has precedence over r ₂ , vehicles coming from r ₁ entering the corridor ahead of vehicles coming from r ₂ will be interrupted Is not considered to do.

[0548] To formally explain this concept, the fact that accidental negligence relies on geometric properties derived from the lane coordinate system and on worst-case assumptions that also depend on the lane coordinate system I want to be recalled. The routes that determine the structure of the road are r ₁ ,. . . , And _{r k.} As a simple example, consider the merge scenario shown in FIG. 24A. Assume that two cars 2401 (c ₁ ) and 2402 (c ₂ ) are traveling on routes r ₁ and r ₂ , respectively, and that r ₁ is the preferred route. For example, r ₁ is a lane of the highway, it is assumed that r ₂ is a merging lane. Since the coordinate system based on the route is determined for each route, the first consideration is that any operation can be considered within the coordinate system of any route. For example, using the coordinate system of r _2, it travels in a straight line towards the r ₁ seems to confluence to the left of r _2. One approach to the definition of RSS is if and only if the ∀i0 {1,2}, if the operation is safe for r _i, may be to each car can be performed the operation. However, this implies that the preferred route c ₁ traveling on it should be very conservative with respect to r ₂ is the root that performs merging, it is possible to c ₂ to accurately travel on the route Because it can, and therefore, win in the lateral position. In this case, it is unnatural because cars on the highway have priority right of way. To overcome this problem, certain areas defining the priority of the route are defined, and only a part of the route is considered to be security relevant.

Definition 22 (Accident Responsibility for Route Priority) Suppose that r ₁ , r ₂ are two overlapping routes with different geometric shapes. longitudinal spacing of the coordinate system of _{r 1 [b, e] r} 1 (Figure 25) uses _{r 1> [b, e]} r 2 to indicate its preference to _{r 2.} Assume that an accident has occurred between vehicles c ₁ and c ₂ running on routes r ₁ and r ₂ . In i0 {1, 2}, when considering the coordinate system _{r i,} the vehicle is at fault for the accident shall indicate the _{b i} ⊂ {1,2}. The accidental negligence is as follows:
R ₁ > _{[b, e] If} one of the cars was within the interval [b, e] on the vertical axis of the system of r ₁ at the time of negligence for r ₂ and r ₁ , the negligence is due to b ₁ .
• Otherwise, the fault is due to b ₁ ∪b2.

[0550] To illustrate this definition, consider again the example of joining a highway. The lines indicated by “b” and “e” in FIG. 25 indicate the values of b and e satisfying r ₁ > _{[b, e]} r ₂ . Thus, it allows the vehicle to travel naturally on the highway, implying that the merging vehicle must be safe for vehicles traveling on the highway. Specifically, if the vehicle 2401 c ₁ travels on a central preferential lanes without lateral velocity, vehicle 2401 c _1, for accident cars 2402 c ₂ that travels non-priority lane, 2402 c ₂ safe Note that there is no fault unless you break into the 2401 c ₁ corridor at a reasonable distance. Note that the end result is very similar to a regular RSS (this is exactly the case where a car on a straight road tries to make a lane change). Note that there may be instances where the route used by another agent is unknown. For example, in FIG. 26, the car 2601 may not be able to determine whether the car 2602 takes the route “a” or the route “b”. In this case, an RSS can be obtained by repeatedly examining all possibilities.

[0551] Two-way traffic
[0552] To address bidirectional traffic, amendments to the definition of negligence have accomplished to clarify parts that depend on the rear / front relationship, because in such cases they have slightly different meanings. It is because it is. Consider two vehicles c ₁ , c ₂ traveling on a straight two-lane road in the opposite vertical direction, that is, v _{1, long} · v _{2, and long} <0. The driving direction for a lane, such as a car going off the oncoming lane to overtake a parked truck or backing into a parking lot, may be negative in a reasonable urban scenario. Therefore, it is necessary to extend the definition of the safe vertical distance introduced in the case where a negative vertical speed is assumed to be unrealistic. If c _r before impinging on c _f gives the maximum braking by c _f a sufficient reaction time to brake, the distance between the c _r and c _f Recall that is safe. In our case, the "worst case" of an oncoming vehicle was revisited in a slightly different way, and of course, the "worst case" is that the oncoming vehicle accelerates towards itself. Without thinking, it is considered to apply the brakes actually to avoid the collision, but to use only some moderate braking force. To capture the difference in responsibility of the cars, if one of those cars is driving in the opposite direction, we start by defining the "right" driving direction.

  [0553] In the definition of the RSS regarding the parallel lane, the relevant lane is defined such that the center is closest to the interrupt position. One can then reduce himself to considering this lane (or, if symmetric, address the two lanes separately as in definition 22). In the following definitions, the term "heading" refers to the arctangent (in radians) of lateral velocity divided by longitudinal velocity.

Definition 23 ((μ ₁ , μ ₂ , μ ₃ ) -victory due to correct driving direction) c ₁ , c ₂ are traveling in opposite directions, that is, v _{1, long} · v _{2, long} <0. Assume that it holds. Their lateral position and heading for lanes x _i, and h _i. If all the following conditions are true, the _{c 1} is correct operating direction _{(μ 1, μ 2, μ} 3) - said to win:
・ | H ₁ | <μ ₁ ,
・ | H ₂ −π | <μ ₂ ,
* | X ₁ | <μ ₃ .

[0555] An indicator of this event is denoted by W _CDD (i).

[0556] In words, c ₁ is wins if c ₂ travels near the center of the lane in the right direction while taking the opposite direction. At most one car can win, and none of the cars may win. Intuitively, assume that there is a conflict in the situation discussed. It is reasonable to impose a number of responsibilities by car c ₁ lose the correct driving direction. If the car wins in the correct driving direction, this is done by redefining a _{max, break} .

Definition 24 (Reasonable Braking Force) α _{max, brake, wcdd} > 0 is a constant less than a _{max, brake} . Assume that c ₁ and c ₂ are traveling in opposite directions. Indicated by RBP _i, is a reasonable braking forces of each car _{c i,} with _{c i} is correct operating direction _{(μ 1, μ 2, μ} 3) - a max in the case of _{winning, brake,} a _Wcdd, or else Otherwise _, it is a _{max, break} .

_{A max} when [0558] that does not correct operation Win a direction / _{win, brake, Wcdd, a max,} the exact value of the _brake is a constant to be defined, depending on the type of road and lanes each vehicle is traveling I can do it. For example, on narrow streets in a city, winning in the correct driving direction may not imply much lower braking values, and in congested traffic the car may or may not be clearly off your lane. Expect to use the same force to brake. However, consider the example of a rural two-way road where high speeds are allowed. When deviating to an oncoming lane, it is not anticipated that vehicles traveling in the correct direction will apply very strong braking force to avoid collisions with the host vehicle, and the responsibility of the host vehicle is greater than those vehicles. In the case where two cars are on the same lane with one backing to the parking lot, different constants can be defined.

  [0559] The safe distance between vehicles traveling in opposite directions, and the immediate derivation of its exact value, is now defined.

[0560] Definition 25 (safe longitudinal distance - bidirectional traffic) and traveling in the opposite direction, longitudinal distance between both the car c ₁ and another car c ₂ within each other forward corridor For any acceleration command a, | a | <a _{max, accel} performed by c ₁ , c ₂ up to the reaction time ρ, if c ₁ and c ₂ apply their reasonable braking force from time ρ to complete stop, It is safe for time ρ and the car does not collide.

The lemma 10 c ₁ , c ₂ is the same as in definition 25. Let RBP _i , a _{max, accel} be the valid brake and acceleration commands (per i) and let the reaction time of the car be ρ. The vertical speeds of the vehicle are v ₁ and v _2, and their lengths are l ₁ and l ₂ . determine + ρ _{· a max,} and _{accel | v i, ρ, max} = | v i. L ₌ a _{(l r + l f) /} 2. Therefore, the minimum safe longitudinal distance is:

  [0562] It can be seen that the term in this sum is the maximum distance that each vehicle travels until it stops completely when the operation of definition 25 is executed. Thus, for a complete stop to be at a distance above L, the initial distance must be greater than this sum and the additional terms of L.

  [0563] To define the negligence / accident liability of a two-way traffic scenario, we use the same RSS negligence definition for the dangerous longitudinal distance defined in definition 25.

[0564] Definition 26 (negligence in bidirectional traffic) negligence in bidirectional traffic accidents between car c ₁ and c ₂ is traveling in the opposite direction is a function of the state of negligence time, defined as follows Is:
If the time of fault is also the time of interruption, fault is defined as in the normal RSS definition.
- Otherwise, for all i, if c _i at some t occurring after fault point is not braked with a force of at least RBP _i, negligence is in c _i.

[0565] For example, assume a safe interrupt that occurs before the time of negligence. For example, out c ₁ is the opposite lane, and performs the interrupt at a safe distance in the corridor of c _2. c ₂ is won in the correct driving direction, therefore, this distance is quite may at long (not expect that c ₂ exerts a strong braking force, expect only reasonable braking force). Thus, both vehicles have the responsibility to avoid collisions with each other. However, if the interrupt was not a safe distance, if c ₂ has traveled a central own lane without lateral movement, using the normal definition indicating that c ₂ is no fault. Negligence, there is only to _{c 1.} This allows cars to travel naturally in the center of their lanes without having to worry about traffic that may be dangerously out of their zone. On the other hand, a safe departure to the oncoming lane, a common operation required in congested traffic in cities, is allowed. Consider an example of a car that initiates a parking operation with a back, and the car should start the back while ensuring that the distance to the car behind it is safe.

[0566] Traffic light
[0567] In a scenario where a traffic light includes an intersection, the simple rule of the traffic light is that if one car route is a green traffic light and the other car route is a red traffic light, Some people may consider this to be "negligence". However, this is not a correct rule, especially in all cases. For example, consider the scenario shown in FIG. Even if the car 2701 is on the green light route, it is not expected that the car 2701 will ignore the car 2703 already in the intersection. The correct rule is that the green light route has priority over the red light route. Thus, a clear reduction from the traffic light is obtained over the concept of route priority described above.

[0568] Unstructured road
Next, considering a road on which a clear geometric shape of a route cannot be defined, a scenario without a lane structure (for example, a parking lot) is first considered. A way to ensure that there are no accidents requires that if all cars are traveling in a straight line, but a change in heading occurs, such changes need to be made when no cars are close to you. It can be. The rationale behind this method is that cars can predict what other cars will do and behave accordingly. If another vehicle deviates from this prediction (by changing heading), the operation will be performed with a sufficiently long distance, and thus may have sufficient time to correct the prediction. With lane structures, smarter predictions of what other vehicles will do may be possible. If there is no lane structure, the car will follow its current heading. Strictly speaking, this is equivalent to assigning a virtual straight route to all vehicles according to their heading. Next, consider a scenario at a large unstructured circular intersection (for example, around the Arc de Triomphe in Paris). Here, one sensible prediction is to assume that the car will follow the geometry of the roundabout while keeping its offset. Strictly speaking, this is equivalent to assigning all cars a virtual arc path according to their current offset from the center of the roundabout.

  [0570] In order to provide a navigation system that takes into account potential accident liability when determining the particular navigation instruction to be implemented, the above driving policy system (eg, RL system) has been described in one of the accident liability rules described. One or more can be implemented. Such rules can be applied during the planning phase, for example in a set of programmed instructions or in a trained model, so that the proposed navigation behavior is developed by a system that already complies with the rules. For example, the driving policy module may, for example, be responsible for or trained on one or more navigation rules on which the RSS is based. Additionally or alternatively, RSS safety constraints as a filter layer that tests all proposed navigation actions proposed by the planning phase against relevant accident liability rules to ensure that the proposed navigation actions are compliant. Can be applied. If a particular action complies with RSS security constraints, that action can be performed. Otherwise, if the proposed navigational action is not compliant with RSS safety constraints (e.g., if the proposed operation is likely to cause an accident responsible for the host vehicle based on one or more of the above rules), No action is taken.

  [0571] In practice, certain implementations may include a navigation system for the host vehicle. The host vehicle can include an image capture device (e.g., one or more cameras such as any of those described above) that captures images representing the environment of the host vehicle during operation. Using the image information, the driving policy can capture a plurality of inputs and output planned navigation actions to achieve the navigation goals of the host vehicle. Driving policies may include various inputs (e.g., images from one or more cameras showing the surroundings of the host vehicle, including target vehicles, roads, objects, pedestrians, etc.), outputs from lidar or radar systems, speed sensors and suspensions. A set of programmed instructions or a trained network that can accept outputs from sensors or the like, one or more targets of the host vehicle, such as information describing a navigation plan for delivering passengers to a particular location, etc. May be included. Based on that input, the processor can identify a target vehicle in the host vehicle's environment, for example, by analyzing camera images, lidar output, radar output, and the like. In some embodiments, the processor identifies a target vehicle in the environment of the host vehicle by analyzing one or more inputs, such as one or more camera images, lidar output and / or radar output. be able to. Further, in some embodiments, the processor (e.g., analyzes one or more camera images, lidar output, and / or radar output, and detects a target vehicle based on a majority match or combination of inputs. Based on matching the majority or combination of the sensor inputs (by receiving the results), a target vehicle within the environment of the host vehicle can be identified.

  [0572] Based on the information available to the driving policy module, an output may be provided in the form of one or more planned navigation actions to achieve the host vehicle's navigation goals. In some embodiments, RSS safety constraints can be applied as a filter for planned navigation actions. That is, the planned navigation actions, once determined, may include at least one accident liability rule (eg, the accident liability rules discussed above) to determine the potential accident liability of the host vehicle with respect to the identified target vehicle. ) Can be tested against As noted above, if the test of the planned navigation action for at least one accident liability rule indicates that there is potential accident liability for the host vehicle when the planned navigation action is performed, the processor Can prevent the host vehicle from performing the planned navigation operation. On the other hand, if the test of the planned navigation action for the at least one accident liability rule indicates that no accident liability will occur for the host vehicle if the planned navigation action is performed, the processor is configured to Navigation operation can be performed by the host vehicle.

  [0573] In some embodiments, the system can test multiple potential navigation actions against at least one accident liability rule. Based on the results of the test, the system can filter those potential navigation actions into a subset of multiple potential navigation actions. For example, in some embodiments, a subset thereof indicates that testing for at least one accident liability rule indicates that no potential accident liability will occur for the host vehicle if a potential navigation action is taken. It may include only navigation operations. The system then potential navigation operation without accident responsible scored with and / or priority, based on one of the navigation operation for implementing the optimized score or highest priority such as selective can do. The score and / or priority may be based on one or more factors, for example, potential navigational activities that are deemed safest, most efficient, most comfortable, etc. for the passenger.

  [0574] In some examples, determining whether to perform a particular planned navigation operation may also depend on whether a default emergency procedure is available in a next state after the planned operation. If DEP is available, the RSS filter can approve its planned operation. On the other hand, if DEP is not available, the next condition may be considered dangerous and the planned navigation action may be rejected. In some embodiments, the planned navigation action may include at least one default emergency procedure.

  [0575] One advantage of the described system is that only the operation of the host vehicle with respect to a particular target vehicle needs to be considered to ensure safe operation by the vehicle. Thus, if there is more than one target vehicle, the host vehicle's planned target vehicle in the affected area near the host vehicle (eg, within a range of 25 meters, 50 meters, 100 meters, 200 meters, etc.). Operation can be tested sequentially for accident liability rules. In practice, at least one processor determines a plurality of other target vehicles in the host vehicle's environment based on an analysis of at least one image representative of the host vehicle's environment (or based on rider information or radar information, etc.). It may be further programmed to repeat the test of the planned navigation action against at least one accident liability rule to identify and determine the potential accident liability of the host vehicle for each of the plurality of other target vehicles. If the repeated testing of the planned navigation action for at least one accident liability rule indicates that there is a potential accident liability for the host vehicle if the planned navigation action is performed, the processor determines Navigation operation can be prevented from being performed by the host vehicle. If the repeated testing of the planned navigation action for the at least one accident liability rule indicates that no accident liability will occur for the host vehicle if the planned navigation action is performed, the processor proceeds to the planned Navigation operation can be performed by the host vehicle.

  [0576] As mentioned earlier, any of the above rules can be used as a basis for RSS security testing. In some embodiments, at least one accident liability rule is a follow-up that defines a distance behind the identified target vehicle, within which the host vehicle cannot travel without the possibility of accident liability. Including rules. In other cases, the at least one accident liability rule includes a preceding rule that defines a distance in front of the identified target vehicle, within which the host vehicle cannot proceed without the possibility of accident liability. .

  [0577] The system described above performs an RSS safety test to test compliance with the rule that the host vehicle should not take any action that would be responsible for the resulting accident, as a single planned navigation action. , But the test can also be applied to more than one planned navigation action. For example, in some embodiments, at least one processor determines two or more planned navigation actions to achieve a host vehicle navigation goal based on applying at least one driving policy. There are cases. In these situations, the processor may test each of the two or more planned navigation actions against at least one accident liability rule to determine potential accident liability. For each of two or more planned navigation actions, if the test indicates that there is potential accident liability for the host vehicle if a particular one of the two or more planned navigation actions is performed, The processor may not cause the host vehicle to perform the particular one of the planned navigation actions. On the other hand, for each of the two or more planned navigation actions, testing has determined that no accident liability will occur for the host vehicle if a particular one of the two or more planned navigation actions is performed. If shown, the processor may identify more than one planned one the specific navigation operation as a viable candidate for implementation. Next, the processor selects, based on the at least one cost function, a navigation action to be performed from among the viable candidates for implementation, and causes the host vehicle to perform the selected navigation action. Can be.

  [0578] Since the implementation of the RSS involves determining the relative potential liability for an accident between the host vehicle and one or more target vehicles, the planned navigation actions for safety compliance Along with testing the system, the system can track the potential for accident liability of the vehicles encountered. For example, not only may the system avoid avoiding actions that would result in the host vehicle being liable for the resulting accident, but the host vehicle system may also track one or more target vehicles, It may also be possible to identify and track which accident liability rules have been violated by those target vehicles. In some embodiments, an accident liability tracking system for a host vehicle receives at least one image representing an environment of the host vehicle from an image capture device, and analyzes the at least one image to analyze an environment of the host vehicle. It may include at least one processing device is programmed to identifying a target vehicle of the inner. Based on the analysis of the at least one image, the processor may include programming to determine one or more characteristics of the navigation state of the identified target vehicle. Can be used to determine potential accident liability based on vehicle speed, proximity to lane center, lateral speed, direction of travel, distance from host vehicle, heading, or any of the above rules The navigation state, such as any other parameters, may include various operating characteristics of the target vehicle. The processor may apply the determined one or more characteristics of the identified target vehicle navigation state to at least one accident liability rule (e.g., win at lateral speed, directional priority, lane to center of lane). Any of the above rules, such as winning by distance, following distance or preceding distance, interruption, etc.). Based on the comparison of the state with one or more rules, the processor may store at least one value indicative of the potential accident responsibility on the identified target vehicle side. If an accident occurs, the processor, the output of the stored at least one value (e.g., via any suitable data interfaces wired or wireless) can be provided. Such output can be provided, for example, after an accident between the host vehicle and at least one target vehicle, and the output can be used to indicate accident liability, or provide an accident liability instruction. can do.

  [0579] The at least one value indicative of potential accident liability can be stored at any suitable time and under any suitable conditions. In some embodiments, if it is determined that the host vehicle cannot avoid a collision with the identified target vehicle, the at least one processing device assigns and stores a collision responsibility value for the identified target vehicle. Can be.

  [0580] The accident liability tracking function is not limited to a single target vehicle, but rather can be used to track the potential accident liability of multiple target vehicles encountered. For example, the at least one processing device detects a plurality of target vehicles in the environment of the host vehicle, determines a navigation state characteristic of each of the plurality of target vehicles, and a respective navigation state for each of the target vehicles. Determining a value indicative of a potential accident liability on each side of each of the plurality of target vehicles based on a comparison of the characteristic with at least one accident liability rule. As noted above, accident liability rules that can be used as a basis for tracking liability may include any of the above rules or any other suitable rules. For example, the at least one accident liability rule may include a lateral speed rule, a lateral position rule, a driving direction priority rule, a traffic light based rule, a traffic sign based rule, a route priority rule, etc. The accident liability tracking function can also be combined with secure navigation based on considering RSS (eg, whether any movement of the host vehicle creates potential liability for the resulting accident).

  [0581] In addition to navigation based on considering accident liability in accordance with the RSS, navigation may be based on whether the navigation state of the vehicle and certain future navigation states are considered safe (for example, as described in detail above, It may also be considered with respect to determining whether DEP is present so that an accident can be avoided or any resulting accident is not considered the responsibility of the host vehicle. The host vehicle can be controlled to navigate from a safe state to a safe state. For example, in any particular situation, the driving policy may be used to generate one or more planned navigation actions and determine whether the future predicted state corresponding to each planned action provides a DEP. By judging, those operations can be tested. When providing a DEP, one or more planned navigation actions that provide the DEP may be considered safe and may be entitled to perform.

  [0582] In some embodiments, a navigation system for a host vehicle receives at least one image representative of an environment of the host vehicle from an image capture device, and a plan for achieving the host vehicle's navigation goals. Determining the determined navigation behavior based on at least one driving policy; analyzing the at least one image to identify a target vehicle in the environment of the host vehicle; and a potential of the host vehicle relative to the identified target vehicle. Testing the planned navigational action against at least one accident liability rule to determine overall accident liability, and testing the planned navigational action against at least one accident liability rule comprises: There is a potential accident liability for the host vehicle when action is taken. Indicating that the planned navigation action is not to be performed by the host vehicle, and testing the planned navigation action against at least one accident liability rule is an accident for the host vehicle when the planned navigation action is performed. At least one processing device programmed to cause the host vehicle to perform the planned navigation operation when indicating that no liability will occur may be included.

  [0583] In some embodiments, the navigation system for the host vehicle receives at least one image representing an environment of the host vehicle from an image capture device, and a plurality of potential navigations for the host vehicle. Determining an operation based on at least one driving policy; analyzing the at least one image to identify a target vehicle in a host vehicle environment; and a potential accident of the host vehicle with respect to the identified target vehicle. Testing a plurality of potential navigation actions against at least one accident liability rule to determine liability, and selecting one of the potential navigation actions, the potential navigation action being For that one of the tests, the test is performed on the host vehicle when the potential navigation action selected is performed. Programmed to select one of the potential navigation actions and to cause the host vehicle to perform the selected potential navigation action, indicating that no accident liability will occur. It may include at least one processing device. In some examples, the potential navigation action that is selected is a subset of the plurality of potential navigation actions, and for that subset of the plurality of potential navigation actions, the test returns the plurality of potential navigation actions. A subset of a plurality of potential navigation actions may be selected, indicating that no accident liability will occur for the host vehicle if any of the subsets of actions are performed. Further, in some examples, the selected potential navigation action may be selected according to a scoring parameter.

  [0584] In some embodiments, a system for navigating a host vehicle includes receiving at least one image representative of an environment of the host vehicle from an image capture device and achieving a navigation goal for the host vehicle. And determining at least one planned navigation action based on at least one driving policy. The processor analyzes the at least one image to identify a target vehicle in the environment of the host vehicle, and a process between the host vehicle and the target vehicle that will occur if a planned navigation operation is performed. Determining the next state distance; determining the current maximum braking capacity of the host vehicle and the current speed of the host vehicle; determining the current speed of the target vehicle; and recognizing at least one of the target vehicles. Assuming the maximum braking capacity of the target vehicle based on the determined characteristics, and the current speed of the target vehicle and the assumed maximum of the target vehicle, given the maximum braking capacity of the host vehicle and the current speed of the host vehicle. The host vehicle stops within a stop distance that is less than the target vehicle travel distance determined based on the braking capacity plus the determined next state distance. To obtain, it can be performed and performing a planned navigation operation. The stopping distance may further include the distance the host vehicle travels during a reaction time without brakes.

  [0585] The recognized characteristics of the target vehicle on which the maximum braking capability of the target vehicle is determined may include any suitable characteristics. In some embodiments, this characteristic may include vehicle type (e.g., motorcycle, car, bus, truck, each of which may be associated with a different braking profile), vehicle size, predicted or known vehicle weight, It may include, for example, a model of the vehicle (which may be used to determine known braking capabilities, etc.).

  [0586] In some cases, the determination of the safe state can be made for two or more target vehicles. For example, in some cases, the determination of a safe state (based on distance and braking capacity) may be based on two or more identified target vehicles preceding the host vehicle. Such a determination may be particularly useful when information about what is in front of the leading target vehicle is not available. In this case, it may be assumed that the detectable lead vehicle experiences an imminent collision with an immovable or almost immobile obstacle to determine the safe condition, safe distance, and / or available DEP. Yes, so the following target vehicle may reach a stop faster than its braking profile allows (e.g., the second target vehicle collides with the first leading vehicle, resulting in an expected maximum Stops can be reached faster than braking conditions). In that case, it may be important to base the determination of the safe condition, safe following distance, and DEP on the position of the identified first target vehicle relative to the host vehicle.

  [0587] In some embodiments, such a safe-to-safe navigation system comprises at least one program programmed to receive from the image capture device at least one image representative of the environment of the host vehicle. One processing device may be included. Again, as in other embodiments, image information captured by an image capture device (eg, a camera) can be supplemented with information obtained from one or more other sensors, such as a lidar or radar system. In some embodiments, the image information used to navigate can even come from a lidar or radar system rather than from an optical camera. The at least one processor may determine a planned navigation action to achieve a navigation goal for the host vehicle based on the at least one driving policy. The processor analyzes at least one image (e.g., obtained from a camera, radar, lidar, or any other device from which images of the environment of the host vehicle can be obtained based on optics, distance maps, etc.). A first target vehicle ahead of the host vehicle and a second target vehicle ahead of the first target vehicle can be identified. Then, the processor would occur if the planned navigation operation is performed, it is possible to determine the distance of the next state between the host vehicle and the second target vehicle. The processor can then determine the host vehicle's current maximum braking capacity and the host vehicle's current speed. Processor, given the current speed of the maximum braking capacity of the host vehicle and the host vehicle, the host vehicle and the host vehicle in the stopping distance determined is less than the distance of the next state between the second target vehicle Can be stopped, a planned navigation action can be performed.

  [0588] That is, if there is sufficient distance between the preceding visible target vehicle and the host vehicle to stop at the next state distance without a collision or without a collision attributable to the host vehicle. If the vehicle's processor determines, and assuming that the preceding visible target vehicle suddenly comes to a complete stop at any moment, the host vehicle's processor can perform the planned navigation action. On the other hand, if there is not enough space to stop the host vehicle without a collision, the planned navigation action may not be performed.

  [0589] Additionally, in some embodiments, the next state distance may be used as a benchmark, but in other cases, another distance value may be used to determine whether to perform a planned navigation action. Can be used. Similar to what was described above, in some cases the actual distance that the host vehicle may have to stop to avoid a collision may be less than the expected next state distance . For example, if the preceding visible target vehicle is followed by one or more other vehicles (the first target vehicle in the example above), the actual predicted required stop distance will be the predicted next stop condition. The distance is obtained by subtracting the length of the target vehicle following the preceding visible target vehicle from the distance. Assuming that the preceding visible target vehicle suddenly stops, the following target vehicle collides with the preceding visible target vehicle, and consequently those vehicles must also be avoided by the host vehicle to avoid the collision. can do. Accordingly, the host vehicle processor evaluates the next state distance minus the total length of any intervening target vehicles between the host vehicle and the preceding visible / detected target vehicle to provide a collision. Without maximum braking conditions, it can be determined whether there is sufficient space to stop the host vehicle.

  [0590] In other embodiments, the benchmark distance for evaluating a collision between the host vehicle and one or more preceding target vehicles may be greater than a predicted next state distance. For example, in some cases, the leading visible / detected target vehicle may stop quickly but not immediately, such that the leading visible / detected target vehicle is short-ranged after a possible collision. Moving. For example, if the vehicle collides with a parked vehicle, the colliding vehicle may continue to travel some distance until it completely stops. The assumed post-collision travel distance may be less than the assumed or determined minimum stopping distance for the associated target vehicle. Thus, in some cases, the host vehicle's processor may increase the distance of the next state when evaluating whether to perform a planned navigation operation. For example, in making this determination, the next state distance should be increased by 5%, 10%, 20%, etc., to account for the reasonable distance that the leading / visible target vehicle can travel after a possible imminent collision. Or supplemented at a predetermined fixed distance (10 m, 20 m, 50 m, etc.).

  [0591] In addition to increasing the distance of the next state during the evaluation by the assumed distance value, the distance traveled after the collision by the preceding visible / detected target vehicle and the preceding visible / detected The next state taking into account both the length of any target vehicle following the followed target vehicle (which may be assumed to be in multiple collisions with the preceding visible / detected vehicle after a sudden stop of that vehicle) Can modify the distance.

  [0592] The determination as to whether to perform the planned navigation operation is made by determining whether the host vehicle and the preceding visible / detected target vehicle have moved (the preceding visible / detected target vehicle has moved after the collision). And / or modified by considering the length of the vehicle following the preceding visible / detected target vehicle). , The braking capacity of one or more preceding vehicles can be taken into account. For example, the host vehicle processor may determine that the host vehicle and a first target vehicle (eg, a target following a preceding visible / detected target vehicle) will occur when a planned navigation operation is performed. Determining a distance of a next state between the first target vehicle and the first target vehicle, and determining a current speed of the first target vehicle and based on at least one recognized characteristic of the first target vehicle. Assuming the maximum braking capability of the target vehicle, and given the maximum braking capability of the host vehicle and the current speed of the host vehicle, the current speed of the first target vehicle and the assumed maximum of the first target vehicle. The host within a stopping distance that is less than the distance traveled by the first target vehicle determined based on the braking capability plus the determined next state distance between the host vehicle and the first target vehicle. Vehicle If sealed obtained not, and not to implement the planned navigation operation can be continued. Again, as in the above example, the recognized characteristics of the first target vehicle may include the vehicle type, vehicle size, vehicle model, and the like.

  In some cases, the host vehicle may determine that a collision is imminent and unavoidable (eg, due to the operation of another vehicle). In that case, the processor of the host vehicle may be configured to select a navigation action, if any, for which the resulting collision does not create responsibility for the host vehicle. Additionally or alternatively, the processor of the host vehicle may cause less potential damage to the host vehicle or less potential damage to the target than the current trajectory or compared to one or more other navigation operations. It can be configured to select a navigation action to give. Further, in some cases, the processor of the host vehicle may select a navigation action based on considering the type of object or objects for which a collision is expected. For example, if a first navigation operation encounters a collision with a parked car or a second navigation operation encounters a collision with an immovable object, an operation that causes less potential damage to the host vehicle (e.g., , An action that results in a collision with a parked car). Less potential damage to the host vehicle if the first navigation operation encounters a collision with a car traveling in the same direction as the host vehicle or the second navigation operation encounters a collision with a parked car (For example, an operation that causes a collision with a moving car). Provide any alternative to collision with pedestrians when facing a collision with a pedestrian as a result of a first navigation operation or a collision with any other object in a second navigation operation Actions can be selected.

  [0594] In practice, a system for navigating the host vehicle, at least one image representing the host vehicle environment (e.g., a visible image, lidar images, radar images, etc.) to receive from the image capture device and Receiving an indicator of a current navigation state of the host vehicle from at least one sensor and analyzing the at least one image and an indicator of the current navigation state of the host vehicle based on the at least one sensor and the one or more objects. And determining that a collision between is inevitable is unavoidable. The processor can evaluate available alternatives. For example, the processor may include a first planned navigation action for the host vehicle with an expected collision with the first object based on the at least one driving policy, and an expected collision with the second object. And a second planned navigation action for the host vehicle with The first planned navigation action and the second planned navigation action can be tested against at least one accident liability rule to determine potential accident liability. If the test of the first planned navigation action against the at least one accident liability rule indicates that there is potential accident liability for the host vehicle when the first planned navigation action is performed, the processor , It is possible to prevent the host vehicle from performing the first planned navigation operation. If the test of the second planned navigation action for the at least one accident liability rule indicates that no accident liability will occur for the host vehicle if the second planned navigation action is performed, the processor may include: , May cause the host vehicle to perform a second planned navigation operation. The objects may include other vehicles or non-vehicle objects (eg, road debris, trees, poles, signs, pedestrians, etc.).

  [0595] The following figures and description provide examples of various scenarios that may occur when navigating and implementing the disclosed systems and methods. In these examples, the host vehicle may avoid performing an operation that results in negligence attributed to the host vehicle for an operation that results if the operation were performed.

  FIGS. 28A and 28B show examples of the following scenarios and rules. As shown in FIG. 28A, the area surrounding a vehicle 2804 (eg, a target vehicle) is the minimum safe distance for a vehicle 2802 (eg, a host vehicle) traveling in a lane at a distance behind vehicle 2804. Represents According to one rule consistent with the disclosed embodiments, vehicle 2802 must maintain a minimum safe distance by staying within the area surrounding vehicle 2802 to avoid an accident in which vehicle 2802 is at fault. . In contrast, as shown in FIG. 28B, when the vehicle 2804 applies a brake, and in the event of an accident, the vehicle 2802 has a cause for liability.

  FIGS. 29A and 29B show examples of negligence in an interrupt scenario. In these scenarios, a safety corridor around the vehicle 2902 determines the culprit of the interrupt operation. As shown in FIG. 29A, the vehicle 2902 has interrupted in front of the vehicle 2902 and has violated a safe distance (indicated by the area surrounding the vehicle 2902), and is therefore attributable. As shown in FIG. 29B, the vehicle 2902 interrupts in front of the vehicle 2902, but maintains a safe distance in front of the vehicle 2904.

  FIGS. 30A and 30B show examples of negligence in an interrupt scenario. In these scenarios, the safety corridor around vehicle 3004 determines whether vehicle 3002 is attributable. In FIG. 30A, vehicle 3002 is traveling behind vehicle 3006 and changes lanes to the lane in which target vehicle 3004 is moving. In this scenario, the vehicle 3002 has violated the safe distance, and thus has an attributable cause in the event of an accident. In FIG. 30B, vehicle 3002 interrupts behind vehicle 3004 and maintains a safe distance.

  FIG. 31A to FIG. 31D show examples of negligence in a drift scenario. In FIG. 31A, this scenario begins with a slight lateral maneuver by vehicle 3104 interrupting the wide corridor of vehicle 3102. In FIG. 31B, vehicle 3104 continues to interrupt the normal corridor of vehicle 3102, violating the safe distance area. If an accident occurs, the vehicle 3104 is negligent. In FIG. 31C, vehicle 3104 has maintained its initial position while vehicle 3102 has moved laterally to "force" a violation of the normal safety distance zone. When an accident occurs, the vehicle 3102 has a fault. In FIG. 31B, vehicle 3102 and vehicle 3104 move laterally toward each other. In the event of an accident, the fault is shared by both vehicles.

  [0600] FIGS. 32A and 32B show examples of negligence in a two-way traffic scenario. In FIG. 32A, the vehicle 3202 overtakes the vehicle 3206, and the vehicle 3202 performs an interruption operation for keeping a safe distance from the vehicle 3204. In the event of an accident, vehicle 3204 is negligible in not braking with reasonable force. In FIG. 32B, vehicle 3202 has interrupted without maintaining a safe vertical distance from vehicle 3204. If an accident occurs, the vehicle 3202 is negligent.

  [0601] FIGS. 33A and 33B show examples of negligence in a two-way traffic scenario. In FIG. 33A, vehicle 3302 has drifted into the path of oncoming vehicle 3204, maintaining a safe distance. In the event of an accident, vehicle 3204 is negligible in not braking with reasonable force. In FIG. 33B, vehicle 3202 has drifted into the path of oncoming vehicle 3204, violating a safe longitudinal distance. If an accident occurs, there is a fault in the vehicle 3204.

  [0602] FIGS. 34A and 34B show examples of negligence in a root priority scenario. In FIG. 34A, vehicle 3202 ignores the temporary stop sign. Negligence is attributed to the vehicle 3202 for not respecting the priority given to the vehicle 3204 by the traffic light. In FIG. 34B, the vehicle 3202 has no priority but has already entered the intersection when the traffic light of the vehicle 3204 turns green. When the vehicle 3204 collides with the vehicle 3202, the vehicle 3204 has negligence.

  [0603] FIGS. 35A and 35B show examples of negligence in a route priority scenario. In FIG. 35A, the vehicle 3502 is backing in the path of the approaching vehicle 3504. The vehicle 3502 is performing an interruption operation for maintaining a safe distance. In the event of an accident, vehicle 3504 is negligible in not braking with reasonable force. In FIG. 35B, the vehicle 3502 is interrupting without maintaining a safe vertical distance. If an accident occurs, the vehicle 3502 is negligent.

  [0604] FIGS. 36A and 36B show examples of negligence in a root priority scenario. In FIG. 36A, vehicle 3602 and vehicle 3604 are traveling in the same direction, while vehicle 3602 is turning left across the path of vehicle 3604. The vehicle 3602 is performing an interruption operation for maintaining a safe distance. In the event of an accident, vehicle 3604 is negligible in not braking with reasonable force. In FIG. 36B, vehicle 3602 is interrupting without maintaining a safe vertical distance. When an accident occurs, the vehicle 3602 is negligent.

  FIG. 37A and FIG. 37B show examples of negligence in a route priority scenario. In FIG. 37A, vehicle 3702 wants to turn left, but must yield to oncoming vehicle 3704. Vehicle 3702 turns left and violates a safe distance to vehicle 3704. The vehicle 3702 has a fault. In FIG. 37B, vehicle 3702 turns left to maintain a safe distance to vehicle 3704. In the event of an accident, vehicle 3704 is negligible in not braking with reasonable force.

  FIG. 38A and FIG. 38B show examples of negligence in a route priority scenario. In FIG. 38A, the vehicle 3802 and the vehicle 3804 are traveling straight, and the vehicle 3802 has a sign of a stop. Vehicle 3802 enters an intersection and violates a safe distance to vehicle 3804. The vehicle 3802 has a fault. In FIG. 38B, vehicle 3802 is entering an intersection while maintaining a safe distance from vehicle 3804. In the event of an accident, vehicle 3804 is negligible in not braking with reasonable force.

  FIG. 39A and FIG. 39B show examples of negligence in a route priority scenario. In FIG. 39A, vehicle 3902 wants to turn left, but must yield to vehicle 3904 coming from the right. Vehicle 3902 enters an intersection and violates the priority right of travel and the safe distance to vehicle 3904. The vehicle 3902 has a fault. In FIG. 39B, the vehicle 3902 is entering the intersection while maintaining a priority right to travel and a safe distance to the vehicle 3904. In the event of an accident, vehicle 3904 is negligent of not braking with reasonable force.

  [0608] FIGS. 40A and 40B show examples of negligence in a traffic light scenario. In FIG. 40A, vehicle 4002 ignores the red light. Negligence is attributed to the vehicle 4002 for not respecting the priority given to the vehicle 4004 by the traffic light. In FIG. 40B, vehicle 4002 has no priority, but vehicle 4002 was already at the intersection when the traffic light for vehicle 4004 turned green. When the vehicle 4004 collides with the vehicle 4002, the vehicle 4004 has negligence.

  [0609] FIGS. 41A and 41B show examples of negligence in a traffic light scenario. Vehicle 4102 is left across the path of the oncoming vehicle 4104. Priority is on the vehicle 4104. In FIG. 41, the vehicle 4102 turns left, violating the safe distance to the vehicle 4104. Negligence is attributed to the vehicle 4102. In FIG. 41B, vehicle 4102 turns left to maintain a safe distance to vehicle 4104. In the event of an accident, vehicle 4104 is negligible in not braking with reasonable force.

  [0610] FIGS. 42A and 42B show an example of a fault in the traffic scenario. In FIG. 42A, the vehicle 4202 turns right and interrupts the route of the vehicle 4204 traveling straight. A right turn at a red light is considered to be a legal operation, but the vehicle 4202 has a right of priority because the vehicle 4202 violates a safe distance to the vehicle 4204. The fault is attributable to the vehicle 4202. In FIG. 42B, vehicle 4202 turns right to maintain a safe distance to vehicle 4204. In the event of an accident, vehicle 4204 is negligible in not braking with reasonable force.

  [0611] FIGS. 43A to 43C show examples of traffic vulnerable (VRU) scenarios. Accidents with animals or VRUs when the vehicle performs operations are treated as a variant of the interruption, and with some exceptions, the default fault is in the vehicle. In FIG. 43A, the vehicle 4302 has been interrupted into the animal (or VRU) path while ensuring that the vehicle 4302 can maintain a safe distance and avoid an accident. In FIG. 43B, vehicle 4302 has interrupted the path of the animal (or VRU), violating the safe distance. The fault is attributable to the vehicle 4302. In FIG. 43C, vehicle 4302 recognizes the animal and stops, giving the animal enough time to stop. If an animal collides with a car, the animal is at fault.

  [0612] FIGS. 44A to 44C show examples of traffic vulnerable (VRU) scenarios. In FIG. 44A, a vehicle 4402 turns left at an intersection with a traffic light and encounters a pedestrian in a crosswalk. Vehicle 4402 is a red light and VRU is a green light. The vehicle 4402 has a cause for retribution. In FIG. 44B, vehicle 4402 is a green light and VRU is a red light. If the VRU enters the pedestrian crossing, the VRU has attributable reasons. In FIG. 44C, vehicle 4402 is a green light and VRU is a red light. If the VRU was already within the pedestrian crossing, the vehicle 4402 has a blame.

  [0613] FIGS. 45A to 45C show examples of traffic vulnerable (VRU) scenarios. In FIG. 45A, vehicle 4402 turns right and encounters a bicycle. Bicycles are green light. The vehicle 4502 has a cause for retribution. In FIG. 45B, the bicycle is a red light. If a bicycle enters an intersection, there is a reason for blaming the bicycle. In FIG. 45C, the bicycle is at a red light, but has already entered the intersection. The vehicle 4502 has a cause for retribution.

  [0614] FIGS. 46A to 46D show examples of traffic vulnerable (VRU) scenarios. The negligence of an accident with a VRU when the vehicle does not perform a maneuver is in the vehicle by default, with a few exceptions. In FIG. 46A, it must always be ensured that the vehicle 4602 maintains a safe distance to ensure that an accident with the VRU can be avoided. In Figure 46B, when the vehicle 4602 is not kept a safe distance, there is a fault in the vehicle 4602. In FIG. 46C, the vehicle 4602 is negligible if the vehicle 4602 does not maintain a speed that is slow enough to avoid a collision with a VRU that may be hidden by the vehicle 5604, or travels above a legal speed. In FIG. 46D, in another scenario where the VRU is optionally hidden by the vehicle 4604, if the vehicle 4602 maintains a sufficiently low speed but the speed of the VRU is above a reasonable threshold, the VRU is negligent.

  [0615] As disclosed herein, RSS defines a framework for multi-agent scenarios. Accidents with static objects, departure from the road, loss of control, or negligence of vehicle failure are on the host side. RSS specifies a careful operation that will not cause an accident with another object unless another object enters the path of the host with danger (in which case the fault is on that object). In the event of a definite crash where the target is negligent, the host applies his brakes. The system, the work around the steering can be considered only if it is "cautious" (that does not cause another accident is observed).

  [0616] Non-collision incidents include accidents caused by vehicle fires, road pits, falling objects, and the like. In these cases, hosts such as road dips and falling objects that can be classified as a "static object" scenario, assuming that a "static object" becomes visible at a safe distance or that there is a careful avoidance maneuver. Except in scenarios where can be avoided, the fault can be on the host by default. In a multi-agent scenario where the host vehicle is stationary, there is no fault to the host. In this case, the target is essentially performing a dangerous interruption. For example, if a bicycle hits a stationary car, there is no fault to the host.

  [0617] RSS also includes guidelines for assigning fault when roads are not clearly structured, such as parking lots without lane markings or large roundabouts. In these unstructured road scenarios, each vehicle's departure from its own path is examined to determine if each vehicle has provided enough distance to allow other objects in the area to make adjustments. Judgment assigns negligence.

  [0618] The descriptions above have been presented for purposes of illustration. The above description is not exhaustive and is not limited to the precise forms or embodiments disclosed. Modifications and adaptations will become apparent to those skilled in the art from consideration of the specification and practice of the disclosed embodiments. Furthermore, although aspects of the disclosed embodiments are described as being stored in memory, these aspects may be implemented on other types of computer readable media, such as auxiliary storage devices, such as hard disks or CD ROMs or other forms. Those skilled in the art will understand that the storage may be on a RAM or ROM, USB media, DVD, Blu-ray®, 4K Ultra HD Blu-ray, or other optical drive media.

  [0619] Computer programs based on the written description and the disclosed method are within the skills of an experienced developer. The various programs or program modules can be created using any technique known to those skilled in the art, or can be designed in conjunction with existing software. For example, the program section or program module may be a .Net Framework, .Net Compact Framework (and related languages such as Visual Basic, C), Java (registered trademark), C ++, Objective-C, HTML, HTML / AJAX combination, XML , Or in HTML that includes Java applets.

  [0620] Further, while example embodiments have been described herein, the scope of any embodiment should be understood as equivalent elements, modifications, abbreviations, combinations, etc. (E.g., aspects across various embodiments), adaptations, and / or alternatives. The limitations in the claims are to be interpreted broadly based on the language utilized in the claims and are not limited to the examples described herein or the working examples of the present application. Examples should be interpreted as non-exclusive. In addition, the steps of the disclosed method may be modified in any manner, including reordering steps and / or inserting or deleting steps. It is therefore intended that the specification and examples be considered as exemplary only, with a true scope and spirit being indicated by the following claims and their full range of equivalents.

Claims (43)

A navigation system for a host vehicle,
Receiving at least one image representative of the environment of the host vehicle from an image capture device;
Determining a planned navigation action for achieving a navigation goal of the host vehicle based on at least one driving policy;
Analyzing the at least one image to identify a target vehicle in the environment of the host vehicle;
Testing the planned navigation behavior against at least one accident liability rule to determine a potential accident liability of the host vehicle with respect to the identified target vehicle;
If the test of the planned navigation action against the at least one accident liability rule indicates that there is potential accident liability for the host vehicle when the planned navigation action is performed, Not allowing the host vehicle to perform a navigation operation;
If the test of the planned navigation action against the at least one accident liability rule indicates that no accident liability will occur for the host vehicle if the planned navigation action is performed. A navigation system including at least one processing device programmed to cause the host vehicle to perform the navigation operation.   The at least one processing device is further programmed to not perform the planned navigation operation unless a default emergency procedure is available to the host vehicle in a next navigation state resulting from performing the planned navigation operation. It is the navigation system of claim 1.   The navigation system according to claim 1, wherein the planned navigation operation includes at least one default emergency procedure. The at least one processing device comprises:
Identifying a plurality of other target vehicles in the environment of the host vehicle based on an analysis of the at least one image representing the environment of the host vehicle, and identifying the host vehicle for each of the plurality of other target vehicles. Repeating the test of the planned navigation action for at least one accident liability rule to determine potential accident liability;
If said repeated test of said planned navigation action against said at least one accident liability rule indicates that there is potential accident liability for said host vehicle when said planned navigation action is performed; Not allowing the host vehicle to perform the planned navigation operation;
If the repeated test of the planned navigation action against the at least one accident liability rule indicates that no accident liability will occur for the host vehicle if the planned navigation action is performed; The navigation system of claim 1, further programmed to cause the host vehicle to perform the planned navigation operation.   The at least one accident liability rule includes a following rule that defines a distance behind the identified target vehicle within which the host vehicle cannot travel without the possibility of accident liability. The navigation system according to claim 1.   The at least one accident liability rule includes a precedence rule that defines a distance ahead of the identified target vehicle within which the host vehicle cannot travel without the possibility of accident liability; The navigation system according to claim 1.   The navigation system according to claim 1, wherein the driving policy is implemented by a trained system trained based on the at least one accident liability rule.   The navigation system according to claim 7, wherein the trained system comprises a neural network. A navigation system for a host vehicle,
Receiving at least one image representative of the environment of the host vehicle from an image capture device;
Determining a plurality of potential navigation actions for the host vehicle based on at least one driving policy;
Analyzing the at least one image to identify a target vehicle in the environment of the host vehicle;
Testing the plurality of potential navigation actions against at least one accident liability rule to determine a potential accident liability of the host vehicle with respect to the identified target vehicle;
Selecting one of the potential navigation actions, wherein for the one of the potential navigation actions, the test is performed when the selected potential navigation action is performed. Selecting, indicating that no liability will arise for
A navigation system including at least one processing device programmed to cause the host vehicle to perform the selected potential navigation operation.   The selected potential navigation action is selected from a subset of the plurality of potential navigation actions, and for the subset of the plurality of potential navigation actions, the test comprises: 10. The navigation system of claim 9, indicating that no accident liability will occur for the host vehicle if any of the subsets are performed.   The navigation system according to claim 10, wherein the selected potential navigation action is selected according to a scoring parameter. A system for navigating a host vehicle,
Receiving at least one image representative of the environment of the host vehicle from an image capture device;
Determining two or more planned navigation actions to achieve a navigation goal of the host vehicle based on at least one driving policy;
Analyzing the at least one image to identify a target vehicle in the environment of the host vehicle;
Testing each of the two or more planned navigation actions against at least one accident liability rule to determine potential accident liability;
For each of the two or more planned navigation actions, the test determines that there is a potential accident liability for the host vehicle if a particular one of the two or more planned navigation actions is performed. If indicating, not allowing said host vehicle to perform said particular one of said planned navigation actions;
For each of the two or more planned navigation actions, the test will determine that if one particular of the two or more planned navigation actions is performed, no accident liability will result for the host vehicle. Identifying the particular one of the two or more planned navigation actions as a viable candidate for implementation;
Selecting a navigation action to be performed among the viable candidates for implementation based on at least one cost function;
A system comprising at least one processing device programmed to cause the host vehicle to perform the selected navigation operation. An accident liability tracking system for the host vehicle,
Receiving at least one image representative of the environment of the host vehicle from an image capture device;
Analyzing the at least one image to identify a target vehicle in the environment of the host vehicle;
Determining one or more characteristics of a navigation state of the identified target vehicle based on an analysis of the at least one image;
Comparing the determined one or more characteristics of the navigation state of the identified target vehicle to at least one accident liability rule;
Determining, based on the comparison of the determined one or more characteristics of the navigation state of the identified target vehicle with the at least one accident liability rule, a potential accident liability on the identified target vehicle side. Storing at least one value to indicate;
At least one process programmed to output the stored at least one value to determine a liability for the accident after an accident between the host vehicle and at least one target vehicle. Accident liability tracking system including devices.   The at least one processing device is further programmed to assign and store a collision liability value for the identified target vehicle if the host vehicle cannot avoid a collision with the identified target vehicle. 14. The tracking system according to claim 13.   14. The tracking system of claim 13, further comprising an interface for accessing and downloading crash liability values after an accident between the host vehicle and the identified target vehicle.   The at least one processing device is for detecting a plurality of target vehicles in the environment of the host vehicle; determining a navigation state characteristic of each of the plurality of target vehicles; Determining and storing a value indicative of a potential accident liability on one side of each of the plurality of target vehicles based on a comparison of a respective navigation state characteristic with the at least one accident liability rule. 14. The tracking system according to claim 13, further programmed.   14. The tracking system of claim 13, wherein the at least one accident liability rule includes a lateral speed rule.   14. The tracking system of claim 13, wherein the at least one accident liability rule includes a lateral position rule.   14. The tracking system of claim 13, wherein the at least one accident liability rule comprises a driving direction priority rule.   14. The tracking system of claim 13, wherein the at least one accident liability rule comprises a traffic light based rule.   14. The tracking system of claim 13, wherein the at least one accident liability rule comprises a traffic sign based rule.   14. The tracking system of claim 13, wherein the at least one accident liability rule comprises a route priority rule. A system for navigating a host vehicle,
Receiving at least one image representative of the environment of the host vehicle from an image capture device;
Determining a planned navigation action for achieving a navigation goal of the host vehicle based on at least one driving policy;
Analyzing the at least one image to identify a target vehicle in the environment of the host vehicle;
Testing said planned navigation actions against at least one accident liability rule to determine potential accident liability;
If the test of the planned navigation action against the at least one accident liability rule indicates that there is potential accident liability for the host vehicle when the planned navigation action is performed, Not allowing the host vehicle to perform a navigation operation;
If the test of the planned navigation action against the at least one accident liability rule indicates that no accident liability will occur for the host vehicle if the planned navigation action is performed. And at least one processing device programmed to cause the host vehicle to perform the navigation operation, wherein the at least one processing device comprises:
Determining one or more characteristics of a navigation state of the identified target vehicle based on an analysis of the at least one image;
Comparing the determined one or more characteristics of the navigation state of the identified target vehicle to the at least one accident liability rule;
Potential accident liability on the side of the identified target vehicle based on the comparison of the determined one or more characteristics of the navigation state of the identified target vehicle with the at least one accident liability rule. Storing at least one value indicative of:   The processor is further programmed to output the stored at least one value after an accident between the host vehicle and at least one target vehicle to determine responsibility for the accident. 24. The system according to 23.   24. The system of claim 23, wherein the at least one accident liability rule comprises a lateral speed rule.   24. The system of claim 23, wherein the at least one accident liability rule includes a lateral position rule.   24. The system of claim 23, wherein the at least one accident liability rule includes a driving direction priority speed rule.   24. The system of claim 23, wherein the at least one accident liability rule comprises a traffic light based rule.   24. The system of claim 23, wherein the at least one accident liability rule comprises a traffic sign based rule.   24. The system of claim 23, wherein the at least one accident liability rule comprises a route priority rule. A system for navigating a host vehicle,
Receiving at least one image representative of the environment of the host vehicle from an image capture device;
Determining a planned navigation action for achieving a navigation goal of the host vehicle based on at least one driving policy;
Analyzing the at least one image to identify a target vehicle in the environment of the host vehicle;
Determining the next state distance between the host vehicle and the target vehicle, which will occur when the planned navigation action is performed;
Determining a current maximum braking capacity of the host vehicle and a current speed of the host vehicle;
Determining a current speed of the target vehicle and assuming a maximum braking capacity of the target vehicle based on at least one recognized characteristic of the target vehicle;
Given the maximum braking capability of the host vehicle and the current speed of the host vehicle, the target vehicle is determined based on the current speed of the target vehicle and the assumed maximum braking capability of the target vehicle. And performing the planned navigation action if the host vehicle can be stopped within a stopping distance that is less than the determined moving distance plus the determined next state distance. A system comprising at least one processing device.   32. The system of claim 31, wherein the stopping distance further comprises a distance that the host vehicle travels during a reaction time without a brake.   32. The system of claim 31, wherein the recognized characteristics of the target vehicle include a vehicle type.   32. The system of claim 31, wherein the recognized characteristic of the target vehicle is a size of the vehicle.   32. The system of claim 31, wherein the recognized characteristics of the target vehicle include a model of the vehicle. A system for navigating a host vehicle,
Receiving at least one image representative of the environment of the host vehicle from an image capture device;
Determining a planned navigation action for achieving a navigation goal of the host vehicle based on at least one driving policy;
Analyzing the at least one image to identify a first target vehicle ahead of the host vehicle and a second target vehicle ahead of the first target vehicle;
Determining a next state distance between the host vehicle and the second target vehicle, which will occur when the planned navigation operation is performed;
Determining a current maximum braking capacity of the host vehicle and a current speed of the host vehicle;
A stop distance that is less than the determined next state distance between the host vehicle and the second target vehicle, given the maximum braking capacity of the host vehicle and the current speed of the host vehicle. And performing at least one processing device when the host vehicle can be stopped. The at least one processor comprises:
Determining a next state distance between the host vehicle and the first target vehicle that will occur when the planned navigation action is performed;
Determining a current speed of the first target vehicle and assuming a maximum braking capability of the first target vehicle based on at least one recognized characteristic of the first target vehicle;
Given the maximum braking capability of the host vehicle and the current speed of the host vehicle, based on the current speed of the first target vehicle and the assumed maximum braking capability of the first target vehicle The host within a stop distance that is less than the sum of the distance traveled by the first target vehicle and the distance determined in the next state between the host vehicle and the first target vehicle. 37. The system of claim 36, further programmed to: do not perform the planned navigation action if the vehicle cannot be stopped.   37. The system of claim 36, wherein the recognized characteristics of the first target vehicle include a vehicle type.   37. The system of claim 36, wherein the recognized characteristic of the first target vehicle is a size of the vehicle.   37. The system of claim 36, wherein the recognized characteristics of the first target vehicle include a model of the vehicle. A system for navigating a host vehicle,
Receiving at least one image representative of the environment of the host vehicle from an image capture device;
Receiving from the at least one sensor an indicator of the current navigation state of the host vehicle;
Determining that a collision between the host vehicle and one or more objects is unavoidable based on the analysis of the at least one image and the indicator of the current navigation state of the host vehicle;
A first planned navigation action for the host vehicle with an expected collision with a first object based on at least one driving policy, and the host with an expected collision with a second object; Determining a second planned navigation action for the vehicle;
Testing the first planned navigation action and the second planned navigation action against at least one accident liability rule to determine potential accident liability;
The test of the first planned navigation action against the at least one accident liability rule indicates that there is potential accident liability for the host vehicle when the first planned navigation action is performed. Not allowing the host vehicle to perform the first planned navigation operation;
The test of the second planned navigation action against the at least one accident liability rule indicates that no accident liability will occur for the host vehicle if the second planned navigation action is performed. If indicated, a system comprising at least one processing device programmed to cause the host vehicle to perform the second planned navigation operation.   42. The system of claim 41, wherein at least one of said first object and said second object comprises another vehicle.   42. The system of claim 41, wherein at least one of the first object and the second object comprises an object that is not a vehicle.