KR20240024735A - Virtual fields driving related operations - Google Patents

Abstract

A method for virtual field driving-related operations, the method comprising: (i) obtaining coarse navigation information for at least a portion of a vehicle's route to a destination of the vehicle; and (ii) using processing circuitry and determining one or more virtual forces based on the approximate navigation information, wherein the one or more virtual forces are for use in applying a driving-related operation of the vehicle, the one or more virtual forces belongs to a cyber-physical model and at least partially represents the approximate navigation information.

Description

VIRTUAL FIELDS DRIVING RELATED OPERATIONS}

This application is a continuation-in-part of U.S. Patent Application No. 17/823,069, filed Aug. 29, 2022, which claims priority from U.S. Provisional Application No. 63/260,839, which is incorporated herein by reference. This application claims priority from U.S. Provisional Patent Serial No. 63/368,156, filed July 11, 2022, the entire contents of which are hereby incorporated by reference.

The present invention relates to a method for virtual field driving related operations.

Autonomous vehicles (AVs) can help significantly reduce the number of traffic accidents and _CO2 emissions, as well as contribute to more efficient transport systems. However, today's candidate AV technologies cannot scale in three ways:

Limited field of view, lighting and weather issues, and occlusion all lead to detection errors and noisy localization/kinematics. To deal with this poor real-world perceptual output, one approach for AV technology is to invest in expensive sensors and/or integrate specialized infrastructure into road networks. However, these efforts are expensive and geographically limited in terms of infrastructure, so they cannot lead to generally accessible AV technology.

AV technology, which is not based on expensive hardware and infrastructure, relies entirely on machine learning and thus relies on data to process real-world situations. In order to handle detection errors and learn good and sufficient driving policies for complex driving tasks, a huge amount of data and computational resources are required, but there are still edge cases that are not handled accurately. The common denominator in these extreme cases is that machine learning models do not generalize well to invisible or confusing situations, and the black box nature of deep neural networks makes it difficult to analyze incorrect behavior.

Currently, road-ready autonomous driving is implemented in the form of separate ADAS functions such as ACC, AEB, and LCA. Reaching fully autonomous driving requires seamlessly combining existing ADAS functions and adding more of these functions (e.g. lane changes, intersection handling, etc.) to fill the gaps that are currently not automated. In short, current autonomous driving is not based on a holistic approach that can be easily expanded to implement fully autonomous driving.

Accordingly, the present invention provides a method for virtual field driving-related operations, the method comprising: obtaining approximate navigation information regarding at least a segment of the vehicle's route to the vehicle's destination; and determining, using processing circuitry and based on the approximate navigation information, one or more virtual forces, wherein the one or more virtual forces are for use in applying a driving-related operation of the vehicle, the one or more virtual forces being virtual forces. belonging to a physical model, and at least partially representing the approximate navigation information.

The invention also provides a non-transitory computer-readable medium storing instructions that, once executed by processing circuitry, cause the processing circuitry to:

obtain approximate navigation information regarding at least a segment of the vehicle's route to the vehicle's destination; and determine, using the processing circuitry and based on the approximate navigation information, one or more virtual forces, the one or more virtual forces for use in applying a driving-related operation of the vehicle, wherein the one or more virtual forces are for use in applying a driving-related operation of the vehicle. The virtual force belongs to the virtual physical model and represents at least in part the approximate navigation information.

Embodiments of the invention will be more fully understood and appreciated from the following detailed description taken in conjunction with the drawings:
Figure 1 shows one embodiment of the method.
Figure 2 shows one embodiment of the method.
Figure 3 shows one embodiment of the method.
Figure 4 shows one embodiment of the method.
5 shows an embodiment of a vehicle.
Figures 6-9 illustrate embodiments of situational and perceptual domains.
Figure 10 shows one embodiment of the method.
Figure 11 shows one embodiment of the method.
Figure 12 shows one embodiment of the method.
Figure 13 shows one embodiment of a scene.
Figure 14 shows one embodiment of a scene.

In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the invention. However, it will be understood by those skilled in the art that the present invention may be practiced without these specific details. In other instances, well-known methods, procedures and components are not described in detail so as not to obscure the invention.

The subject matter considered to be the invention is specifically pointed out and clearly asserted in the conclusion of the detailed description. However, the objects, features and advantages of the present invention, together with its construction and method of operation, may be best understood by reference to the following detailed description when read in conjunction with the accompanying drawings.

For simplicity and clarity of explanation, elements shown in the drawings are not necessarily drawn to scale. For example, the dimensions of some elements may be exaggerated relative to others for clarity. Additionally, where deemed appropriate, reference numerals may be repeated among the drawings to indicate corresponding or similar elements.

Since the illustrated embodiments of the present invention can be implemented using mostly electrical components and circuits known to those skilled in the art, the details are provided to facilitate understanding and appreciation of the underlying concepts of the present invention and to avoid confusing the teachings of the present invention. In order not to be confusing or distracting, no more will be explained than is deemed necessary as aforesaid.

Any reference in the detailed description of the method shall apply mutatis mutandis to a device or system capable of executing the method and/or a non-transitory computer-readable medium storing instructions for executing the method.

Any reference in the detailed description to a system or device should apply mutatis mutandis to a method executable by the system and/or may apply mutatis mutandis to a non-transitory computer-readable medium storing instructions executable by the system.

Any reference in the detailed description to a non-transitory computer-readable medium shall apply mutatis mutandis to a device or system capable of executing instructions stored on the non-transitory computer-readable medium and/or may apply mutatis mutandis to a method for executing the instructions. there is.

Any combination of modules or units listed in any of the drawings, detailed description, and/or claims may be provided.

Any one of the units and/or modules illustrated in the application may be implemented in hardware and/or code, and instructions and/or instructions stored in a non-transitory computer-readable medium may be stored in a vehicle, outside the vehicle, mobile device, server, etc. may be included.

The vehicle may be any type of vehicle, including ground transport vehicles, air vehicles, and watercraft.

This detailed description and/or drawings may refer to images. An image is an example of a media unit. Any reference to an image may apply mutatis mutandis to a media unit. A media unit may be an example of a Sensed Information Unit (SIU). Any reference to a media unit may include any type of natural signal, such as a signal generated by nature, a signal representing human behavior, a signal representing operations related to the stock market, a medical signal, a financial series, or a geodetic signal. , may be applied mutatis mutandis to signals representing operations related to geophysics, chemistry, molecules, text and numeric signals, time series, etc., but is not limited thereto. Any reference to a media unit may apply mutatis mutandis to a sensed information unit (SIU). SIUs can be of any type and include visible light cameras, audio sensors, sensors capable of detecting infrared, radar images, sensors capable of detecting ultrasound, electro-optics, radiography, LIDAR (light detection and ranging), and thermal sensors. , can be sensed by any type, such as passive sensors, active sensors, etc. Sensing may involve generating samples (e.g., pixels, audio signals) that represent the transmitted signal or otherwise reaching the sensor. An SIU may be one or more images, one or more video clips, text information about one or more images, text describing kinematic information about an object, etc.

Object information may include, but is not limited to, any type of information related to an object, such as the location of the object, the motion of the object, the speed of the object, the acceleration of the object, the direction of propagation of the object, the type of object, one or more dimensions of the object, etc. It doesn't work. Object information may be raw SIU, processed SIU, text information, information derived from SIU, etc.

Obtaining object information includes receiving object information, generating object information, participating in processing of object information, processing only part of object information, and/or receiving only other parts of object information. It can be included.

Obtaining object information may include object detection or may be performed without performing object detection.

Processing of object information may include at least one of object detection, noise reduction, signal-to-noise ratio improvement, and definition of a bounding box.

Object information may be received from one or more sources, such as one or more sensors, one or more communication units, one or more memory units, one or more image processors, etc.

Object information can be shared in one or more ways - for example in an absolute way (for example - giving coordinates of the object's position) or in a relative way - for example in relation to a vehicle (for example - an object is stored at a certain distance in relation to the vehicle). (at a certain angle) can be provided.

The vehicle is also called an ego-vehicle.

The detailed description and/or drawings may represent a processor or processing circuitry. A processor may be a processing circuit. The processing circuitry may be implemented as a central processing unit (CPU) and/or one or more other integrated circuits, such as an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), a fully custom integrated circuit, etc., or a combination of these integrated circuits. there is.

Any combination of any steps of any method shown in the description and/or drawings may be provided.

Any combination of any of the subject matter of the claims may be provided.

Any combination of systems, units, components, processors, and sensors illustrated in the description and/or drawings may be provided.

Any reference to an object can be applied to the pattern. Therefore, any reference to object detection can be applied mutatis mutandis to pattern detection.

Successful driving depends on circumventing surrounding road objects based on their positions and movements, but humans are not good at estimating their kinematics. The inventor suspects that humans circumvent the need for kinematic estimation by using internal representations of surrounding objects in the form of virtual force fields that immediately imply action. Consider a scenario where your vehicle is driving in one lane and a vehicle diagonally ahead in an adjacent lane begins to swerve into your lane. The human response to braking or turning is immediate and instinctive and can be experienced as a virtual force pushing the ego away from the turning vehicle. These virtual force representations are learned and associated with specific road objects.

Inspired by the above considerations, the present inventor proposes a novel concept of perception field. The perceptual field is a learned representation of road objects in the form of a virtual force field that is “sensed” through the ego vehicle's control system in the form of ADAS and/or AV software. In the present invention, a field is defined as a mathematical function that varies depending on spatial location (or similar quantity).

One embodiment of inference method 100 is shown in Figure 1 and includes:

Method 100 may be executed for each one or more frames of the vehicle environment.

Step 110 of method 100 may include detecting and/or tracking one or more objects (eg, including one or more road users). Detection and/or tracking may be performed in any manner. The one or more objects may be any object that can affect the operation of the vehicle. For example - road users (pedestrians, other vehicles), the road and/or path on which the vehicle is traveling (e.g. the condition of the road or path, the shape of the road - e.g. curved, straight road segments), traffic signs. , traffic lights, road crossings, schools, kindergartens, etc. Step 110 may include obtaining additional information, such as kinematic and contextual variables related to one or more objects. Acquiring can include receiving or generating. Acquisition may include processing one or more frames to generate kinematic and contextual variables.

It should be noted that step 110 may include acquiring kinematic variables (without acquiring one or more frames).

Method 100 may also include step 120 of obtaining respective perceptual fields associated with one or more objects. Step 120 may include determining which mapping between objects should be retrieved and/or used, etc.

Step 110 (and even step 120) includes step 130 determining one or more virtual forces associated with one or more objects by passing relevant input variables such as perceptual fields (and one or more pseudo-physical model functions), kinematic and contextual variables. It can follow.

Step 130 may be followed by step 140 of determining a total virtual force applied to the vehicle based on one or more virtual forces associated with one or more objects. For example, step 140 may include performing a vector weighted sum (or other function) on one or more virtual forces associated with one or more objects.

Step 140 may be followed by step 150 of determining, based on the total virtual force, a desired (or target) virtual acceleration - for example, based on an equivalent of Newton's second law. The desired virtual acceleration may be vector - or otherwise have a direction.

Step 150 may be followed by step 160 of converting the desired virtual acceleration into one or more vehicle driving operations that will cause the vehicle to propagate along the desired virtual acceleration.

For example, step 160 may include translating a desired acceleration to acceleration or deceleration or changing the direction of travel of the vehicle using gas pedal movement, brake pedal movement, and/or steering wheel angle. The translation may be based on a dynamic model of the vehicle with a specific control scheme.

Advantages of perceptual fields include, for example, explainability, generalizability, and robustness to noisy input.

Explainability. Representing ego movements as configurations of individual perceptual fields means decomposing behavior into its more fundamental components and is itself an important step towards explainability. The possibility to visualize these fields and apply intuitions from physics to predict ego movements represents additional explanatory potential as compared to typical end-to-end black-box deep learning approaches. This increased transparency also allows passengers and drivers to trust AV or ADAS technology more.

Generalizability. Representing one's reaction to an unknown road object as an aversive virtual force field constitutes an inductive bias in an unseen situation. There is a potential advantage to this representation in that it allows edge cases to be handled in a safe manner with little training. Additionally, perceptual field models are holistic in the sense that the same approach can be used for all aspects of driving policy. It can also be divided into narrow driving functions used in ADAS such as ACC, AEB, LCA, etc. Finally, the complex nature of the perceptual field allows the model to be trained on atomic scenarios and still be able to adequately handle more complex scenarios.

Robustness to noisy inputs: Physical constraints on the time evolution of the perceptual field in combination with potential filtering of the input can better handle noise in the input data compared to pure filtering of localization and kinematic data.

Physical or virtual forces allow for mathematical formulation, for example in terms of second-order ordinary differential equations involving so-called dynamical systems. The advantage of representing the control policy in this way is that it is sensitive to the intuitions of dynamical systems theory and that it is simple to integrate external modules such as prediction, exploration and filtering of inputs/outputs.

Another advantage of the perceptual field approach is that it does not rely on any specific hardware and is not computationally more expensive than existing methods.

training process

The process of learning a perceptual field can be one or a combination of two types: behavioral replication (BC) and reinforcement learning (RL). BC approximates a control policy by fitting a neural network to observed human state-action pairs, whereas RL involves learning through trial and error in a simulated environment without reference to expert demonstration.

We can combine the learning algorithms of these two classes by first learning a policy through BC to use it as an initial policy to be fine-tuned using RL. Another way to combine the two approaches is to first learn a so-called reward function (used in RL) through behavioral replication to infer what constitutes desirable behavior for humans, and later use regular RL to Training is done through trial and error. This latter approach goes by the name of inverse RL (IRL).

Figure 2 is an example of a training method 200 used for learning through BC.

Method 200 may begin at step 210 by collecting human data that is considered an expert demonstration of how to handle a scenario.

Step 210 may be followed by step 220 of constructing a loss function that finds the difference between the kinematic variables resulting from the perceptual field model and the corresponding kinematic variables of the human demonstration.

Step 220 is followed by step 230 of updating the parameters of the perceptual field and the auxiliary function (which may be a cyber-physical model function different from the perceptual field) to minimize the loss function by some optimization algorithm such as gradient descent. You can.

Figure 3 is an example of a training method 250 used for reinforcement learning.

Method 250 may begin at step 260 of building a realistic simulation environment.

Step 260 may be followed by step 270, where the reward function is constructed by learning from an expert demonstration or manual design.

Step 270 may be followed by step 280 of executing the episode in a simulation environment and continuing to update the parameters of the perceptual field and auxiliary function to maximize the expected cumulative reward by some algorithm such as proximity policy optimization.

Figure 4 shows one embodiment of method 400.

Method 400 may be for perceptual field navigation related operations.

Method 400 may begin by initializing step 410.

Initialization step 410 may include receiving a group of NNs trained to execute step 440 of method 400.

Alternatively, step 410 may include training a group of NNs to perform step 440 of method 400.

Various embodiments of training NN groups are provided below.

● A group of NNs can be trained to map object information to one or more virtual forces using behavioral cloning.

● A group of NNs can be trained to map object information to one or more virtual forces using reinforcement learning.

● A group of NNs can be trained to map object information to one or more virtual forces using a combination of reinforcement learning and action replication.

● A group of NNs can be trained to map object information to one or more virtual forces using reinforcement learning with a reward function defined using behavioral replication.

● A group of NNs can be trained to map object information to one or more virtual forces using reinforcement learning with an initial policy defined using behavioral replication.

● A group of NNs may be trained to map object information to one or more virtual force and perceptual fields and one or more different virtual physical model functions.

● The NN group may include a first NN and a second NN, where the first NN is trained to map object information to one or more perceptual fields and the second NN is trained to map object information to one or more virtual physical model functions. trained.

The initialization step 410 may lead to a step 420 of obtaining object information about one or more objects located within the vehicle's environment. Step 410 may be repeated multiple times - and the following steps may be repeated multiple times. Object information may include video, images, audio, or other sensed information.

Step 420 may be followed by step 440, which determines one or more virtual forces applied to the vehicle, using one or more neural networks (NNs).

One or more NNs may be an entire NN group (from initialization step 410) or may be part of a NN group, leaving one or more unselected NNs in the group.

One or more virtual forces represent one or more influences of one or more objects on the motion of the vehicle. The impact may be a future impact or a present impact. Impacts can change the vehicle's progress.

One or more virtual forces belong to a virtual physical model. A virtual physical model is a virtual model that can virtually apply physical rules (eg, mechanical rules, electromagnetic rules, optical rules) for vehicles and/or objects.

Step 440 may include at least one of the following steps:

● Calculating the total virtual force acting on the vehicle, based on one or more virtual forces acting on the vehicle.

● Determining the desired virtual acceleration of the vehicle based on the total virtual acceleration exerted on the vehicle by the total virtual force. The desired virtual acceleration may be the same as or different from the total virtual acceleration.

Method 400 may also include at least one of steps 431, 432, 433, 434, 435, and 436.

Step 431 may include determining the status of the vehicle based on the object information.

Step 431 may be followed by step 432, which selects one or more NNs based on the situation.

Additionally or alternatively, step 431 may be followed by step 433 of supplying context metadata to one or more NNs.

Step 434 may include detecting the class of each of one or more objects based on the object information.

Step 434 may be followed by step 435 of selecting one or more NNs based on the class of at least one of the one or more objects.

Additionally or alternatively, step 434 may be followed by step 436 of supplying one or more NNs with class metadata representing a class of at least one of the one or more objects.

Step 440 may be followed by step 450, which performs one or more driving-related operations of the vehicle based on one or more virtual forces.

Step 450 may be executed without human driver intervention and may include changing the speed and/or acceleration and/or direction of travel of the vehicle. This may include performing autonomous driving or advanced driver assistance systems (ADAS), which may include temporarily controlling the vehicle and/or one or more drive-related units of the vehicle. This may include setting the vehicle's acceleration to a desired virtual acceleration, without human driver intervention.

Step 440 may include suggesting to the driver to set the acceleration of the vehicle to a desired virtual acceleration.

5 shows an embodiment of a vehicle. The vehicle includes one or more sensing units 501, one or more driving-related units (e.g., autonomous driving units, ADAS units, etc.), a processor 560 configured to execute any method, and storing instructions and/or method results, functions, etc. It may include a memory unit 508 and a communication unit 504 to do this.

Figure 6 shows embodiments of a method 600 for suboptimal centering RL with suboptimal sample points as input. Lane sample points are located within the vehicle's environment.

RL assumes a simulation environment in which an agent (self-vehicle) generates input data that can implement its learned policies (perceptual fields).

Method 600 may begin by step 610 detecting the nearest lane or side of the road for road sample points (X _L,i ,Y _L,i ) and (X _R,i ,Y _R,i ), where L is the left, R is the right, and index i is the sample point. The speed of the ego vehicle (formerly called vehicle) is denoted by V _ego .

After step ₆₁₀ , the left lane input vectors (X _{L,i ,} Y _{L,i )} and _{V ego are centered} in A focusing step 620 may follow.

Step 620 may be followed by step 630 that calculates the suboptimal perceptual fields f _θ (X _L ) and f _θ (X _R ). This is performed by one or more NNs.

Step 630 may be followed by step 640 of constructing a differential equation describing the ego acceleration applied to the ego vehicle: a = f _θ (X _L ) + f _θ (X _R ).

This may be the output of an inference process. Step 640 may be followed by step 450 (not shown).

The method may include updating one or more NNs. In these cases, RL can assume a reward function that is learned based on expert demonstrations or created by hand. In the embodiment of Figure 6 the reward function may be incremented for every timestamp that the subject vehicle remains in its lane.

The update may include step 670 of implementing in a simulation environment and the RL learning algorithm records what happens at the next time step, including the earned rewards.

Step 670 may include using a particular RL algorithm (e.g., PPO, SAC, TTD3) to sequentially update the network parameter θ to maximize the average reward.

Figure 7 shows a method 700 for multi-object RL with visual input.

Step 710 of method 700 includes receiving a sequence of panoramicly segmented images over a short time window from the ego perspective (image acquired by the ego _vehicle ), the relative distances to individual objects can do.

Step 710 may be followed by step 720, which captures high-level spatio-temporal features (X _i ) by applying a spatio-temporal CNN to individual instances (objects).

Step 720 may be followed by step 730 of calculating the individual perceptual fields f _θ (X _i ,i) and the sum Σf _θ (X _rel,I ,X _i ,i).

Step 730 may be followed by step 740 of constructing a differential equation describing the ego acceleration applied to the ego vehicle: a = Σf _θ (X _rel,I ,X _i ,i).

This may be the output of an inference process. Step 740 may be followed by step 450 (not shown).

The method may include updating one or more network parameters (θ) using some RL process.

The method may include step 760 of implementing in a simulation environment the RL learning algorithm recording what happens at the next time step, including the rewards obtained.

RL can assume reward functions that are learned based on expert demonstrations or created by hand.

Step 760 may be followed by step 770, which sequentially updates the network parameters θ to maximize the average reward using a specific RL algorithm such as PPO, SAC, or TTD3.

Figure 8 shows a method 800 for multi-object BC with kinematic input.

Step 810 of method 800 may include receiving a list of detected object relative kinematics (X _rel,i ,V _rel,i ), where X _rel,i is the detected object i relative to the ego vehicle. is the relative position of and V _rel,i is the relative velocity of the detected object i with respect to the host vehicle. It also receives ego vehicle speed V _ego .

Step 810 may be followed by step 820 of calculating the perceptual field f _θ (X _rel,i ,V _rel,i ,V _ego ,i) for each object.

Step 820 may be followed by step 830, which sums contributions from individual perceptual fields. Step 830 may also include normalizing the magnitude of the resulting 2d vector so that it is equal to the highest magnitude of the individual terms: N*Σf _θ (X _rel,i ,V _rel,i ,V _ego ,i).

Step 830 may be followed by step 840 of constructing a differential equation describing the ego acceleration applied to the ego vehicle: a = N*Σf _θ (X _rel,i ,V _rel,i ,V _ego ,i).

This may be the result of an inference process. Step 840 may be followed by step 450 (not shown).

The method may include updating one or more network parameters.

The method initial conditions If (t;x ₀ ,v ₀ ) is provided, a step 860 of calculating the ego trajectory may be included.

Following step 860, loss function = Σ( Step 870 may follow to calculate (t;x ₀ ,v ₀ ))-x(t;x ₀ ,v ₀ )) ² . This propagates the loss accordingly.

Figure 9 shows an inference method 900 that adds a loss function to an adaptive cruise control model implemented with kinematic variables as input.

Step 910 of method 900 determines the position of the ego vehicle ₍ X _ego ), the velocity of the ego vehicle (V _ego ), the position of the nearest vehicle in front of the ego vehicle ( It may include receiving Vd _CIPV ).

Step 910 may be followed by step 920 of calculating the relative position X _rel = X _ego - X _CIPV and the relative velocity V _rel = V _ego - V _CIPV .

Step 920 may be followed by step 930:

● Calculating, by the first NN, the perceptual field function g _θ (X _rel ,V _CIPV ).

● Calculating the auxiliary function h _ø (V _rel ) by the second NN.

● Multiplying g _θ (X _rel ,V _CIPV ) by h _ø (V _rel ) to give the target acceleration (equal to the target force).

This may be the output of an inference process. Step 930 may be followed by step 450 (not shown).

The method may include updating one or more NN parameters.

The method initial conditions If (t;x ₀ ,v ₀ ) is provided, a step 960 of calculating the ego trajectory may be included.

Following step 960, loss function = Σ( Step 970 may follow to calculate (t;x ₀ ,v ₀ ))-x(t;x ₀ ,v ₀ )) ² . This propagates the loss accordingly.

Consider the vehicle's desired route

The vehicle may perform driving sessions to reach the destination.

At any given moment, the vehicle may be positioned on a segment of the route to reach the destination. These segments are called local segments. Other segments of the path may be further away than the local segment.

A local segment may include a segment of a road on which the vehicle is located, or may include more than a single segment of one or more roads. Segments of a road can be of any length, for example 2-500 meters or longer. Segments of a road can be defined based on the route to the destination. For example, a segment may end when the route required to proceed to another road has ended.

At the given moment, the vehicle knows approximate navigation information. Rough navigation information may describe or provide at least an indication of a local segment of the route.

Coarse navigation information is approximate in the sense that it does not define the exact progression of the vehicle within a local segment. The exact progression of a vehicle within a local segment must be fine-tuned.

For example, rough navigation information may define road segments, but it does not define which lane to proceed in, when to change lanes, etc. As another example, rough navigation information may define road lanes, but does not define how to drive within those lanes.

Fine-tuning may be based on coarse navigation information and may be based on the presence within the vehicle's environment or one or more objects that may affect the vehicle's behavior.

Figure 10 shows an embodiment in which fine adjustment is performed based on rough navigation information.

10 illustrates one embodiment of a method 1000 for virtual field driving related operations.

Method 1000 is a computerized method executed by processing circuitry.

According to one embodiment, method 1000 begins with step 1010 of obtaining approximate navigation information regarding at least a segment of the vehicle's route to the vehicle's destination.

Step 1010 may include receiving coarse navigation information, generating coarse navigation information, participating in processing of coarse navigation information, processing only a portion of coarse navigation information, and/or processing coarse navigation information. It may involve receiving only other parts.

Approximate navigation information may be provided by a vehicle device (such as a multimedia system), by a navigation application such as GOOGLE MAPS ^™ , WAZE ^™ , a navigation application run by the driver device, or by a computer located outside the vehicle. It can be communicated from a localized system, etc.

Step 1010 is followed by step 1020, which uses processing circuitry and determines one or more virtual forces based on approximate navigation information.

According to one embodiment, step 1020 includes step 1020-1 of determining one or more virtual fields representing coarse navigation information. Step 1020-1 may be followed by step 1020-2 (step 1020), which determines one or more virtual forces based on one or more virtual fields.

The one or more virtual forces are for use in applying driving-related operations of the vehicle, and the one or more virtual forces belong to a cyber-physical model and at least partially represent approximate navigation information.

According to one embodiment, coarse navigation information may be represented by one or more virtual fields and a cyberphysical model may provide a mapping between one or more virtual fields and one or more virtual forces.

According to one embodiment, a cyber-physical model may define one or more virtual fields.

According to one embodiment, the determination of one or more virtual forces is responsive to the position of the vehicle.

According to one embodiment, the determination of one or more virtual forces is responsive to movement information of the vehicle.

Step 1020 may apply the same calculations as applied during step 440 of FIG. 4, and may apply calculations to approximate navigation information instead of applying calculations to object information.

Step 1020 may apply calculations similar to those applied during step 440 of FIG. 4 .

According to one embodiment, step 1020 includes analyzing approximate navigation information for the vehicle's location. This may include discovering local segments.

One or more virtual fields may be virtually created from one or more virtual field sources having one or more locations determined based on coarse navigation information. For example, a virtual field source may be located in a local segment of the path. For example, assuming that the approximate navigation information defines a segment in the road over which the vehicle will pass, the virtual field source may be located on that road segment or close to the vehicle. Proximity (e.g. between 2 and 100 meters from the vehicle) or greater distance from the vehicle. Additionally or alternatively, proximity refers to whether a route involves multiple route changes close to each other, whether the vehicle is located within an urban environment, whether the vehicle is located within an array of intersections in close proximity to one another, along a road with intersections where vehicles are sparse. Whether or not the vehicle is driven (for example, each intersection is 5-10 km or more) may be defined by the vehicle's route.

One or more virtual fields may include attractive virtual fields. An attractive virtual field can guide a vehicle to proceed toward a destination.

One or more virtual fields may include repelling virtual fields.

According to one embodiment, the approximate navigation information may also include information about one or more traffic rules to be applied along the route, for example a maximum permitted speed. In these cases, one or more virtual fields may respond to one or more traffic rules. For example, it can be defined taking into account the maximum allowable speed. For example, the size of the virtual field may be defined such that the virtual force representing the virtual field (calculated during step 1030) prevents the vehicle from propagating at a speed that exceeds the maximum allowable speed. Maximum speed limits or any other traffic rule restrictions may be represented by dedicated virtual forces and/or dedicated virtual fields. The maximum rule limit can be enforced by changing the desired virtual acceleration (calculated during step 1030) to provide the desired virtual acceleration, which prevents the vehicle from exceeding the maximum rule limit. In emergency situations, such as when a collision is imminent, changes may be omitted.

According to one embodiment, step 1020 is followed by step 1030, which determines, based on one or more virtual forces, a desired (or target) virtual acceleration of the vehicle, for example based on the equivalent of Newton's second law. The desired virtual acceleration may be vector or otherwise have a direction.

One or more virtual forces may be combined to provide an overall virtual force virtually applied to the vehicle. The total virtual force has the total virtual acceleration.

According to one embodiment - the desired virtual acceleration may have the same magnitude as the overall virtual acceleration - but may be directed in the opposite direction.

According to one embodiment, the desired virtual acceleration may have a size different from the size of the overall virtual acceleration.

According to one embodiment - the desired virtual acceleration may have a direction other than the direction of the overall virtual acceleration.

According to one embodiment - the desired virtual acceleration may be valid for a certain period of time (e.g. between 1-10 seconds, between 2-20 seconds, between 2-60 seconds, up to 2-5 minutes or longer).

According to one embodiment, fine tuning of the vehicle path requires performing multiple determinations of desired virtual accelerations.

According to one embodiment, step 1030 is followed by step 1040 in response to the desired virtual acceleration.

Step 1040 may include at least one of the following:

● Triggering the calculation of one or more vehicle driving operations that cause the vehicle to propagate according to the desired virtual acceleration.

● Triggering the execution of one or more vehicle driving operations that cause the vehicle to propagate according to the desired virtual acceleration.

● Computation of one or more vehicle driving operations that cause the vehicle to propagate according to the desired virtual acceleration.

● Execution of one or more vehicle driving operations that cause the vehicle to propagate according to the desired virtual acceleration.

● Triggering the creation of a user graphical interface for displaying at least one of (i) one or more virtual fields, (ii) one or more virtual forces, or (iii) desired accelerations.

● Creation of a user graphical interface for displaying at least one of (i) one or more virtual fields, (ii) one or more virtual forces, or (iii) desired accelerations.

● By a user graphical interface, a display of at least one of (i) one or more virtual fields, (ii) one or more virtual forces, or (iii) a desired acceleration.

For example, step 1040 may include converting the desired acceleration into acceleration or deceleration of the vehicle or a change in direction of travel of the vehicle using gas pedal movement, brake pedal movement, and/or steering wheel angle.

Steps 1020, 1030, and 1040 may be executed by the same processing circuitry. Alternatively, two or more of steps 1020, 1030, and 1040 may be executed by different processing circuitry. For example, step 1020 is executed by a first processing circuit, step 1030 is executed by a second processing circuit different from the first processing circuit, and step 1040 is executed by a third processing circuit different from the first and second processing circuits, respectively. It is executed by. In another embodiment, two of steps 1020, 1030, and 1040 are executed by the same processing circuitry and other steps of steps 1020, 1030, and 1040 are executed by different processing circuitry.

11 illustrates one embodiment of a method 1001 for virtual field driving related operations.

Method 1001 is a computerized method implemented by processing circuitry.

Method 1001 includes steps 1020, 1030, and 1040.

Method 1001 differs from method 1000 of FIG. 10 in that it further includes step 1025.

Step 1025 follows step 1020 and includes at least in part the creation of a user graphical interface for displaying one or more virtual fields. Step 1025 includes triggering the creation of a user graphical interface, creating a user graphical interface, and performing some of the steps necessary to provide the user graphical interface (e.g., displaying the user graphical interface once executed). may include determining some of the commands that trigger it.

12 illustrates one embodiment of a method 1002 for virtual field driving related operations.

Method 1002 of FIG. 12 differs from method 1000 of FIG. 10 in that it considers object information regarding one or more objects located within the vehicle's environment.

According to one embodiment, method 1002 includes step 1010 of obtaining coarse navigation information regarding at least a segment of the vehicle's route to the vehicle's destination.

According to one embodiment, method 1002 includes step 1012 of obtaining object information regarding one or more objects located within the environment of the vehicle.

Steps 1010 and 1012 lead to step 1022, which uses processing circuitry and determines one or more virtual forces based on coarse navigation information and based on object information. One or more virtual forces represent coarse navigation information and object information.

According to one embodiment, step 1022 includes step 1022-1 of determining one or more virtual fields representing coarse navigation information and object information. Step 1022-1 may be followed by step 1022-2 (step 1022), which determines one or more virtual forces based on one or more virtual fields and based on object information.

Step 1022 may also be based on location information associated with the vehicle and/or movement information associated with the vehicle.

According to one embodiment, step 1022 is followed by step 1030, which determines a desired (or target) virtual acceleration of the vehicle, based on one or more virtual forces.

One or more virtual forces represent the influence of coarse navigation information and object information on the vehicle's behavior.

The virtual force may be determined based on object information and approximate navigation information.

The virtual force may be determined based on approximate navigation information rather than based on object information.

The virtual force may be determined based on approximate navigation information rather than object information.

According to one embodiment, step 1030 is followed by step 1040 in response to the desired virtual acceleration.

Figure 13 shows one example of a scene.

13 shows one embodiment of a vehicle 1031 located within segment 1011-1 of a first roadway 1011. Approximate navigation information is displayed as a dotted line 1111 passing the first road 1011 and then proceeding to the second road 1102.

A virtual field source 1201 representing approximate navigation information generates a virtual field (illustrated by virtual equipotential field lines 1021') that attracts the vehicle 1031 to proceed along a path associated with the dashed line 1101. do.

Figure 13 also shows the desired virtual acceleration 1041 that the vehicle 1031 should follow.

Figure 14 shows one embodiment of a vehicle 1031 located within a segment 1011-1 of a first roadway. Approximate navigation information is displayed as a dotted line 1111 passing through the first road 1011.

A first virtual field source 1201 representing approximate navigation information is a virtual field (virtual equipotential field line 1021') that pushes the vehicle 1031 and causes the vehicle 1031 to proceed along a path associated with the dashed line 1101. (shown as) is generated.

Figure 14 also shows a pedestrian 1022 starting to cross the segment in front of a vehicle 1301. The pedestrian is represented by a second virtual field (illustrated by virtual equipotential field line 1022') pushing the vehicle 1031.

Figure 14 also shows the desired virtual acceleration 1041 that the vehicle 1031 should follow.

The first and second virtual fields are taken into account when calculating the total virtual force virtually applied to the vehicle. The desired virtual acceleration 1041 of vehicle 1031 takes into account the total virtual force.

The desired virtual acceleration 1041 of vehicle 1031 may be calculated based on the first and second virtual fields, with or without calculation of the total virtual force.

In the foregoing detailed description of the invention, the invention has been described with reference to specific examples of embodiments of the invention. However, it will be apparent that various modifications and changes may be made without departing from the scope and broader spirit of the invention as asserted in the appended claims. Additionally, in the description of the invention and the patent claims, terms such as “front,” “rear,” “top,” “bottom,” “top,” and “bottom” are used for descriptive purposes and necessarily describe permanent relative positions. It's not meant to be done. It is to be understood that the terms so used are interchangeable in appropriate contexts so that the embodiments of the invention described herein may operate in different directions, for example, than those illustrated or otherwise described herein. Additionally, the terms "assert" or "set" and "negation" (or "deassert" or "clear") are used herein to refer to a signal, status bit, or similar device as logically true or logically true. It is used to refer to rendering each in a false state. A logically true state is logic level 1, and a logically false state is logic level 0. And if a logically true state is logic level 0, then a logically false state is logic level 1. Those skilled in the art will recognize that the boundaries between logical blocks are merely examples and that alternative embodiments may merge logical blocks or circuit elements or impose alternative decompositions of functionality on various logical blocks or circuit elements. Accordingly, it will be appreciated that the architecture depicted herein is merely illustrative and that many other architectures may be implemented in practice that achieve the same functionality. Any arrangement of components to achieve the same function is effectively “related” so that the desired function is achieved. Accordingly, any two components combined in the present invention to achieve a particular function may be viewed as “related” to each other such that the desired function is achieved, regardless of architecture or intermediate components. Likewise, any two such related components may be viewed as being “operably connected” or “operably coupled” to each other to achieve the desired functionality. Additionally, those skilled in the art will recognize that the boundaries between the foregoing operations are illustrative only. Multiple operations may be combined into a single operation, a single operation may be distributed into additional operations and the operations may be executed at least partially temporally overlapping. Moreover, alternative embodiments may include multiple instances of a particular operation, and the order of operations may vary in various other embodiments. Also for example, in one embodiment, the illustrated embodiments may be implemented on a single integrated circuit or as a circuit located within the same device. Alternatively, embodiments may be implemented as any number of separate integrated circuits or individual devices interconnected with each other in any suitable manner. However, other modifications, variations and alternatives are also possible. Accordingly, the specification and drawings are to be regarded in an illustrative rather than a restrictive sense. In the claims, any reference signs between parentheses should not be construed as limiting the scope of the claims. The word 'comprising' does not exclude the presence of elements or steps other than those listed in the claims. Additionally, the term “one” or “one” used in this specification is defined as one or more than one. Additionally, the use of introductory phrases such as "at least one" and "one or more" in a claim should not be construed as limiting a particular claim, nor should the introduction of other claim elements by the indefinite article "one" or "an". , for inventions that contain only one such element in a claim containing such introduced claim element, if the same claim contains the preface "one or more" or "at least one" and an indefinite article such as "an" or "one" The same is true in this case. The same goes for the use of definite articles. Unless otherwise specified, terms such as “first” and “second” are used to optionally distinguish between the elements described by such terms. Accordingly, these terms are not necessarily intended to indicate the temporal or other priority of these elements. The mere fact that certain measures are recited in different claims does not indicate that a combination of these measures cannot be used to advantage. Although certain features of the invention have been illustrated and described herein, many modifications, substitutions, changes and equivalents will now occur to those skilled in the art. Accordingly, the appended claims are to be construed as including all such modifications and changes as fall within the true spirit of the invention. It is understood that various features of embodiments of the disclosure that are described in the context of separate embodiments for the sake of clarity may also be provided in combination in a single embodiment. Conversely, various features of embodiments of the disclosure that are described in the context of a single embodiment for the sake of brevity may also be provided individually or in any suitable sub-combination. Those skilled in the art will understand that the embodiments of the invention are not limited by what has been specifically shown and described above. Rather, the scope of embodiments of the invention is defined by the appended claims and their equivalents.

Claims (20)

A method for virtual field driving-related operations, said method comprising:
obtaining approximate navigation information regarding at least a segment of the vehicle's route to the vehicle's destination; and
determining, using processing circuitry and based on the approximate navigation information, one or more virtual forces, wherein the one or more virtual forces are for use in applying drive-related operations of the vehicle, the one or more virtual forces being virtual physical forces; belonging to a model, and at least partially representing said coarse navigation information. The method of claim 1, comprising analyzing coarse navigation information about the location of the vehicle. The method of claim 1, wherein determining the one or more virtual forces includes obtaining one or more virtual fields associated with the coarse navigation information. 4. The method of claim 3, wherein obtaining the one or more virtual fields leads to determining the one or more virtual forces based on the one or more virtual fields. 4. The method of claim 3, wherein the one or more virtual fields comprise an attractive virtual field. 4. The method of claim 3, wherein the one or more virtual fields include rappelling virtual fields. 4. The method of claim 3, comprising creating a user graphical interface for displaying the one or more virtual fields. The method of claim 1, comprising obtaining object information regarding one or more objects located within the environment of the vehicle during the driving session. 9. The method of claim 8, wherein determining the one or more virtual forces is based on object information, and the one or more virtual forces are indicative of one or more effects of the one or more objects on the operation of the vehicle. The method of claim 8, wherein the one or more virtual forces are multiple virtual forces including a virtual force representing the influence of an object among the one or more objects and a virtual force representing the coarse navigation information. 9. The method of claim 8, wherein determining the one or more virtual forces is based on at least one of location information associated with the vehicle and motion information of the vehicle. A non-transitory computer-readable medium storing instructions that, once executed by processing circuitry, cause the processing circuitry to:
obtain approximate navigation information regarding at least a segment of the vehicle's route to the vehicle's destination; and
Using the processing circuitry and based on the coarse navigation information, determine one or more virtual forces, the one or more virtual forces for use in applying a driving-related operation of the vehicle, the one or more virtual forces The force belongs to a cyber-physical model and represents at least in part the approximate navigation information. 13. The non-transitory computer-readable medium of claim 12, wherein determining the one or more virtual forces includes obtaining one or more virtual fields associated with coarse navigation information. 14. The non-transitory computer-readable medium of claim 13, wherein obtaining the one or more virtual fields results in determining the one or more virtual forces based on the one or more virtual fields. 14. The non-transitory computer-readable medium of claim 13, wherein the one or more virtual fields comprise an attractive virtual field. 14. The non-transitory computer-readable medium of claim 13, storing instructions for creating a user graphical interface for displaying the one or more virtual fields. 13. The non-transitory computer-readable medium of claim 12, storing instructions for obtaining object information about one or more objects located within the environment of the vehicle during a driving session. 18. The non-transitory computer-readable medium of claim 17, wherein determining the one or more virtual forces is based on the object information, and the one or more virtual forces are indicative of one or more effects of the one or more objects on operation of the vehicle. The non-transitory computer-readable medium of claim 17, wherein the one or more virtual forces are multiple virtual forces including a virtual force representing an impact of one of the one or more objects and a virtual force representing approximate navigation information. 18. The non-transitory computer-readable medium of claim 17, wherein determining the one or more virtual forces is based on at least one of location information associated with the vehicle and motion information associated with the vehicle.