TW202231089A - Vehicle-to-everything (v2x) misbehavior detection using a local dynamic map data model - Google Patents

Abstract

Embodiments include methods performed by a processor of a vehicle-to-everything (V2X) system within a vehicle for detecting misbehavior conditions by comparing information received in V2X messages to local dynamic map data. Various embodiments may include receiving V2X messages from other V2X system participants, determining whether a misbehavior condition is detected by comparing data contained in the received V2X messages to information in a locally maintained or stored local dynamic map data model, detecting a misbehavior condition and generating a misbehavior report identifying the misbehavior condition in response to a conflict or inconsistency between some data in the received V2X message and the local dynamic map.

Description

Vehicle-to-everything (V2X) misbehavior detection using local dynamic map data model

cross reference

This patent application claims priority to U.S. Provisional Application No. 63/138,909, filed on January 19, 2021, entitled "Vehicle-to-Everything (V2X) Misbehavior Detection Using an LDM Data Model," for The entire contents of which are incorporated herein by reference for all purposes.

This disclosure is about the detection of misbehavior in the Internet of Vehicles (V2X) using the local dynamic map data model.

The Cellular Vehicle-to-Vehicle (C-V2X) protocol serves as the basis for vehicle-based wireless communications and can be used to support smart highways, autonomous and semi-autonomous vehicles, and improve the overall efficiency and safety of road transportation systems. C-V2X defines two transmission modes that together provide 360° non-line-of-sight perception and higher levels of predictability to enhance road safety and autonomous driving. The first transmission mode includes direct C-V2X, which includes vehicle-to-vehicle (V2V), vehicle-to-infrastructure (V2I), and vehicle-to-pedestrian (V2P), and in dedicated Intelligent Transportation Systems (ITS) 5.9 Provides enhanced communication range and reliability in the gigahertz (GHz) spectrum. The second transmission mode includes vehicle-to-network communication (V2N) in mobile broadband systems and technologies, such as third generation wireless mobile communication technology (3G) (eg, Global System for Mobile Communications (GSM) Evolution (EDGE) system, code division multiple access (CDMA) 2000 system, etc.), fourth-generation wireless mobile communication technology (4G) (for example, long-term evolution (LTE) system, advanced LTE system, global interoperability for microwave access (mobile WiMAX) system, etc.), Fifth-generation wireless mobile communication technology (5G NR system, etc.), etc. Other V2X wireless technologies are also being considered in different parts of the world. The techniques described in this patent are applicable to any V2X wireless technology.

Standards for vehicle-based communication systems and functions are being developed in several parts of the world, such as the Institute of Electrical and Electronics Engineers (IEEE) Standard 1609 and the Society of Automotive Engineers (SAE) standards for North America, or European Telecommunications for Europe Standards Institute (ETSI) and European Committee for Standardization (CEN) standards. Part of the system enables vehicles to broadcast Basic Safety Messages (BSM) in North America or Cooperative Awareness Messages (CAM) in Europe, which other vehicles can receive and process to improve traffic safety. The processing of sending and receiving such messages in the vehicle takes place in the in-vehicle equipment that provides V2X functionality (herein referred to as "V2X in-vehicle equipment").

Aspects include methods performed by a misconduct management system executing on a V2X device processor to model data through local dynamic map (LDM) data contained in received V2X messages and maintained or stored locally to detect misconduct in received V2X messages. The LDM aggregates and synthesizes information received by V2X system participants from all relevant inputs (including but not limited to V2X messages and local sensor inputs) to model the local environment around the V2X system participants. The LDM can be updated based on the observed dynamics of objects tracked in the LDM and based on new inputs.

Various aspects may include receiving a V2X message from another V2X system participant, where the V2X message includes data about the environment surrounding the vehicle, and correlating the data included in the received V2X message with a local dynamic map maintained or stored locally ( LDM) data models are compared to detect a misconduct condition, in response to detecting a misconduct condition based on the comparison results, generating a misconduct report identifying the misconduct condition, and sending the generated misconduct report to misconduct management mechanism.

Certain aspects may include monitoring a plurality of sensors in a vehicle to collect additional data about the environment surrounding the vehicle, generating LDM data representing the environment surrounding the vehicle based at least in part on aggregation of the additional data collected from the plurality of sensors model, and maintain or store the LDM data model in local memory.

Certain aspects may also include, in response to determining that a misconduct condition was not detected: performing calculations based on at least one of observed object dynamics in the LDM data model or new data inputs received from V2X messages; modifying the LDM data model To merge computations and data included in received V2X messages; and to replace LDM data models maintained or stored in memory with modified LDM models.

In some aspects, sending the generated misconduct report to the misconduct management agency may include sending a representation of the LDM data model.

In some aspects, the representation of the LDM data model may include an incomplete data set for the LDM data model.

Certain aspects may also include receiving feedback from the misconduct management agency, wherein the feedback includes corrective actions for mitigating the misconduct condition.

In some aspects, the data about the surroundings of the vehicle included in the received V2X message includes traffic information. In some aspects, the data about the environment surrounding the vehicle included in the received V2X message includes location information of neighboring vehicles based on Global Navigation Satellite System (GNSS) data (eg, Global Positioning System (GPS)) data. In some aspects, the data about the surroundings of the vehicle included in the received V2X message includes map data detailing road geometry and street furniture.

In some aspects, comparing the data included in the received V2X message with an LDM data model maintained or stored by the local end to detect misconduct conditions may include determining the data included in the received V2X message Whether any data conflicts with the information in the LDM data model maintained or stored by the local end.

In some aspects, comparing the data included in the received V2X message with the LDM data model maintained or stored by the local to detect misconduct conditions may be included in the LDM data model maintained or stored by the local Select a subset of data elements for comparison with the data included in the received V2X message; determine whether any data included in the received V2X message is consistent with the selected data in the LDM data model maintained or stored at the local end Subset conflict of data element.

In some aspects, comparing the data included in the received V2X message with an LDM data model maintained or stored by the local end to detect misconduct conditions may include determining that the received V2X message was sent. Whether the status or location information of adjacent vehicles conflicts with the status or location information of adjacent vehicles in the LDM data model maintained or stored by the local end.

Other aspects may include V2X equipment having a processor configured to perform one or more operations of the methods described above. Other aspects may include a non-transitory processor-readable storage medium having stored thereon processor-executable instructions configured to cause a processor of the V2X equipment to perform the operations of the above-described methods. Other aspects include V2X equipment having means for performing the functions of the methods described above.

Various embodiments will be described in detail with reference to the accompanying drawings. Wherever possible, the same reference numbers will be used throughout the drawings to refer to the same or like parts. References to specific examples and implementations are for illustrative purposes and are not intended to limit the scope of the claimed items.

In V2X communication, it is important to detect inaccurate, corrupted or compromised (i.e. bad) data to prevent further dissemination of such inaccurate data. If V2X equipment sends inaccurate, damaged or attacked (i.e. bad) data, the consequences may be only minor inconvenience and traffic jams, but can also be life-threatening. Therefore, it is desirable for the detection of misconduct conditions to perform a rigorous analysis of a comprehensive set of information to ensure that any such misconduct conditions are indeed detected.

The term "mobile device" is used herein to refer to wireless router devices, wireless appliances, cellular telephones, smart phones, portable computing devices, personal or mobile multimedia players, laptop computers, tablet computers, smart computers , Ultrabooks, Palmtops, Wireless Email Receivers, Multimedia Networked Cell Phones, Medical Devices and Equipment, Biometric Sensors/Devices, Wearables (including Smart Watches, Smart Clothing, Smart Glasses, Smart Wrists) belts, smart jewelry (e.g., smart rings, smart bracelets, etc.), entertainment devices (e.g., wireless game controllers, music and video players, satellite radios, etc.), wireless-enabled Internet of Things (IoT) devices (including smart meters/sensors, industrial manufacturing equipment, large and small machinery and electrical appliances for home or business use), wireless communication components in automatic and semi-automatic vehicles, mobile devices fixed or integrated into various mobile platforms, Any or all of global positioning system devices, and similar electronic devices including memory, wireless communication elements, and programmable processors.

The term "system on a chip" (SOC) is used herein to refer to a single integrated circuit (IC) die that includes multiple resources and/or processors integrated on a single substrate. A single SOC may contain circuitry for digital, analog, mixed-signal, and RF functions. A single SOC may also include any number of general-purpose and/or special-purpose processors (digital signal processors, modem processors, video processors, etc.), storage blocks (eg, ROM, RAM, flash memory, etc.) and resources (eg, timers, voltage regulators, oscillators, etc.). The SOC may also include software for controlling integrated resources and processors and for controlling peripheral devices.

The term "system-in-package" (SIP) may be used herein to refer to a single module or a package. For example, a SIP may include a single substrate on which multiple IC dies or semiconductor dies are stacked in a vertical configuration. Similarly, a SIP may include one or more multi-die modules (MCMs) on which multiple ICs or semiconductor dies are packaged into a unitary substrate. A SIP may also include multiple independent SOCs coupled together via high-speed communication circuitry and tightly packaged, such as on a single motherboard or in a single mobile device. The proximity of the SOC facilitates high-speed communication and the sharing of memory and resources.

As used in this case, the terms "component," "system," "unit," "module," etc. include computer-related entities such as, but not limited to, hardware, firmware, combinations of hardware and software, software , or software in execution that is configured to perform a specific operation or function. For example, an element may be, but is not limited to, a program running on a processor, a processor, an object, an executable, a program of execution, a program, and/or a computer. By way of illustration, both an application running on a communication device and the communication device may be referred to as components. One or more elements may be resident within a program and/or thread of execution, and an element may be localized on one processor or core and/or distributed between two or more processors or cores. In addition, the elements can execute from various non-transitory computer readable media having various instructions and/or data structures stored thereon. Components may communicate via local and/or remote procedures, function or procedure calls, electronic signals, data packets, memory read/write, and other known computer, processor and/or program related communication methods.

In general, various embodiments include methods and mechanisms for a V2X system participant to determine which V2X message is received in the V2X message by comparing the received V2X message to the vehicle's Local Dynamic Map (LDM) data model. Whether there is any inconsistency between the data and the LDM data model maintained or stored at the local end to detect misconduct.

V2X systems and technologies hold great promise for improving traffic flow and vehicle safety by enabling vehicles to share information about their position, speed, direction of travel, braking and other factors that may be useful to other vehicles for collision avoidance and other safety features . Vehicles equipped with V2X/V2V onboard equipment will frequently (eg up to 20 times per second) send their vehicle information in packets called Basic Safety Messages (BSMs) or CAMs. Since all V2X-equipped vehicles send such BSM/CAM messages, all receiving vehicles have the information they need to control their speed and direction to avoid collisions and to efficiently and safely position vehicles relative to each other. It is envisioned that V2X-equipped vehicles can improve traffic flow by safely reducing separation distances, platooning several vehicles together, and avoiding vehicle breakdowns.

For ease of reference, some embodiments are described in this case using a misconduct management system implemented in V2X terminology. It should be understood, however, that the various embodiments encompass any or all V2X/V2V or vehicle-based communication standards, messages or technologies. As such, nothing in this case should be construed to limit the claim to V2X/V2V systems unless expressly stated in the claim. Furthermore, the embodiments described herein discuss in-vehicle equipment that performs V2X/V2V communications. In a V2X/V2V system, system participant equipment may include, but is not limited to, in-vehicle equipment, mobile equipment, and roadside units (RSUs). The RSU may include fixed equipment such as traffic signals, roadside lights, traffic cameras, and the like. Each system participant device can broadcast information to other system participant devices. V2X communication between system participant devices can allow applications executing on each system participant device to provide safety applications for the vehicle (for example, applications that can determine imminent hazards, such as the vehicle braking hard or speeding out of the way) blind lanes) or mobility (planning of traffic signal changes), or provide other useful functions throughout the vehicle transportation system.

Local Dynamic Map (LDM) is a data model typically built by mobile devices to support navigation in their environment. A mobile device obtains information about its environment from one or more sensors, and may receive other LDM data from other mobile devices (eg, via a V2X communication system) or from network elements such as cloud-based servers, and use This information constructs its LDM. The LDM can be a dynamic data model that changes over time, even though no new data is received to update the positions of other V2X system participants via dead reckoning rather than messages from their other V2X system participants. The LDM data model can aggregate and synthesize information received by V2X system participants from all relevant inputs (including but not limited to V2X messages and sensor inputs) to model the local environment around the V2X system participants. The LDM can be updated based on observed dynamics of objects tracked in the LDM as well as based on new inputs.

The misconduct management system implemented on the V2X participant's equipment may be via aggregation from one or more sensors of the host vehicle (eg, cameras, radar, lidar, etc.), from one or more other mobile devices, or via V2X messages are received by the vehicle and/or from remote data sources and network elements, such as cloud-based servers, such as information obtained via roadside units to construct the LDM. The vehicle V2X system can process this information to generate and update locally maintained or stored LDM data in a usable or presentable form (such as a digital map). Portions of the LDM map may also be received from external sources, such as computing devices capable of performing intensive processing operations. Such LDM data models may include various types of information, which may be structured or organized in multiple layers or data elements. For example, an LDM data model may include a physical map of a road, such as a physical map of a road downloaded from a map database, a data layer of observed road conditions (eg, rough or flat, wet, dry, or icy, etc.), observed Layers of data on the position and speed of other vehicles, data layers on which the network reports road changes (e.g., construction, closed lanes), layers of data on nearby traffic signals (e.g., the light cycle time of traffic lights ahead of this vehicle), and Additional information useful for autonomous driving, collision avoidance, and general safety features such as driver alerts.

Information used by misconduct management systems is generally limited to data that can be maintained or stored in memory (eg, static maps) and data from onboard sensors. LDM data received from in-vehicle sensors and other mobile devices may be limited by the sensitivity, field of view, and perception limitations of each sensor. LDM data received from remote network elements typically do not include recent changes in the environment near the mobile device vehicle and therefore may not reflect highly dynamic environmental conditions (eg, road closures, construction, accidents, etc.). By combining all sources of information about the road ahead of the vehicle and other nearby vehicles, vehicle systems (eg, misconduct management systems) can produce more comprehensive LDM data models that can be used in complex programs such as autonomous and semi-autonomous driving assistance tools.

LDM data models can be structured into various types to reflect the extent to which such information may change dynamically. For example, LDM data can be classified (eg, in the relevant ETSI standard) as: Type 1 for permanent static information, such as road locations and geographic features, which can be considered map data; Type 2 for transient static information, This can include signals not included in map data, such as speed limits; Type 3 for transient dynamic information, such as weather and traffic congestion and other traffic condition information; and Type 4 for highly dynamic information, such as car sensor data , the location of other moving vehicles, pedestrians, parked vehicles, the status of traffic signals, and other highly transient conditions. Examples of reported LDM implementations include Bosch™'s PG-LDM and Tele Atlas™ and NAVTEQ™'s NAVTEQ-LDM. The PG-LDM implementation uses PostgreSQL as its database engine and provides PostGIS stored procedures and spatial operations. Meanwhile, the NAVTEQ-LDM implementation uses SQLite as its database engine.

In various embodiments, the misconduct management system executing on the V2X equipment processor may receive the first LDM data from one or more data sources other than V2X system participants, which data sources may include adjacent vehicles, mobile devices, and RSU, can send CAM messages or Decentralized Environmental Notification Message (DENM) messages, and various Internet-based or cloud-based data resources. In some embodiments, the first LDM data received may be Type 4 information, or "highly dynamic" information reflecting highly transient conditions. In some embodiments, the received LDM data may be obtained from a sensor or another information source within a threshold amount of time, such as two seconds, one second, 250 milliseconds, or another suitable threshold or time window. In some embodiments, the first LDM data may include data collected by a plurality of sensors mounted on the vehicle and mobile device. Such sensor data may include data such as speed, temperature, revolutions per minute, GPS location, image data, audio data, or vehicle/equipment operating status data. The misconduct management system can aggregate all collected sensor data and received V2X message data to generate an LDM data model representing the V2X participant's surroundings. For ease of reference, various embodiments are described by referring to the environment around the V2X participant as the environment around the vehicle; however, the V2X participant may be equipment other than the vehicle, such as RSUs and other stationary equipment.

The LDM data model generated in this way can be used to assess the accuracy or authenticity of the information included in the received V2X message. Because the LDM data model may consist of data received from various information sources, it may include one or more data elements related to the information provided in the V2X message. However, not all the information included in the LDM data model will be relevant to the information in the V2X message to be verified or verified.

Various embodiments may be implemented in various vehicles, an example vehicle 101 of which is shown in FIGS. 1A and 1B . 1A and 1B, the vehicle 101 may include a control unit 140 and a plurality of sensors 144-170, including a satellite geolocation system receiver 142, occupancy sensors 144, 146, 148, 150, 152, tire pressure sensing Sensors 154 , 156 , cameras 158 , 160 , microphones 162 , 164 , impact sensor 166 , radar 168 , and lidar 170 . The plurality of sensors 144-170 disposed in or on the vehicle may be used for various purposes, such as automatic and semi-automatic navigation and control, collision avoidance, location determination, etc., as well as providing information about objects in or on the vehicle 101 and Human sensor data. Sensors 144-170 may include one or more of a variety of sensors capable of detecting a variety of information useful for navigation and collision avoidance. Each of the sensors 144-170 may be in wired or wireless communication with the control unit 140 and each other. In particular, the sensors may include one or more cameras 158, 160 or other optical or photoelectric sensors. The sensors may also include other types of object detection and ranging sensors, such as radar 168, lidar 170, IR sensors, and ultrasonic sensors. Sensors may also include tire pressure sensors 154, 156, humidity sensors, temperature sensors, satellite geolocation sensor 142, control input sensor 145, accelerometers, vibration sensors, gyroscopes , gravimeter, impact sensor 166, dynamometer, stress gauge, strain sensor, fluid sensor, chemical sensor, gas content analyzer, pH sensor, radiation sensor, Geiger counter , neutron detectors, biomaterial sensors, microphones 162, 164, occupancy sensors 144, 146, 148, 150, 152, proximity sensors, and other sensors.

The vehicle control unit 140 may be configured with processor-executable instructions to perform navigation and collision avoidance operations using information received from various sensors, particularly the cameras 158 , 160 . In some embodiments, control unit 140 may supplement processing of the camera image with distance and relative position (eg, relative position angle) that may be sensed from radar 168 and/or lidar 170 device obtained. The control unit 140 may also be configured to control the steering, braking and speed of the vehicle 101 when operating in an automatic or semi-automatic mode using information regarding other vehicles as determined using various embodiments.

FIG. 1C is a block diagram of components illustrating a communication system 100 suitable for implementing components and supporting systems of various embodiments. Referring to FIGS. 1A-1C , the vehicle 101 may include a control unit 140 , which may include various circuits and devices for controlling the operation of the vehicle 101 . In the example shown in FIG. 1D, the control unit 140 includes a processor 140a, a memory 140b, an input module 140c, an output module 140d, and a radio module 140e. Control unit 140 may be coupled to and configured to control drive control elements 172 a , navigation elements 172 b and one or more sensors 172 c of vehicle 101 . The processor 140a may be configured with processor-executable instructions to control steering, navigation, and/or other operations of the vehicle 101 , including operations of various embodiments. The processor 140a may be coupled to the memory 140b.

The radio module 140e may be configured for wireless communication. Radio module 140e may exchange signals (eg, command signals for control maneuvers, signals from navigation aids, etc.) with a network transceiver (eg, base station 110 ) via communication link 122 and may provide signals to processing controller 140a and/or navigation unit 172b. In some embodiments, the radio module 140e may enable the vehicle 101 to communicate with the wireless communication device 120 via the wireless communication link 124 . As described, the wireless communication link 124 may be a bidirectional or unidirectional communication link, and may use one or more communication protocols.

The input module 140c may receive sensor data from one or more vehicle sensors 172c, as well as electronic signals from other elements, including the drive control element 172a and the navigation element 172b. The output module 140d may communicate with or activate various elements of the vehicle 101, including the drive control element 172a, the navigation element 172b, and the sensor 172c.

The control unit 140 may be coupled to the drive control elements 172a to control physical elements of the vehicle 101 such as engines, motors, throttles, steering elements, flight control elements, braking or retarding elements, etc. related to the steering and navigation of the vehicle. Drive control elements 172a may also include elements that control other equipment of the vehicle, including environmental control equipment (eg, air conditioning and heating equipment), exterior and/or interior lighting equipment, interior and/or exterior information displays (which may include display screens or other devices that display information), security devices (e.g., haptic devices, audible sirens, etc.), and other similar devices.

The control unit 140 may be coupled to the navigation element 172b, and may receive data from the navigation element 172b and be configured to use such data to determine the current position and orientation of the vehicle 101, and an appropriate route towards the destination. The navigation element 172b may include or be coupled to a GNSS receiver system (eg, one or more GPS receivers), enabling the vehicle 101 to determine its current location using GNSS signals. Alternatively or additionally, the navigation element 172b may include a radio navigation receiver for receiving navigation information from radio nodes, such as Wi-Fi access points, cellular web sites, radio stations, remote computing devices, other vehicles, and the like. mark or other signal. Via control of drive control element 172a, processor 140a may control vehicle 101 for navigation and maneuvering. The processor 140a and/or the navigation element 172b may be configured to communicate via wireless communication links 122, 126 with network elements such as servers in a communication network (eg, core network 132) to receive commands for control manipulation, Receive data useful for navigation, provide instant location reports, and evaluate other data.

The control unit 140 may be coupled to one or more sensors 172c. Sensor 172c may include sensors 144-170 as described, and may be configured to provide various data to processor 140a.

Although control unit 140 is described as including separate components, in some embodiments some or all of the components (eg, processor 140a, memory 140b, input module 140c, output module 140d, and radio module) 140e) may be integrated in a single device or module, such as a system-on-chip (SOC) processing device. Such an SOC processing device may be configured for use in a vehicle and configured with processor-executable instructions, such as executing in processor 140a, to perform navigation and collision avoidance operations using the LDM profile when installed in the vehicle.

FIG. 1D illustrates a portion of a V2X system 103 including three vehicles 12 , 14 , 16 . In the example shown, each vehicle 12, 14, 16 includes V2X onboard equipment 102, 104, 106, respectively, that is configured to periodically broadcast basic safety messages 30, 40, 50 for onboard equipment of other vehicles (eg, 102, 104, 106) receive and process. By sharing vehicle location, speed, direction, braking and other information, vehicles can maintain safe distances and identify and avoid potential collisions. For example, a trailing vehicle 12 receiving a basic safety message 40 from a preceding vehicle 16 may determine the speed and position of the vehicle 16 so that the vehicle 12 can match the speed and maintain a safe separation distance 20 . By being notified via the basic safety message 40 when the preceding vehicle 16 applies the brakes, the V2X equipment 102 in the trailing vehicle 12 can simultaneously apply the brakes to maintain a safe separation distance 20 even when the preceding vehicle 16 suddenly stops. As another example, the V2X equipment 104 within the truck vehicle 14 may receive basic safety messages 30, 50 from the two vehicles 12, 16 and thus be told that the truck vehicle 14 should stop at the intersection to avoid a collision. Each of the vehicle V2X onboard equipment 102 , 104 , 106 can communicate with each other using any of a variety of short-range communication protocols. In addition, the vehicle can transmit data and information about detected basic safety messages and detected misconduct reports to OEMs via communications network 18 (eg, cellular, WiFi, etc.) via communications links 60 , 62 (OEM) (70, 72) and/or Remote Misconduct Authority 74. Misconduct reports (MBRs) may be sent directly to the misconduct management agency 74 (eg, via communication links 64 , 66 ). In other embodiments, the MBR may first be sent via communication links 64, 66 to an MBR preprocessing unit such as OEM servers 70, 72 for preprocessing. The pre-processed MBR may then be sent from the MBR pre-processing servers 70 , 72 to the misconduct management authority 74 via the communication links 64 , 66 .

FIG. 2A is an element block diagram illustrating elements of an example misconduct management system 200 . The vehicle management system 200 may include various subsystems, communication elements, computing elements, computing devices or units that may be used within the vehicle 101 . 1A-2A, various computing elements, computing devices, or units within misconduct management system 200 may be implemented within a system of interconnected computing devices (ie, subsystems) that communicate information and commands to each other (eg, by arrows in Figure 2A). In some embodiments, the various computing elements, computing devices or units within the misconduct management system 200 may be implemented within a single computing device, such as separate threads, programs, algorithms or computing elements. Accordingly, each of the subsystems/computing elements shown in FIG. 2A may also generally be referred to herein as a "layer" within the computing "stack" that constitutes the misconduct management system 200. However, the use of the terms layer and stack in describing various embodiments is not intended to imply or require implementation of the corresponding functionality within a single automated (or semi-autonomous) vehicle management system computing device, although this is one potential implementation example. Rather, use of the term "layer" is intended to encompass subsystems with independent processors, computational elements (eg, threads, algorithms, subroutines, etc.) executing in one or more computing devices, as well as subsystems and computations combination of elements.

The misconduct management system stack may include radar perception layer 202, camera perception layer 204, location engine layer 206, map fusion and arbitration layer 208, route planning layer 210, sensor fusion and road world model (RWM) management layer 212, motion Planning and control layer 214, and behavior planning and prediction layer 216. Layers 202 - 216 are merely examples of some of the layers in one example configuration of misconduct management system stack 200 . In other configurations, other layers may be included, such as additional layers for other perception sensors (eg, LIDAR perception layers, etc.), additional layers for planning and/or control, additional layers for modeling, etc. etc., and/or some of the layers 202-216 may be excluded from the misconduct management system stack 200. As indicated by the arrows in Figure 2A, each of the layers 202-216 may exchange data, computation results, and commands. Additionally, the misconduct management system stack 200 may receive and process information from sensors (eg, radar, lidar, cameras, inertial measurement units (IMUs), etc.), navigation systems (eg, GPS receivers, IMUs, etc.), Data from vehicle networks (eg, Controller Area Network (CAN) busses) and databases in memory (eg, digital map data). The misconduct management system stack 200 may output vehicle control commands or signals to a drive-by-wire (DBW) system/control unit 220 , which is a system, subsystem, or computing device that interfaces directly with the vehicle's steering, throttle, and brake control interfaces. The configuration of the misconduct management system stack 200 and DBW system/control unit 220 shown in FIG. 2A is only an example configuration, and other configurations of vehicle management systems and other vehicle elements may be used. As an example, the configuration of the misconduct management system stack 200 and DBW system/control unit 220 shown in FIG. 2A may be used in a vehicle configured for automatic or semi-automatic operation, while a different configuration may be used in a non-automatic vehicle.

Radar perception layer 202 may receive data from one or more detection and ranging sensors, such as radar (eg, 132 ) and/or lidar (eg, 138 ), and process the data to identify and determine other nearby vehicles 100 Location of vehicles and objects. Radar perception layer 202 may include the use of neural network processing and artificial intelligence methods to identify objects and vehicles and pass such information to sensor fusion and RWM management layer 212 .

The camera perception layer 204 may receive data from one or more cameras, such as cameras (eg, 158 , 160 ), and process the data to identify and determine the location of other vehicles and objects in the vicinity of the vehicle 100 . The camera perception layer 204 may include the use of neural network processing and artificial intelligence methods to identify objects and vehicles and pass such information to the sensor fusion and RWM management layer 212 .

The location engine layer 206 may receive data from various sensors and process the data to determine the location of the vehicle 100 . Various sensors may include, but are not limited to, GPS sensors, IMUs, and/or other sensors connected via the CAN bus. Positioning engine layer 206 may also utilize input from one or more cameras, such as cameras (eg, 158, 160) and/or any other available sensors, such as radar, lidar, or the like.

Misconduct management system 200 may include or be coupled to vehicle wireless communication subsystem 230 . The wireless communication subsystem 230 may be configured to communicate with other vehicle computing devices and highway communication systems, such as via vehicle-to-vehicle (V2V) communication links, and/or to remote information sources, such as cloud-based resources, Via cellular wireless communication systems, such as 5G networks. In various embodiments, the wireless communication subsystem 230 may communicate with other V2X system participants via wireless communication links to receive LDM data.

The map fusion and arbitration layer 208 may access LDM data received from other V2X system participants, receive output received from the positioning engine layer 206, and process the data to further determine the location of the vehicle 101 within the map, such as within a traffic lane. location, location within a street map, etc. LDM data may be maintained or stored in the vehicle's memory (eg, memory 432 ). For example, the map fusion and arbitration layer 208 may convert latitude and longitude information from GPS to locations within the surface map of roads in the LDM data. GPS position determination includes errors, so the map fusion and arbitration layer 208 may be used to determine the vehicle's best guess position within the road based on arbitration between GPS coordinates and LDM data. For example, while GPS coordinates may place the vehicle near the middle of a two-lane road in the LDM profile, the map fusion and arbitration layer 208 may determine, based on the direction of travel, that the vehicle is most likely to be aligned with the lane that aligns with the direction of travel. Map fusion and arbitration layer 208 may pass map-based location information to sensor fusion and RWM management layer 212 .

The route planning layer 210 can utilize the LDM data and input from the operator or scheduler to plan the route the vehicle 101 is to follow to a particular destination. Routing layer 210 may pass map-based location information to sensor fusion and RWM management layer 212 . However, other layers (such as sensor fusion and RWM management layer 212, etc.) do not necessarily use previous maps. For example, other stacks may operate and/or control the vehicle based solely on perception data without providing a map, constructing the concept of lanes, boundaries, and local maps when perception data is received.

Sensor fusion and RWM management layer 212 may receive data and outputs produced by radar perception layer 202, camera perception layer 204, map fusion and arbitration layer 208, and route planning layer 210, and use some or all of such inputs to The position and state of the vehicle 101 relative to the road, other vehicles on the road, and other objects in the vicinity of the vehicle 100 are estimated or refined. For example, sensor fusion and RWM management layer 212 may combine image data from camera perception layer 204 with arbitration map location information from map fusion and arbitration layer 208 to refine the vehicle's determined location within a traffic lane. As another example, sensor fusion and RWM management layer 212 may combine object identification and image data from camera perception layer 204 with object detection and ranging data from radar perception layer 202 to determine and refine The relative position of other vehicles and objects in the vicinity of the vehicle. As another example, the sensor fusion and RWM management layer 212 may receive information about the location and direction of travel of other vehicles from vehicle-to-vehicle (V2V) communication, such as via a CAN bus, and associate the information with information from the radar perception layer 202 Combined with information from the camera perception layer 204 to refine the position and motion of other vehicles. The sensor fusion and RWM management layer 212 may output the refined position and status information of the vehicle 100 and the refined position and status information of other vehicles and objects in the vicinity of the vehicle to the motion planning and control layer 214 and/or behavior planning and prediction Layer 216.

As a further example, the sensor fusion and RWM management layer 212 may use dynamic traffic control commands to instruct the vehicle 101 to change speed, lane, direction of travel, or other navigational elements, and combine this information with other received information to determine refinement location and status information. The sensor fusion and RWM management layer 212 may communicate the refined location and status information of the vehicle 101 as well as the refined location and status information of other vehicles and objects in the vicinity of the vehicle 100 via wireless communication, such as via a C-V2X connection, other wireless Connections, etc., output to motion planning and control layer 214, behavior planning and prediction layer 216, and/or devices remote from vehicle 10, such as data servers, other vehicles, and the like.

As yet another example, sensor fusion and RWM management layer 212 may monitor sensing data from various sensors, such as sensing data from radar sensing layer 202, camera sensing layer 204, other sensing layers, etc., and/or from a or data from multiple sensors themselves to analyze conditions in vehicle sensor data. The sensor fusion and RWM management layer 212 may be configured to detect conditions in the sensor data, such as sensor measurements equal to a threshold, above or below a threshold, certain types of sensing sensor measurements, etc., and the sensor data that is part of the provided partial refined position and status information of the vehicle 101 can be output to the behavior via wireless communication, such as via a C-V2X connection, other wireless connection, etc. Planning and forecasting layer 216 and/or equipment remote from vehicle 100, such as data servers, other vehicles, and the like.

The refined location and status information may include vehicle descriptors associated with the vehicle and the owner and/or operator, such as: vehicle specifications (eg, size, weight, color, onboard sensor type, etc.); vehicle location, speed , acceleration, driving direction, attitude, heading, destination, fuel/power levels, and other status information; vehicle emergency status (for example, whether the vehicle is an emergency vehicle or a personal vehicle in an emergency); vehicle limitations (for example, heavy/ wide load, turning restrictions, high occupancy vehicle (HOV) authorization, etc.); vehicle capabilities (e.g., all-wheel drive, four-wheel drive, snow tires, chains, types of connections supported, on-board sensor implementation status, on-board sensor resolution level, etc.); equipment issues (e.g., low tire pressure, weak brakes, sensor malfunctions, etc.); owner/operator travel preferences (e.g., preferred lane, road, route, and/or purpose location, preference to avoid tolls or highways, preference for fastest route, etc.); permission to provide sensor data to the data proxy server (eg, 184); and/or owner/operator identification information.

The behavior planning and prediction layer 216 of the automated vehicle system stack 200 can use the refined position and state information of the vehicle 101 and the position and state information of other vehicles and objects output from the sensor fusion and RWM management layer 212 to predict other vehicles and / or the future behavior of the object. For example, the behavior planning and prediction layer 216 may use such information to predict the future relative positions of other vehicles in the vicinity of the vehicle based on its own vehicle position and velocity and the positions and velocities of other vehicles. Such predictions can take into account information from LDM data and route planning to predict changes in relative vehicle position as the host vehicle and other vehicles travel along the road. The behavior planning and prediction layer 216 may output other vehicle and object behavior and position predictions to the motion planning and control layer 214 . Additionally, the behavior planning and prediction layer 216 may use the object behavior in conjunction with the position prediction results to plan and generate control signals for controlling the motion of the vehicle 101 . For example, based on route planning information, refined locations in road information, and relative positions and motions of other vehicles, behavior planning and prediction layer 216 may decide that vehicle 101 needs to change lanes and accelerations, such as to maintain or achieve a minimum distance from other vehicles, and/or preparing to turn or exit. As a result, behavior planning and prediction layer 216 may calculate or otherwise determine the steering angle of the wheels and changes to throttle settings to command to motion planning and control layer 214 and DBW system/control unit 220 and the like to implement such lane changes and Various parameters required for acceleration. A similar parameter could be the calculated steering wheel command angle.

Motion planning and control layer 214 may receive data and information outputs from sensor fusion and RWM management layer 212 and other vehicle and object behaviors and position predictions from behavior planning and prediction layer 216, and use the information to plan and generate Control signals used to control the motion of the vehicle 101 and verify that such control signals meet the safety requirements of the vehicle 100 . For example, motion planning and control layer 214 may validate and communicate various control commands or instructions to DBW system/control unit 220 based on route planning information, refined locations in road information, and relative positions and movements of other vehicles.

The DBW system/control unit 220 may receive commands or instructions from the motion planning and control layer 214 and convert such information into mechanical control signals for controlling the wheel angles, brakes, and throttle of the vehicle 100 . For example, the DBW system/control unit 220 may respond to the calculated steering wheel command angle by sending corresponding control signals to the steering wheel controller.

In various embodiments, the wireless communication subsystem 230 may communicate with other V2X system participants via wireless communication links to transmit sensor data, location data, vehicle data, and information about the vehicle surroundings collected by the onboard sensors data of. Other V2X system participants can use such information to update LDM data for relay to other V2X system participants.

In various embodiments, the misconduct management system stack 200 may include functionality to perform safety inspections or to oversee various orders, planning, or other decisions that may affect various layers of vehicle and occupant safety. Such security checking or oversight functionality may be implemented within a dedicated layer or distributed across layers and included as part of the functionality. In some embodiments, various safety parameters may be stored in memory, and a safety inspection or supervision function may correlate the determined values (eg, relative distance from nearby vehicles, distance from road centerline, etc.) to the corresponding safety The parameters are compared, and if the safety parameters are violated or are about to be violated, a warning or command is issued. For example, a safety or supervisory function in the behavior planning and prediction layer 216 (or a separate layer) may determine the current or future separation distance between another vehicle (as defined by the sensor fusion and RWM management layer 212 ) and the vehicle (eg, , based on the world model refined by sensor fusion and RWM management layer 212), compares the separation distance with the safe separation distance parameter stored in memory, and if the current or predicted separation distance violates the safe separation distance parameter, Commands are then issued to the motion planning and control layer 214 to speed up, slow down, or turn. As another example, a safety or supervisory function in the motion planning and control layer 214 (or a separate layer) may compare the determined or commanded steering wheel command angle to a safe wheel angle limit or parameter, and respond to exceeding the safe wheel An override command and/or an alarm is issued when the angle limit is commanded.

Certain safety parameters stored in memory may be static (ie not changing over time), such as maximum vehicle speed. Other safety parameters stored in memory may be dynamic in that they may be determined or updated continuously or periodically based on vehicle state information and/or environmental conditions. Non-limiting examples of safety parameters include maximum safe speed, maximum brake pressure, maximum acceleration, and safe wheel angle limits, all of which may be a function of road and weather conditions.

FIG. 2B illustrates examples of subsystems, computing elements, computing devices, or units within vehicle management system 250 that may be used within vehicle 101 . Referring to Figures 1A-2B, in some embodiments, layers 202, 204, 206, 208, 210, 212, and 216 of misconduct management system stack 200 may be similar to those described with reference to Figure 2A, and except for misconduct management System stack 250 may operate similarly to misconduct management system stack 200 , which may communicate various data or instructions to vehicle safety and collision avoidance system 252 other than DBW system/control unit 220 . For example, the configuration of the misconduct management system stack 250 and vehicle safety and collision avoidance system 252 shown in FIG. 2B may be used for non-autonomous vehicles.

In various embodiments, behavior planning and prediction layer 216 and/or sensor fusion and RWM management layer 212 may output data to vehicle safety and collision avoidance system 252 . For example, the sensor fusion and RWM management layer 212 may output the sensor data to the vehicle safety and collision avoidance system 252 as part of the provided refined position and status information of the vehicle 101 . The vehicle safety and collision avoidance system 252 may use the refined location and state information of the vehicle 101 to make safety decisions about the vehicle 101 and/or the occupants of the vehicle 100 . As another example, the behavior planning and prediction layer 216 may output behavior models and/or predictions related to the motion of other vehicles to the vehicle safety and collision avoidance system 252 . The vehicle safety and collision avoidance system 252 may use behavioral models and/or predictions related to the motion of other vehicles to make safety decisions about the vehicle 101 and/or the occupants of the vehicle 101 .

In various embodiments, the vehicle safety and collision avoidance system 252 may include functionality to perform safety checks or to supervise various commands, planning or other decisions, and human driver behavior at various layers that may affect vehicle and occupant safety. In some embodiments, various safety parameters may be stored in memory, and the vehicle safety and collision avoidance system 252 may associate the determined values (eg, relative distance from nearby vehicles, distance from road centerline, etc.) to the corresponding If the safety parameters are violated or about to be violated, a warning or command will be issued. For example, the vehicle safety and collision avoidance system 252 may determine the current or future separation distance between another vehicle (as defined by the sensor fusion and RWM management layer 212 ) and the vehicle (eg, based on the vehicle 212 Refinement of the World Model), compares the separation distance with the safe separation distance parameter stored in memory, and if the current or predicted separation distance violates the safe separation distance parameter, issues instructions to the driver to speed up, slow down or turn. As another example, the vehicle safety and collision avoidance system 252 may compare changes in the steering wheel angle of the human driver to a safe wheel angle limit or parameter and issue an override command in response to a steering wheel angle exceeding the safe wheel angle limit and/or or alert.

FIG. 3 illustrates an example system-on-chip (SOC) architecture of a processing device SOC 300 suitable for implementing various embodiments in a vehicle. 1A-3, the processing device SOC 300 may include multiple heterogeneous processors, such as a digital signal processor (DSP) 303, a modem processor 304, an image and object recognition processor 306, a mobile display processor 307, Application processor 308 , and resource and power management (RPM) processor 317 . The processing device SOC 300 may also include one or more auxiliary processors 310 (eg, vector auxiliary processors) coupled to one or more of the heterogeneous processors 303 , 304 , 306 , 307 , 308 , 317 . Each of the processors may include one or more cores, and independent/internal clocks. Each processor/core can perform operations independently of other processors/cores. For example, the processing device SOC 300 may include a processor executing a first type of operating system (eg FreeBSD, LINUX, OS X, etc.) and a second type of operating system (eg Microsoft Windows). In some embodiments, the application processor 308 may be a main processor, a central processing unit (CPU), a microprocessor unit (MPU), an arithmetic logic unit (ALU), or the like of the SOC 300 . Graphics processor 306 may be a graphics processing unit (GPU).

The processing device SOC 300 may include analog and custom circuits 314 for managing sensor data, analog-to-digital conversion, wireless data transmission, and for performing other specialized operations, such as processing encoded audio and video signals for display in a web browser render. Processing device SOC 300 may also include system components and resources 316 such as voltage regulators, oscillators, phase locked loops, peripheral bridges, data controllers, memory controllers, system controllers, access ports, timers, and Other similar elements supporting processors and software clients (eg, web browsers) executing on computing devices.

Processing device SOC 300 also includes dedicated circuitry for camera actuation and management processor 305 that includes, provides, controls and/or manages one or more cameras 158, 160 (eg, main camera, web camera, 3D camera, etc.), video display data from camera firmware, image processing, video preprocessing, video front end (VFE), online JPEG, HD video transcoder, etc. The camera actuation and management processor 305 may be a separate processing unit and/or include a separate or internal clock.

In some embodiments, the image and object recognition processor 306 may be configured with processor-executable instructions and/or dedicated hardware configured to perform the image processing and object recognition analysis involved in various embodiments. For example, the image and object recognition processor 306 may be configured to perform processing operations on images received from the cameras (eg, 158 , 160 ) via the camera actuation and management processor 305 to identify and/or identify other vehicles, and perform the functions of the camera perception layer 204 in other ways. In some embodiments, the processor 306 may be configured to process radar or lidar data and perform the functions of the radar perception layer 202 as described.

System elements and resources 316, analog and custom circuits 314, and/or camera actuation and management processor 305 may include circuitry to interface with peripheral devices, such as cameras 158, 160, radar 168, lidar 170, electronic displays, wireless Communication equipment, external storage chips, etc. Processors 303, 304, 306, 307, 308 may be interconnected via interconnect/bus module 324 to one or more memory elements 312, system elements and resources 316, analog and custom circuits 314, camera actuation and management Processor 305 and RPM processor 317, which may include a reconfigurable logic gate array and/or implement a bus architecture (eg, CoreConnect, AMBA, etc.). Communication can be provided by advanced interconnects, such as high-performance circuits on a chip (NoC).

Processing device SOC 300 may also include an input/output module (not shown) for communicating with resources external to the SOC, such as clock 318 and voltage regulator 320 . Resources external to the SOC (eg, clock 318, voltage regulator 320) may be provided by two or more internal SOC processors/cores (eg, DSP 303, modem processor 304, graphics processor 306, application processor 308, etc.) shared.

In some embodiments, the processing device SOC 300 may be included in a control unit (eg, 140 ) for use in a vehicle (eg, 100 ). As described, the control unit may include communication links for communicating with a telephone network (eg, 180 ), the Internet, and/or a web server (eg, 184 ).

Processing device SOC 300 may also include additional hardware and/or software components suitable for acquiring sensor data from sensors, including motion sensors (eg, accelerometers and gyroscopes of an IMU), user interface components ( For example, input buttons, touch screen displays, etc.), microphone arrays, sensors for monitoring physical conditions (eg, position, orientation, motion, orientation, vibration, pressure, etc.), cameras, compasses, GPS receivers, Communication circuits (eg, Bluetooth®, WLAN, WiFi, etc.), and other well-known elements of modern electronic equipment.

4 is a block diagram of elements illustrating a system 400 configured to generate a local dynamic map material model in accordance with various embodiments. In some embodiments, system 400 may include one or more V2X-equipped computing platforms 402 and/or one or more other V2X system participants 404 . 1A-4 and 7-9, V2X equipment 402 may include processors (eg, 434, 702, 802). V2X equipment 402 may be configured by machine-executable instructions 406 . Machine-executable instructions 406 may include one or more instruction modules. Instruction modules may include computer program modules. The command module may include an LDM data receiving module 408, an LDM data integration module 410, an LDM data determination module 412, an LDM data providing module 414, a map generation module 416, a map sending module 418 and/or other command modules. one or more of the group.

The LDM data receiving module 408 may be configured to receive new LDM data for the misconduct management system executing on the V2X equipment processor. In some embodiments, the LDM data receiving module 408 may be configured to receive registration messages from other V2X system participants 404 . In some embodiments, the LDM data receiving module 408 may be configured to receive planned route information from other V2X system participants 404 . In some embodiments, the LDM data receiving module 408 may be configured to receive mobile device kinematics information from other V2X system participants 404 . In some embodiments, the LDM data receiving module 408 may be configured to receive data from other V2X system participants 404 , such as sensor data, image data, audio data, or operations obtained by other V2X system participants 404 status information.

The LDM data integration module 410 may be configured to integrate new LDM data into the LDM data model.

The LDM profile determination module 412 may be configured to determine the LDM profile of the LDM profile model associated with other specific V2X system participants 404 . In some embodiments, the LDM profile determination module 412 may be configured to determine the LDM profile associated with another particular V2X system participant 404 based on information included in the registration message. In some embodiments, the LDM data determination module 412 may be configured to determine LDM data related to another particular V2X system participant based on the planned route information. In some embodiments, the LDM data determination module 412 may be configured to determine LDM data related to another particular V2X system participant 404 based on the kinematic information. In some embodiments, the LDM data determination module 412 may be configured to determine information related to the LDM data from the received data.

The LDM data provision module 414 may be configured to provide the determined relevant LDM data to other V2X system participants 404 . In some embodiments, the determined relevant LDM data may include highly dynamic LDM information.

The map generation module 416 may be configured to generate a digital map covering an area within a predetermined distance of other V2X system participants. In some embodiments, the map sending module 418 may be configured to send the digital map to other V2X system participants 404 . The digital map may be generated and transmitted in a format suitable for automatic navigation by other V2X system participants 404 .

In certain embodiments, V2X equipment 402, other V2X system participant devices 404, and/or external resources 430 may be operably linked via one or more electronic communication links. For example, such electronic communication links may be established at least in part via a network such as the Internet and/or other networks. It should be understood that this is not intended to be limiting, and the scope of this case includes implementations in which V2X equipment 402, other V2X system participants 404, and/or external resources 430 may be operably linked via some other communication medium.

Each of the other V2X system participants 404 may include one or more processors configured to execute computer program modules. Computer program modules may be configured to enable experts or users associated with a given V2X system participant 404 to interface with the system 400 and/or external resources 430 and/or to provide information attributed herein to other V2X system participants 404 other functions.

External resources 430 may include information sources outside of V2X system 400, external entities participating in V2X system 400, and/or other resources. In some embodiments, some or all of the functionality attributed herein to external resources 430 may be provided by resources included in system 400 .

V2X equipment 402 may include electronic memory 432, one or more processors 434, and/or other elements. V2X equipment 402 may include communication lines or ports to enable exchange of information with networks and/or other computing platforms. The illustration of V2X equipment 402 in FIG. 4 is not intended to be limiting. V2X equipment 402 may include a plurality of hardware, software, and/or firmware elements that operate together to provide the functionality attributed to V2X equipment 402 herein. For example, V2X equipment 402 may be implemented by a computing platform cloud that operates in conjunction with V2X equipment 402 .

Electronic memory 432 may include a non-transitory storage medium that electronically stores information. The electronic storage media of electronic memory 432 may include one or both of system memory and/or removable memory, where system memory is provided integrally (ie, substantially non-removable) with V2X equipment 402 , and removable memory may be removably connected to V2X equipment 402 via, for example, ports (eg, Universal Serial Bus (USB) ports, Firewire ports, etc.) or drives (eg, disk drives, etc.). Electronic memory 432 may include optically readable storage media (eg, optical disks, etc.), magnetically readable storage media (eg, magnetic tape, magnetic hard disk, floppy disk drive, etc.), charge-based storage media (eg, EEPROM, RAM, etc.) , one or more of solid state storage media (eg, flash memory drives, etc.), and/or other electronically readable storage media. Electronic memory 432 may include one or more virtual storage resources (eg, cloud memory, virtual private network, and/or other virtual storage resources). Electronic memory 432 may store software algorithms, information determined by processor 434, information received from V2X equipment 402, information received from other V2X system participants 404, and/or information that enables V2X equipment 402 to function as described herein. other information.

The processor 434 may be configured to provide information processing capabilities in the V2X equipment 402 . As such, processor 434 may include one or more of a digital processor, an analog processor, digital circuits designed to process information, analog circuits designed to process information, state machines, and/or other mechanisms for electronically processing information. multiple. Although processor 434 is shown in FIG. 4 as a single entity, this is for illustration purposes only. In some implementations, the processor 434 may include a plurality of processing units. The processing units may be physically located within the same device, or the processor 434 may represent the processing functions of a plurality of devices operating in conjunction. Processor 434 may be configured to execute modules 408, 410, 412, 414, 416, 418, and/or other modules. The processor 434 may be configured to execute the modules 408, 410, 412, 414, 416, 418 and/or other modules via software; hardware; firmware; some combination of software, hardware and/or firmware; and/or other mechanisms for configuring processing capabilities on processor 434 . As used herein, the term "module" may refer to any element or set of elements that perform the function attributed to the module. This may include one or more physical processors, processor-readable instructions, circuits, hardware, storage media, or any other element during execution of processor-readable instructions.

It should be understood that although modules 408-418 are shown in FIG. 4 as being implemented within a single processing unit, in embodiments where processor 434 includes multiple processing units, one or more of modules 408-418 may be Implement away from other mods. The description of the functionality provided by the various modules 408-418 described below is for illustrative purposes and not limiting, as any of the modules 408-418 may provide more or less functionality than described . For example, one or more of modules 408-418 may be eliminated and some or all of its functionality may be provided by other of modules 408-418. As another example, the processor 434 may be configured to execute one or more additional modules that may perform some or all of the functions attributed to one of the modules 408-418 below.

5 is a program flow diagram illustrating the operations of a method 500 performed by a processor observing a vehicle's V2X equipment for comparing received V2X messages with the vehicle's Local Dynamic Map (LDM) data model , and determine whether there is any inconsistency between the data received in the V2X message and the LDM data model maintained or stored at the local end to detect misconduct. 1-5, the operations of method 500 may be performed by a processor observing vehicle V2X equipment (eg, vehicle 12 in FIG. ID).

At block 502 , the processor may monitor a plurality of vehicle sensors associated with observing control maneuvers, navigation, and/or other operations of a vehicle (eg, vehicle 12 ). By monitoring data from the vehicle's own sensors, the misconduct management system executing on the observing vehicle's V2X equipment 402 can begin collecting data that can be used to generate an LDM data model representing the observing vehicle's surroundings. This could include information on suspicious vehicles transmitting V2X messages with data indicative of misconduct conditions. Additionally, the LDM data model may include information about nearby vehicles.

In block 504 , an LDM data model representing the observed environment around the vehicle may be generated by the V2X equipment 402 based at least in part on the aggregation of additional data collected from the plurality of sensors, as discussed above with respect to FIGS. 2A-4 . LDM data models can contain type 1-4 data. In block 506, the generated LDM data model may be maintained locally or stored in memory (eg, electronic memory 432).

In block 508 , the misconduct management system executing on the V2X equipment processor may receive a V2X message from another V2X system participant 404 . V2X messages may include traffic information, GPS information of the reporting vehicle and other nearby vehicles calculated from the reporting vehicle's onboard equipment and/or provided by the neighboring vehicle itself, and/or map data detailing road geometry and street characteristics.

In decision block 510, the V2X message, the misconduct management system executing on the V2X equipment processor can detect misconduct by comparing the data contained in the received V2X message with the LDM data model maintained or stored locally Conduct status to determine whether received V2X messages include data indicative of misconduct status. For example, an instant vehicle (called a observing vehicle) can receive V2X messages that include data indicating that another vehicle (called a reporting vehicle) may be passing between two nearby vehicles. In some embodiments, both the reporting vehicle and the two neighboring vehicles may be V2X system participants. However, in some cases, neither the reporting vehicle nor the two neighboring vehicles are V2X system participants. In other cases, the reporting vehicle and some of the two neighboring vehicles may be V2X system participants, while others are not.

In an example, the observing vehicle may obtain information about the location/position, driving direction, speed, operation, etc. of the reporting vehicle and two neighboring vehicles via the observing vehicle's own sensors. For example, cameras and lidars mounted on observation vehicles. Additionally, the observing vehicle may receive V2X messages from either the reporting vehicle and either/or both of the two neighboring vehicles. Such received V2X messages may include camera and/or lidar information that confirms observations by the observing vehicle of the position/position, driving direction, speed, operation, etc. of the reporting vehicle and two nearby vehicles. Additionally, V2X messages from the reporting vehicle or either of the two nearby vehicles may include GPS position/location information, speedometer data, and the like. Further camera and lidar information about the reporting vehicle or either of the two nearby vehicles can be received from the RSU V2X system participant.

Any or all of the data received in the various V2X messages can enhance the locally maintained LDM data model established by the observing vehicle to represent the environment surrounding the observing vehicle. Thus, the LDM profile model of the observed vehicle may include information on the position/position, direction of travel and speed of the reporting vehicle and two neighboring vehicles. The observing vehicle can detect misconduct conditions by comparing the data contained in the received V2X messages with data in the observing vehicle's LDM data model. For example, a V2X message received from a reporting vehicle may indicate that the reporting vehicle is passing between two adjacent vehicles. However, based on all other information that has been collected by the observing vehicle, the LDM data model of the observing vehicle may indicate that the reporting vehicle does not have enough space to pass two adjacent vehicles. Thus, a misconduct management system executing on the V2X equipment processor of the observing vehicle may identify V2X messages received from the reporting vehicle as containing or attesting to a misconduct condition. Any data indicating position/position, speed, direction of travel, etc. received from the reporting vehicle in the V2X message may be inaccurate, corrupted or maliciously altered.

In response to determining that the V2X message received from the reporting vehicle includes data indicative of a misconduct condition based on the comparison in block 510 (ie, determination 510=Yes), the misconduct management system operating in the V2X equipment 402 of the observing vehicle A Misbehavior Report (MBR) may be generated in block 512 identifying the misbehavior condition and the suspicious vehicle that sent the V2X message.

In block 514, the misconduct management system executing in the V2X equipment 402 of the observing vehicle may send the MBR to a misconduct management agency (MA) for further analysis, reporting, and corrective action. For example, an MA might send a message to a suspicious vehicle that its sensors need to be repaired or replaced. In order to conduct an accurate and comprehensive analysis of the misconduct situation, the MA can utilize the data within the LDM data model, so that it can determine the overall analysis of the situation leading to the misconduct situation. In addition, by analyzing the data in the LDM data model, more appropriate and effective corrective actions can be issued to mitigate the misconduct situation. To achieve this, certain embodiments include a misconduct management system for providing better quality misconduct detection (such as providing fewer false negative detections), improving the robustness and resiliency of the V2X system 103 .

However, as more and more vehicles are equipped with V2X equipment, the amount of misconduct that may be detected is growing exponentially. Furthermore, due to the large number of MBRs to be sent, the amount of material that may contain MBRs can be very large given the bandwidth constraints. To address this problem and reduce the amount of material that can be sent with the MBR, in some embodiments, the misconduct management system may only send a representation of the LDM material model to the MA. In some embodiments, the misconduct management system may send an incomplete set of data for the LDM data model when sending the MBR to the MA, such as including only data most relevant to the detected misconduct condition and/or exclusions Information not related to a state of misconduct.

After sending the MBR to the MA in block 514, the misconduct management system executing in the observing vehicle's V2X equipment 402 may again perform the operations in block 502 to continue monitoring the observing vehicle's sensors to collect information about the observing vehicle's surroundings. material.

In response to determining that the data included in the received V2X message does not indicate a misconduct condition (ie, decision outcome 510=no), the misconduct management system may be based on at least one of the observed object dynamics in the LDM data model or New data input from the V2X messages received to perform computations, then in block 516 the LDM data model is modified to incorporate computations and data based on and included in the received V2X messages to enhance, improve or otherwise update the observed vehicle The updated LDM data is then stored in block 506. Therefore, the misbehavior management system implemented in the V2X equipment 402 of the observed vehicle can continuously refine the LDM profile model. The updated LDM data model may include additional data from V2X messages that refine and/or refine the LDM data model representing the observed environment around the vehicle. In block 506, the updated LDM data model may be maintained locally or stored in memory.

As described, after sending the MBR to the MA in block 514 or storing the updated LDM data model in block 506 , the misconduct management system operating in the V2X equipment 402 observing the vehicle may continue to receive additional V2X in block 508 message.

In some embodiments, the misconduct management system executing in the observing vehicle's V2X equipment 402 may also periodically perform the operations in block 502 to continue monitoring the observing vehicle's sensors to collect data about the observing vehicle's surroundings . In this way, the range of data representing the observed environment around the vehicle can be continuously expanded to improve and update the LDM so that misconduct conditions can be detected more accurately.

6 is a program flow diagram illustrating example operations that may be performed as part of block 510 of method 500 for determining whether a misconduct condition is detected in a received V2X message. 1A-6, the operations of block 510 may be performed by an inappropriate behavior management system operating in the V2X equipment 402 of the observed vehicle.

After receiving the V2X message in block 508 of method 500, the misconduct management system may obtain selected data (eg, location/location, speed, direction of travel, temperature, etc.) from the received V2X message in block 518. As part of the operations in block 518, the identifier of the original sender of the received V2X message may be obtained.

In optional block 519, the misconduct management system may select data elements or data elements within the LDM data model for use in determining whether the information reported in the V2X message indicates misconduct or is the product of misconduct. As previously mentioned, the LDM data model will include a number of data elements that define the vehicle's surroundings and other nearby vehicles, many of which can be used to assess the accuracy or reliability of the information received in the V2X message. Thus, in some embodiments, the misconduct management system may select certain data elements (eg, subsets of information) within the LDM data model for use in validating or validating received V2X messages. In some embodiments, the misconduct management system may select information or elements within the LDM data model based on or in response to the type of information within the received V2X message. For example, if the V2X message includes the location of another vehicle or the location of a hazard on the road ahead of the vehicle, the misconduct management system may select location-related data elements in the LDM data model for use in attestation or V2X messages, and to avoid access Data elements related to speed, weather conditions, road conditions, or the position behind the vehicle. As another example, if the V2X information includes dynamic information about another vehicle (eg, turning angle, speed, braking status, etc.), then static or legacy data elements in the LDM data model will not be able to be used to evaluate the V2X information. By selecting and accessing subsets of information in the LDM data model that are relevant to verifying or validating the information in the received V2X messages, the misbehavior management system can save processing resources and memory utilization, and if all the information in the LDM data model is The data is used in the message evaluation process, which enables faster evaluation of messages.

In block 520 , the misconduct management system executing in the V2X equipment of the observed vehicle may associate the parsed data contained in the received V2X messages with the LDM maintained or stored in the observed vehicle's memory (eg, memory 432 ) The data within the data model is compared.

In decision block 522, the misconduct management system may determine whether any analytical data contained in the received V2X message conflicts or is inconsistent with data within the LDM data model maintained or stored in the observing vehicle's memory ( For example, memory 432).

For example, in decision block 522, the misconduct management system may analyze the location information of the vehicle issuing the received V2X message to determine whether the information reported in the V2X message is inconsistent or conflicting with location information in the LDM data model (eg, indicating the position corresponding to the position of the other vehicle in the LDM).

As another example of a possible misconduct condition that may be detected in decision block 522 using the LDM data, the observing vehicle may track nearby vehicles via the observing vehicle's sensors. Neighboring vehicles may not be V2X system participants. The suspicious vehicle may transmit a V2X message to the observing vehicle, implying that the location of the suspicious vehicle overlaps with the location of the tracked neighbor vehicle, where the location of the tracked neighbor vehicle is observed and determined by the sensor of the observing vehicle. In other words, the suspicious vehicle can transmit a V2X message to the observing vehicle, implying that the location of the suspicious vehicle is consistent with the location of the tracked neighbor vehicle, where the location of the tracked neighbor vehicle is observed by the observing vehicle's sensors (such as cameras) and Decide. In this case, the V2X messages received by the observing vehicle from the suspect vehicle contain conflicting or inconsistent data with the data in the observing vehicle's LDM data model. Thus, observing a vehicle may yield an MBR indicative of a suspicious vehicle's misbehaving condition.

In another example of a possible misconduct condition that may be detected using the LDM data in decision block 522, there may be an obstacle in the road. For example, a sofa being transported might fall off a flatbed truck. The observing vehicle can detect obstacles via the sensors of the observing vehicle. The observing vehicle may receive V2X messages from the suspicious vehicle, including information suggesting that the suspicious vehicle maintains its driving direction and speed so that the suspicious vehicle travels through obstacles without slowing down. In this case, the V2X messages received by the observing vehicle from the suspect vehicle contain conflicting or inconsistent data with the data in the observing vehicle's LDM data model. Thus, observing a vehicle may yield an MBR indicative of a suspicious vehicle's misbehaving condition.

As another example of a possible misconduct condition that may be detected in decision block 522 using the LDM data, the observing vehicle may track nearby vehicles via the observing vehicle's sensors. Neighboring vehicles may not be V2X system participants. A neighboring vehicle's line of sight may be temporarily blocked by an obstacle (i.e. something blocks the way for a short time, so the observing vehicle cannot directly see the AV), and then becomes trackable again. During the occlusion period, the suspicious vehicle can transmit V2X messages about the driving direction (i.e. trajectory) of the suspicious vehicle to the observing vehicle. Observing a vehicle can reconstruct the route that neighboring vehicles can travel when they are not visible. Based on the information, the observing vehicle can notice that all possible routes suggest a collision between the suspect vehicle and neighboring vehicles. Since no collisions were recorded by the observing vehicle, the V2X messages received by the observing vehicle from the suspect vehicle contained conflicting or inconsistent data with the data in the observing vehicle's LDM data model. Thus, observing a vehicle may yield an MBR indicative of a suspicious vehicle's misbehaving condition.

As another example of a possible misconduct condition that may be detected using the LDM data in decision block 522, the observing vehicle may receive a V2X traffic signal message from the RSU indicating that the traffic lights are red. Observing vehicles can receive V2X messages from suspicious vehicles indicating that the suspicious vehicle is driving through an intersection. Although the suspicious vehicle may actually be running a red light, the V2X messages received by the observing vehicle from the suspicious vehicle appear to contain conflicting or inconsistent data with the data in the observing vehicle's LDM data model. Thus, observing a vehicle may yield an MBR indicative of a suspicious vehicle's misbehaving condition.

As another example of a possible misconduct condition that may be detected using the LDM data in decision block 522, the observing vehicle may receive a V2X message from the RSU indicating that road construction is causing lane deviation. Lane offsets are not shown in the static map available for viewing vehicles. However, based on the incoming V2X information, the observing vehicle noticed that all nearby vehicles behaved as if a lane shift had occurred, i.e. they all shifted the width of one lane to the left at a specific location. Suspicious vehicles can transmit V2X messages implying that the suspicious vehicle did not observe a lane deviation, i.e., the suspicious vehicle appears to be driving by, even though the driver should be able to see other nearby vehicles doing a lane deviation. This means that the information of the suspicious vehicle is generated remotely to the actual location by witnesses who are unaware of the lane deviation. In this case, the V2X messages received by the observing vehicle from the suspect vehicle contain conflicting or inconsistent data with the data in the observing vehicle's LDM data model. Thus, observing a vehicle may yield an MBR indicative of a suspicious vehicle's misbehaving condition.

In response to determining that the parsed data contained in the received V2X message conflicts or disagrees with data within the LDM data model maintained or stored in the observing vehicle's memory (eg, memory 432 ) (ie, decision result 522 = yes ), an inappropriate behavior management system operating in the V2X equipment 402 of the observed vehicle may perform the operations in block 512 of the method 500 to generate the described MBR.

In response to a determination that the parsed data contained in the received V2X message does not conflict or agree with data within the LDM data model maintained or stored in the observing vehicle's memory (eg, memory 432 ) (ie, determination result 522 = No), the misconduct management system operating in the V2X equipment 402 of the observed vehicle may perform the operations in block 516 to modify or update the described LDM profile model.

Various embodiments, including but not limited to the embodiments described above with reference to FIGS. 1A-6 , may be implemented in a variety of computing systems including in-vehicle equipment as well as mobile computing devices, one example of which is suitable for use with the various embodiments in FIG. 7 . shown in. The mobile computing device 700 may include a processor 702 coupled to a touch screen controller 704 and an internal memory 706 . The processor 702 may be one or more multi-core integrated circuits designated for general or specific processing tasks. Internal memory 706 may be volatile or non-volatile memory, secure and/or encrypted memory, or unsecured and/or unencrypted memory, or any combination thereof. Examples of memory types that may be utilized include, but are not limited to, DDR, LPDDR, GDDR, WIDEIO, RAM, SRAM, DRAM, P-RAM, R-RAM, M-RAM, STT-RAM, and embedded DRAM. The touch screen controller 704 and the processor 702 may also be coupled to a touch screen panel 712, such as a resistive-sensing touch-screen, a capacitive-sensing touch screen, an infrared-sensing touch screen, and the like. Additionally, the display of the mobile computing device 700 does not need to have a touch screen function.

The mobile computing device 700 may have one or more radio signal transceivers 708 (eg, Peanut, Bluetooth, ZigBee, Wi-Fi, RF radios) and antennas 710 for transmitting and receiving communications, coupled to each other and/or Connected to processor 702 . Transceiver 708 and antenna 710 may be used with the circuits described above to implement various wireless transmission protocol stacks and interfaces. The mobile computing device 700 may include a cellular network wireless modem chip 716 that enables communication via a cellular network and is coupled to the processor.

The mobile computing device 700 may include a peripheral connection interface 718 coupled to the processor 702 . The peripheral connection interface 718 may be individually configured to accept one type of connection, or may be configured to accept various types of physical and communication connections, common or proprietary, such as Universal Serial Bus (USB), FireWire, Thunderbolt, or PCIe . The peripheral device connection interface 718 may also be coupled to a similarly configured peripheral device port (not shown).

The mobile computing device 700 may also include a speaker 714 for providing audio output. The mobile computing device 700 may also include a housing 720 made of plastic, metal, or a combination of various materials for containing all or some of the components described herein. Those of ordinary skill in the art will recognize that, in an in-vehicle embodiment, the housing 720 may be a dashboard console of the vehicle. Mobile computing device 700 may include a power source 722 coupled to processor 702, such as a disposable or rechargeable battery. The rechargeable battery may also be coupled to the peripheral port to receive charging current from a source external to the mobile computing device 700 . The mobile computing device 700 may also include physical buttons 724 for receiving user input. The mobile computing device 700 may also include a power button 726 for turning the mobile computing device 700 on and off.

Various embodiments, including but not limited to those described above with reference to FIGS. 1A-6 , may be implemented in a variety of computing systems including laptop computer 800 , one example of which is shown in FIG. 8 . Many laptop computers include a trackpad touch surface 817 that acts as a pointing device for the computer, and thus can receive similar dragging, scrolling, and flicking as implemented on computing devices equipped with touchscreen displays as previously described. flick gesture. Laptop 800 will typically include a processor 802 coupled to volatile memory 812 and mass non-volatile memory, such as a disk drive 813 of flash memory. Additionally, computer 800 may have one or more antennas 808 for transmitting and receiving electromagnetic radiation, which may be connected to a wireless data link and/or to a cellular telephone transceiver 816 coupled to processor 802. Computer 800 may also include a floppy disk drive 814 and a compact disk (CD) drive 815 coupled to processor 802 . In a notebook configuration, the computer housing includes a trackpad 817 , a keyboard 818 and a display 819 , all of which are coupled to the processor 802 . Other configurations of computing devices may include well-known computer mice or trackballs coupled to a processor (eg, via a USB input), which may also be used in conjunction with the various embodiments.

Various embodiments, including but not limited to the embodiments described above with reference to FIGS. 1A-6 , may also include misconduct management agencies utilizing fixed computing systems, such as any of a variety of commercially available servers. An example server 900 is illustrated in FIG. 9 . Such servers 900 typically include one or more multi-core processor assemblies 901, such as disk drives 904, coupled to volatile memory 902 and mass non-volatile memory. As shown in FIG. 9, a multi-core processor assembly 901 may be added to server 900 by inserting the multi-core processor assembly 901 into the rack of the assembly. Server 900 may also include a network access port 907 coupled to multi-core processor assembly 901 for establishing network interface connections to network 908, such as areas coupled to other broadcast system computers and servers Network, Internet, public switched telephone network and/or cellular data network (eg, CDMA, TDMA, GSM, PCS, 3G, 4G, 5G, LTE, or any other type of cellular data network).

Many different cellular and mobile communication services and standards are available or foreseeable in the future, all of which can be implemented and benefit from various embodiments. Such services and standards include, for example, 3rd Generation Partnership Project (3GPP), Long Term Evolution (LTE) systems, 3rd Generation Wireless (3G), 4th Generation (4G), 5th Generation Mobile wireless technology (5G), Global System for Mobile Communications (GSM), Universal Mobile Telecommunications System (UMTS), 3GSM, General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA) systems (eg, cdmaOne, CDMA1020TM) ), Enhanced Packet Radio Traffic (EDGE), Advanced Mobile Phone System (AMPS), Digital AMPS (IS-136/TDMA), Evolutionary Data Optimized Technology (EV-DO), Digitally Enhanced Cordless Telecommunications (DECT), Worldwide Interoperability for Microwave Access (WiMAX), Wireless Local Area Network (WLAN), Wi-Fi Protected Access I and II (WPA, WPA2) and Integrated Digital Enhanced Network (iDEN). Each of these techniques involves, for example, the sending and receiving of voice, data, signaling, and/or content messages. It should be understood that any reference to terms and/or technical details related to a single telecommunications standard or technology is for illustrative purposes only and is not intended to limit the scope of the claim to a particular communication system or technology, unless in the language of the claim specifically stated in.

Implementation examples are described in the following paragraphs. While some of the following implementation examples are described in terms of example methods, further example implementations may include the example methods discussed in the following paragraphs implemented by an inappropriate behavior management system executing on a V2X equipment processor, which may be an onboard vehicle A unit, a mobile device unit, a mobile computing unit, or a fixed roadside unit including a processor configured with processor-executable instructions to perform the operations of the methods of implementing the examples below; the examples discussed in the following paragraphs implemented by V2X equipment methods, V2X equipment comprising means for performing the functions of the methods of the following implementation examples; and the example methods discussed in the following paragraphs may be implemented as non-transitory processor-readable storage having processor-executable instructions stored thereon The media, processor-executable instructions are configured to cause a processor of the V2X equipment to perform the operations of the method below implementing the example.

Example 1. A method of detecting a misconduct condition in a vehicle-to-everything (V2X) system, executed by a processor of a vehicle, comprising receiving a V2X message from another V2X system participant, wherein the V2X message contains information about the environment surrounding the vehicle. Data; compare the data contained in the received V2X message with the data in the local dynamic map (LDM) data model maintained or stored by the local end to detect misconduct conditions; respond to the detection of misconduct based on the comparison results condition, generate a misconduct report identifying the misconduct condition; and send the generated misconduct report to the misconduct management agency.

Example 2. The method of claim 1, further comprising monitoring a plurality of sensors in a vehicle to collect additional data about the environment surrounding the vehicle; generating a representation based at least in part on the aggregation of the additional data collected from the plurality of sensors an LDM data model of the surrounding environment of the vehicle; and storing the LDM data model in memory.

Example 3. The method of any one of Example 1 or Example 2, further comprising, in response to determining that a misconduct condition was not detected: based on at least one of observed object dynamics in the LDM data model or based on Perform calculations on new data input received by a V2X message; modify the LDM data model to incorporate the calculations and data included in the received V2X message; and replace LDM data locally maintained or stored in memory with the modified LDM model Model.

Example 4. The method of any of examples 1-3, wherein sending the generated misconduct report to a misconduct management agency includes sending a representation of the LDM profile model.

Example 5. The method of example 4, wherein the representation of the LDM data model includes an incomplete data set for the LDM data model.

Example 6. The method of any of Examples 1-5, further comprising receiving feedback from the misconduct management agency, wherein the feedback includes corrective actions for mitigating the misconduct condition.

Example 7. The method of any one of Examples 1-6, wherein the data about the surroundings of the vehicle included in the received V2X message includes traffic information.

Example 8. The method of any one of Examples 1-7, wherein the data about the surroundings of the vehicle included in the received V2X message includes location information of nearby vehicles based on GNSS (eg, GPS) data.

Example 9. The method of any of Examples 1-8, wherein the data about the surroundings of the vehicle included in the received V2X message includes map data detailing road geometry and street furniture.

Example 10. The method according to any one of Examples 1-9, wherein comparing the data included in the received V2X message with the LDM data model maintained or stored at the local end to detect the misconduct situation includes, deciding to receive Whether any data included in the received V2X message conflicts with the information in the LDM data model maintained or stored locally.

Example 11. The method according to any one of Examples 1-10, wherein comparing the data included in the received V2X message with an LDM data model maintained or stored by the local end to detect misconduct conditions includes, Select a subset of data elements in the LDM data model maintained or stored locally for comparison with the data included in the received V2X message; and determine whether any data included in the received V2X message is consistent with the local A subset of selected data elements within the LDM data model maintained or stored by the end conflict.

Example 12. The method according to any one of Examples 1-11, wherein comparing the data included in the received V2X message with an LDM data model maintained or stored at the local end to detect misconduct conditions includes, determining Whether the position of the first adjacent vehicle included in the received V2X message is consistent with the position of the second adjacent vehicle in the LDM data model maintained or stored at the local end.

Example 13. The method according to any one of Examples 1-12, wherein comparing the data included in the received V2X message with an LDM data model maintained or stored at the local end to detect misconduct conditions includes, determining Whether the status information of the neighboring vehicle that sends the received V2X message conflicts with the status information of the neighboring vehicle in the LDM data model maintained or stored on the local end.

The various embodiments shown and described are provided by way of example only to illustrate the various features of the claimed item. However, the features shown and described with respect to any given embodiment are not necessarily limited to the associated embodiment and may be used or combined with the other embodiments shown and described. Furthermore, this claim is not intended to be limited to any one example embodiment.

The foregoing method descriptions and program flow diagrams are provided as illustrative examples only, and are not intended to require or imply that the operations of the various embodiments must be performed in the order presented. As will be appreciated by those skilled in the art, the sequence of operations in the foregoing embodiments may be performed in any sequence. Words such as "thereafter," "then," "next," etc. are not intended to limit the order of operations; such words are used to guide the reader through the description of the methods. Furthermore, any reference to an element of a claim in the singular, eg, using the articles "a", "an", or "the", should not be construed as limiting the element to the singular.

The various illustrative logical blocks, modules, elements, circuits, and algorithmic operations in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative elements, blocks, modules, circuits, and operations have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the particular application and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such example decisions should not be interpreted as causing a departure from the scope of the present claims.

The hardware used to implement the various illustrative logics, logic blocks, modules, and circuits described in connection with the various embodiments disclosed herein may be general-purpose processors, digital signal processors (digital signal processors) designed to perform the functions described herein. DSP), application-specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), or other programmable logic devices, individual gate or transistor logic, individual hardware elements, or any combination thereof to implement or perform . A general-purpose processor may be a microprocessor, but in the alternative, the processor may be any known processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of receiver intelligence, eg, a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors combined with a DSP core, or any other such configuration. Alternatively, certain operations or methods may be performed by circuitry specific to a given function.

In one or more embodiments, the described functions may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored as one or more instructions or code on a non-transitory computer-readable storage medium or a non-transitory processor-readable storage medium. The operations of the methods or algorithms disclosed herein may be embodied in processor-executable software modules or processor-executable instructions, which may reside on a non-transitory computer-readable or processor-readable storage medium. A non-transitory computer-readable or processor-readable storage medium can be any storage medium that can be accessed by a computer or processor. By way of example and not limitation, such non-transitory computer-readable or processor-readable storage media may include RAM, ROM, EEPROM, flash memory, CD-ROM or other optical, magnetic or other magnetic Stores smart objects, or any other medium that can be used to store the required code in the form of instructions or data structures that can be accessed by a computer. Disk and disc, as used herein, includes compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disc, and blu-ray disc, where a disc usually reproduces data magnetically, and a disc usually reproduces data via laser optics reproduce the data. Combinations of the above are also included within the scope of non-transitory computer-readable and processor-readable media. Additionally, the operations of a method or algorithm may be resident as one or any combination or set of code and/or instructions on a non-transitory processor-readable storage medium and/or computer-readable storage medium, which may be incorporated into a computer program product.

The foregoing description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present claims. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the scope of the claims. Thus, the present case is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the following claims and the principles and novel features disclosed herein.

12: Vehicles 14: Vehicles 16: Vehicles 18: Communication Network 20: separation distance 30: Basic Safety Information 40: Basic Safety Information 50: Basic Safety Information 60: Communication link 62: Communication link 64: Communication link 66: Communication link 70: MBR preprocessing server 72: MBR preprocessing server 74: Misconduct Management Agency 100: Vehicle 101: Vehicles 102:V2X vehicle equipment 104:V2X vehicle equipment 106:V2X vehicle equipment 110: Base Station 120: Wireless Communication Equipment 122: Wireless communication link 124: Wireless communication link 126: Wireless communication link 132: Core Network 140: Control unit 140a: Processor 140b: memory 140c: Input module 140d: Output module 140e: Radio Module 142: Satellite Geolocation System Receiver 144: Occupancy Sensor 145: Control input sensor 146: Occupancy Sensor 148: Occupancy Sensor 150: Occupancy Sensor 152: Occupancy Sensor 154: Tire pressure sensor 156: Tire pressure sensor 158: Camera 160: Camera 162: Microphone 164: Microphone 166: Shock sensor 168: Radar 170: LIDAR 172a: Drive control elements 172b: Navigation Elements 172c: Sensor 200: Example Misconduct Management System 202: Radar perception layer 204: Camera perception layer 206: Positioning Engine Layer 208: Map Fusion and Arbitration Layer 210: Route planning layer 212: Sensor Fusion and Road World Model (RWM) Management 214: Motion Planning and Control Layer 216: Behavioral Planning and Prediction Layers 220: Drive-by-Wire (DBW) System/Control Unit 230: Wireless Communication Subsystem 250: Vehicle Management System 252: Collision Avoidance System 300: Processing equipment SOC 303: Digital Signal Processor (DSP) 304: modem processor 305: Processor 306: Image and Object Recognition Processor 307: Processor 308: Application Processor 310: Auxiliary Processor 312: Memory Components 314: Analogies and Custom Circuits 316: System Components and Resources 317: RPM Processor 318: Clock 320: Regulator 324: Interconnect/Bus Module 400: System 402: Computing Platform 404: Other V2X system participants 406: Machine executable instruction 408: LDM data receiving module 410: LDM data integration module 412: LDM data decision module 414: LDM data supply module 416: Map Generation Module 418: Map sending module 430: External Resources 432: Electronic Memory 434: Processor 500: Method 502: Blocks 504: Blocks 506: Blocks 508: Blocks 510: Decision Block 512: Square 514: Square 516: Square 518: Square 519: optional block 520: Square 522: Decision Block 700: Mobile Computing Devices 702: Processor 704: Touch Screen Controller 706: Internal memory 708: Transceiver 710: Antenna 712: Touch screen panel 714: Speaker 716: Cellular network wireless modem chip 718: Peripheral device connection interface 720: Shell 722: Power 724: Entity button 726: Power button 800: Laptop 802: Processor 808: Antenna 812: Volatile memory 813: Disk Drive 814: Floppy Disk Drive 815: Compact Disc (CD) Drive 816: Cellular Telephone Transceiver 817: Trackpad 818: Keyboard 819: Display 900: Server 901: Multicore Processor Assembly 902: volatile memory 904: Disk Drive 907: network access port 908: Internet

The accompanying drawings, which are incorporated herein and constitute a part of this specification, illustrate exemplary embodiments of the claimed item and, together with the general description provided above and the detailed description provided below, serve to explain the features of the claimed item.

1A and 1B are block diagrams illustrating elements of a vehicle suitable for implementing various embodiments.

1C is an element block diagram illustrating elements of a vehicle suitable for implementing various embodiments.

Figure ID is a schematic block diagram illustrating a subset of a V2X communication system suitable for implementing various embodiments.

2A is an element block diagram illustrating elements of an example vehicle management system in accordance with various embodiments.

2B is an element block diagram illustrating elements of another example vehicle management system in accordance with various embodiments.

3 is a block diagram illustrating elements of a system-on-a-chip for use in a vehicle, according to various embodiments.

4 is a block diagram of elements illustrating a system configured to generate local dynamic map material, according to various embodiments.

5 is a program flow diagram illustrating the operation of a method performed by a V2X-equipped processor for comparing data in a received V2X message with data in an LDM data model, according to various embodiments Detect inappropriate behavior in V2X messages.

6 is a program flow diagram illustrating the operation of a method for comparing data in a received V2X message to data in an LDM data model, according to various embodiments.

7 is a block diagram illustrating elements of an example mobile computing device suitable for use with various embodiments.

8 is a block diagram illustrating elements of an example mobile computing device suitable for use with various embodiments.

9 is a block diagram of components illustrating an example server suitable for use with various embodiments.

Domestic storage information (please note in the order of storage institution, date and number) none Foreign deposit information (please note in the order of deposit country, institution, date and number) none

500: Method

502: Blocks

504: Blocks

506: Blocks

508: Blocks

510: Decision Block

512: Square

514: Square

516: Square

Claims (32)

A method executed by a processor of a vehicle for detecting a misconduct condition in a vehicle-to-everything (V2X) system, comprising: receive a V2X message from another V2X system participant, wherein the V2X message includes data about an environment surrounding the vehicle; Compare the data contained in the received V2X message with a local dynamic map (LDM) data model maintained or stored by a local terminal to detect misconduct; generating a misconduct report identifying a misconduct condition in response to detecting the misconduct condition based on the comparison; and Send the resulting misconduct report to a misconduct management agency. According to the method of claim 1, further comprising: monitoring a plurality of sensors in the vehicle to collect additional information about an environment surrounding the vehicle; generating the LDM data model representing the vehicle surroundings based at least in part on an aggregation of additional data collected from the plurality of sensors; and The LDM data model is stored in memory. According to the method of claim 1, further comprising: In response to a determination that a misconduct condition was not detected: perform calculations based on at least one of the observed object dynamics in the LDM data model or new data inputs received from the V2X message; modify the LDM data model to incorporate calculations and data included in the received V2X message; and The LDM data model maintained or stored in memory is replaced with the modified LDM data model. The method of claim 1, wherein sending the generated misconduct report to a misconduct management agency includes sending a representation of the LDM data model. The method of claim 4, wherein the representation of the LDM data model includes an incomplete data set of the LDM data model. The method of claim 1, further comprising receiving feedback from the misconduct management agency, wherein the feedback includes corrective actions for mitigating the misconduct condition. The method of claim 1, wherein the data about the surroundings of the vehicle included in the received V2X message includes traffic information. The method of claim 1, wherein the data about the surroundings of the vehicle included in the received V2X message includes location information of neighboring vehicles based on global navigation satellite system data. The method of claim 1, wherein the data about the surroundings of the vehicle included in the received V2X message includes map data detailing road geometry and street furniture. The method of claim 1, wherein comparing the data included in the received V2X message with an LDM data model maintained or stored by the local end to detect misconduct conditions includes determining the data included in the received V2X message. Whether any data conflicts with the information in the LDM data model maintained or stored by the local end. The method according to claim 1, wherein comparing the data included in the received V2X message with the LDM data model maintained or stored by the local end to detect misconduct conditions includes: selecting a subset of data elements in the locally maintained or stored LDM data model for comparison with data included in the received V2X message; and Determines whether any data included in the received V2X message conflicts with the selected subset of data elements in the LDM data model maintained or stored by the local end. The method of claim 1, wherein comparing the data included in the received V2X message with an LDM data model maintained or stored by the local end to detect misconduct conditions includes determining the proximity to which the received V2X message was sent Whether the status or position information of the vehicle conflicts with the status or position information of the adjacent vehicle in the LDM data model maintained or stored by the local terminal. A vehicle-to-everything (V2X) processing device, comprising: a processor configured with processor-executable instructions for: receiving a V2X message from another V2X system participant, wherein the V2X message contains data about an environment surrounding the vehicle in which the V2X processing device is installed; Compare the data contained in the received V2X message with a local dynamic map (LDM) data model maintained or stored by a local terminal to detect misconduct; generating a misconduct report identifying a misconduct condition in response to detecting the misconduct condition based on the comparison; and Send the resulting misconduct report to a misconduct management agency. The V2X processing device of claim 13, wherein the processor is further configured with processor-executable instructions to: monitoring a plurality of sensors in the vehicle to collect additional information about an environment surrounding the vehicle; generating the LDM data model representing the vehicle surroundings based at least in part on an aggregation of additional data collected from the plurality of sensors; and The LDM data model is stored in memory. The V2X processing device of claim 13, wherein the processor is further configured with processor-executable instructions to: In response to determining that a misconduct condition was not detected, performing a calculation based on at least one of the observed object dynamics in the LDM data model or new data input received from the V2X message; modify the LDM data model to incorporate calculations and data included in the received V2X message; and The LDM data model maintained or stored in memory is replaced with the modified LDM data model. The V2X processing apparatus of claim 13, wherein the processor is further configured with processor-executable instructions to include a representation of the LDM data model in the generated misconduct report sent to the misconduct authority. The V2X processing apparatus of claim 16, wherein the representation of the LDM data model includes an incomplete data set of the LDM data model. The V2X processing device of claim 13, wherein the processor is further configured with processor-executable instructions to receive feedback from the misconduct management agency, wherein the feedback includes corrective actions for mitigating the misconduct condition. The V2X processing apparatus according to claim 13, wherein the data about the surroundings of the vehicle included in the received V2X message includes traffic information. The V2X processing apparatus of claim 13, wherein the data about the vehicle's surroundings included in the received V2X message includes location information of neighboring vehicles based on global navigation satellite system data. The V2X processing apparatus of claim 13, wherein the data about the environment surrounding the vehicle included in the received V2X message includes map data detailing road geometry and street furniture. The V2X processing device according to claim 13, wherein the processor is further configured with processor-executable instructions to compare the data included in the received V2X message with the LDM data model maintained or stored by the local end to detect To determine whether any data included in the received V2X message conflicts with the information in the LDM data model maintained or stored on the local end by detecting misconduct. The V2X processing device according to claim 13, wherein the processor is further configured with processor-executable instructions to perform processing between the data included in the received V2X message and the LDM data model maintained or stored by the local end in the following manner Compare to detect misconduct conditions: selecting a subset of data elements in the locally maintained or stored LDM data model for comparison with data included in the received V2X message; and Determines whether any data included in the received V2X message conflicts with the selected subset of data elements in the LDM data model maintained or stored by the local end. The V2X processing device according to claim 13, wherein the processor is further configured with processor-executable instructions to compare the data included in the received V2X message with the LDM data model maintained or stored by the local end to detect The misbehavior is detected to determine whether the state or location information of a neighboring vehicle sending the received V2X message conflicts with the state or location information of the neighboring vehicle in the LDM data model maintained or stored by the local end. A vehicle-to-everything (V2X) processing device, comprising: means for receiving a V2X message from another V2X system participant, wherein the V2X message contains data about an environment around a vehicle in which the V2X processing device is installed; A component used to detect misconduct by comparing the data contained in the received V2X message with a local dynamic map (LDM) data model maintained or stored by a local terminal; means for generating a misconduct report identifying a misconduct condition in response to detecting a misconduct condition based on the comparison; and A component for sending generated misconduct reports to a misconduct management agency. V2X processing equipment according to claim 25, also including: means for monitoring a plurality of sensors in the vehicle to collect additional information about an environment surrounding the vehicle; means for generating the LDM data model representing the vehicle surroundings based at least in part on an aggregation of additional data collected from the plurality of sensors; and The component used to store the LDM data model. V2X processing equipment according to claim 25, also including: means for performing a calculation based on at least one of the observed object dynamics in the LDM data model or new data input received from the V2X message in response to determining that a misconduct condition was not detected; means for modifying the LDM data model to incorporate calculations and data included in the received V2X message; and A component for replacing the LDM data model maintained or stored in memory with the modified LDM data model. The V2X processing apparatus of claim 25, wherein the means for sending the generated misconduct report to a misconduct management agency includes means for sending a representation of the LDM data model. The V2X processing device of claim 25, further comprising means for receiving feedback from the misconduct management agency, wherein the feedback includes corrective actions for mitigating the misconduct condition. The V2X processing device of claim 25, wherein the means for comparing the data included in the received V2X message with the LDM data model maintained or stored at the local end to detect misconduct conditions includes means for determining the A component of whether any data included in the received V2X message conflicts with the information in the LDM data model maintained or stored by the local end. The V2X processing device of claim 25, wherein the means for comparing the data included in the received V2X message with the LDM data model maintained or stored at the local end to detect misconduct conditions include: means for selecting a subset of data elements in the locally maintained or stored LDM data model for comparison with the data included in the received V2X message; and Means for determining whether any data included in the received V2X message conflicts with the selected subset of data elements within the LDM data model maintained or stored by the local end. The V2X processing device according to claim 25, wherein the means for comparing the data included in the received V2X message with the LDM data model maintained or stored at the local end to detect misconduct conditions includes means for determining to send A component of whether the state or location information of a neighboring vehicle in the received V2X message conflicts with the state or location information of the neighboring vehicle in the LDM data model maintained or stored by the local end.

Images