WO2022000127A1

WO2022000127A1 - Target tracking method and device therefor

Info

Publication number: WO2022000127A1
Application number: PCT/CN2020/098494
Authority: WO
Inventors: 李晓波; 劳大鹏
Original assignee: 华为技术有限公司
Priority date: 2020-06-28
Filing date: 2020-06-28
Publication date: 2022-01-06
Also published as: CN113498529A; CN113498529B

Abstract

Provided in the embodiments of the present application is a target tracking method, which is applied to the field of automatic driving. The method comprises: acquiring point cloud data of a target object, and performing centroid estimation on the point cloud data; acquiring a target covariance matrix, wherein the target covariance matrix represents a centroid position error introduced during the centroid estimation; and performing target tracking on the target object on the basis of the target covariance matrix so as to obtain a target tracking result. In the present application, the target covariance matrix may relatively accurately quantify the error caused by the centroid estimation of the point cloud data, and performing target tracking on the target object on the basis of the target covariance matrix may improve the accuracy of target tracking.

Description

A target tracking method and device thereof

technical field

The present application relates to the field of automatic driving, and in particular, to a target tracking method and device thereof.

Background technique

Advanced driver assistant system (ADAS) and autonomous driving system (ADS) can automatically plan the vehicle's driving path reasonably and control the driving state of the vehicle. For example, ADAS and ADS can sense the vehicle in time. Use the relevant road information on the current road, and make timely and correct driving operations accordingly, so as to avoid vehicle accidents caused by the driver's inattention or unresponsiveness. It is precisely because ADAS and ADS have the above advantages that ADAS and ADS have become a research hotspot of current artificial intelligence. As the future development trend of automobiles, ADAS and ADS have broad development prospects.

With the development of ADAS and ADS, ADAS and ADS put forward higher requirements for the performance of vehicle radar such as distance and angular resolution. The improvement of the range and angular resolution of the on-board radar enables the on-board radar system to detect multiple measurement points when imaging the target, forming a high-resolution point cloud, which we call point cloud imaging radar.

Target tracking is a key part of vehicle-mounted radar for intelligent driving applications. The performance of target tracking depends on the selection of target motion model and filtering algorithm, while the performance of track filtering mainly depends on the covariance matrix of the measurement equation and the state equation, and the covariance matrix The more accurate the estimation, the higher the target tracking accuracy.

In the prior art, when performing target tracking, only the measurement covariance matrix related to the performance of the vehicle-mounted radar is considered, so that the accuracy of the target tracking result is poor.

SUMMARY OF THE INVENTION

In a first aspect, the present application provides a target tracking method, the method includes: obtaining point cloud data of a target object, and performing centroid estimation on the point cloud data; obtaining a target covariance matrix; wherein, the target covariance matrix represents The centroid position error introduced when the centroid estimation is performed; and, based on the target covariance matrix, target tracking is performed on the target object to obtain a target tracking result.

It should be understood that the centroid estimation result obtained after the above centroid estimation may be the state measurement value at the current moment, and the state measurement value and the target covariance matrix may be used as input data during target tracking.

It should be understood that, in this embodiment of the present application, the radar system may scan the target object to obtain point cloud data of the target object. The point cloud data is obtained by scanning the target object with the radar system, and the point cloud data can be recorded in the form of points, and each point can include coordinates and motion states. Or other types of target objects have turns or occlusions, and the centroid estimation of the point cloud data of the same target object in different frames may experience positional disturbance. The measurement error related to the point cloud measurement performance of the system cannot be accurately quantified due to the estimation of the centroid of the point cloud data. Therefore, the accuracy of the target tracking will be reduced. In the embodiment of the present application, the target covariance matrix can be relative to To accurately quantify the error caused by the centroid estimation of the point cloud data, if the target object is tracked based on the target covariance matrix, the accuracy of the target tracking can be improved.

In an optional implementation, an innovation covariance matrix may be constructed based on the target covariance matrix, and target tracking is performed on the target object based on the innovation covariance matrix to obtain a target tracking result.

In an optional implementation, the obtaining the target covariance matrix includes:

Obtain the target tracking result of the target object at the previous moment;

Based on the motion model, state prediction is performed on the target tracking result at the previous moment to obtain the target tracking prediction value at the current moment;

Obtaining a state measurement value at the current moment, where the state measurement value is obtained by performing centroid estimation on the point cloud data;

The target covariance matrix is obtained based on the target tracking predicted value and the state measurement value.

If the current target tracking process is still in the initial stage, the covariance matrices used in the filtering are not close to the true value (ie, the correct value), and the target tracking at the current moment cannot be performed based on the historical target tracking results. The target tracking is performed according to the target covariance matrix set by the empirical value, wherein the target covariance matrix can represent the centroid position error introduced during centroid estimation.

In the case that the number of data associations is less than or equal to a preset value, the preset covariance matrix is used as the target covariance matrix; wherein, the number of data associations indicates that the difference between the centroid measurement value and the state predicted value is less than a threshold value number of times, the centroid measurement value is obtained by estimating the centroid of the point cloud data of the target object at the current moment, and the state prediction value is obtained by predicting the centroid measurement value at the previous moment. The measured value is obtained by estimating the centroid of the point cloud data of the target object at the previous moment.

Optionally, the moment in this embodiment may be granular in frames or other time units, for example, the previous moment may be the previous frame, and the current moment may be the current frame.

When the number of data associations is greater than the preset value, the target covariance matrix is obtained based on the target tracking predicted value and the state measurement value; wherein the target tracking predicted value is the target tracking at the previous moment The result is obtained by performing state prediction, and the state measurement value is obtained by performing centroid estimation on the point cloud data.

In this embodiment of the present application, the target covariance matrix may be continuously updated according to the progress of target tracking, and each iteration (that is, each time the target covariance matrix at the current moment is calculated) is based on the target tracking prediction value and The state measurement value at the current moment is updated. With the iteration of the target tracking process, the target covariance matrix will gradually approach or reach the true value, thereby making the target tracking result more accurate.

In an optional implementation, obtaining the target covariance matrix based on the target tracking predicted value and the state measurement value includes: obtaining the target by the following formula based on the target tracking predicted value and the state measurement value Covariance matrix:

Among them, the

is the state measurement value, the

Track the predicted value for the target, the R _{1 is} the target covariance matrix.

In an optional implementation, the method further includes: acquiring a measurement equation covariance matrix, where the point cloud data is acquired based on a radar system, and the measurement equation covariance matrix represents the measurement deviation of the radar system; The measurement equation covariance matrix and the target covariance matrix are combined to obtain a combined target covariance matrix; correspondingly, the target tracking of the target object based on the target covariance matrix includes: based on the combined target covariance matrix The target covariance matrix of the target object is tracked.

In the embodiment of the present application, the synthesized target covariance matrix includes information related to the centroid position error introduced during centroid estimation, and the synthesized target covariance matrix relatively accurately quantifies the error caused by the centroid estimation of the point cloud data. Tracking the target object based on the synthesized target covariance matrix can improve the accuracy of target tracking.

In an optional implementation, the measurement equation covariance matrix is a matrix represented in a polar coordinate system, and the target covariance matrix is a matrix represented in a Cartesian coordinate system; the measurement equation covariance matrix and Synthesizing the target covariance matrix includes: converting the measurement equation covariance matrix into a matrix represented in a Cartesian coordinate system; performing matrix addition on the converted measurement equation covariance matrix and the target covariance matrix to obtain The synthesized target covariance matrix.

Usually due to the characteristics of the radar system, the point cloud data collected by the radar system is usually the data expressed in the polar coordinate system. Therefore, the measurement equation covariance matrix related to the performance of the radar system is usually the matrix expressed in the polar coordinate system. The target covariance matrix is a matrix represented in the Cartesian coordinate system. In this case, the measurement equation covariance matrix can be converted into a matrix represented in the Cartesian coordinate system, and the converted measurement equation covariance The matrix and the target covariance matrix are matrix added to obtain the combined target covariance matrix.

It should be noted that in some scenarios, the point cloud data collected by the radar system is not the data expressed in the polar coordinate system, it only needs to be converted into the data expressed in the same coordinate system as the covariance matrix of the measurement equation. For example, the point cloud data collected by the radar system is the data expressed in the Cartesian coordinate system, and the target covariance matrix is the matrix expressed in the polar coordinate system, then only the covariance matrix of the measurement equation needs to be converted into the polar coordinate system. Can.

In an optional implementation, converting the covariance matrix of the measurement equation into a matrix represented in a Cartesian coordinate system includes: acquiring a state measurement value at the current moment, where the state measurement value is a measurement of the point cloud data. Obtained by centroid estimation; based on the state measurement value, a covariance transformation synthesis matrix is obtained; based on the covariance transformation synthesis matrix, the measurement equation covariance matrix is transformed into a matrix represented in a Cartesian coordinate system.

In the embodiment of the present application, a covariance transformation synthesis matrix may be constructed first, and the covariance transformation synthesis matrix may be used to convert the covariance matrix from a polar coordinate system to an expression in a Cartesian coordinate system; in one implementation, The covariance transformation synthesis matrix is related to the state measurement value at the current moment. Specifically, the state measurement value at the current moment can be Z _k = [R _k , θ _k , v _rk ], where R _k can represent the radar system and the center of mass. Distance, θ _k can represent the orientation of the centroid, and v _rk can represent the radial velocity of the centroid; based on the distance between the radar system and the centroid in the state measurements, and the orientation of the centroid, the covariance transformation can be constructed in the synthesis matrix , and fill the other part of the covariance transformation composition matrix with 0s and 1s.

In an optional implementation, performing matrix addition on the converted measurement equation covariance matrix and the target covariance matrix includes: by the following formula, adding the converted measurement equation covariance matrix to the target covariance matrix Do matrix addition:

R _all =R ₀ +A _k R _m A _k ′;

Wherein, the R _{0 is} the target covariance matrix, the A _k R _m A _k ′ is the converted measurement equation covariance matrix, the R _k and the θ _{k are} the state measurement values, and the A _{k is} the covariance matrix The variance transformation synthesis matrix, the A _k ' is the transpose of the covariance transformation synthesis matrix.

In a second aspect, the present application provides a target tracking device, the device comprising:

an acquisition module for acquiring point cloud data of the target object, and performing centroid estimation on the point cloud data; acquiring a target covariance matrix; wherein, the target covariance matrix represents a centroid position error introduced when performing the centroid estimation; and,

The target tracking module is configured to perform target tracking on the target object based on the target covariance matrix to obtain a target tracking result.

In an optional implementation, the acquisition module is used to acquire the target tracking result of the target object at the previous moment;

In an optional implementation, the acquisition module is configured to use a preset covariance matrix as the target covariance matrix when the number of data associations is less than or equal to a preset value; wherein the data The number of associations represents the number of times that the difference between the centroid measurement value and the state predicted value is less than the threshold, the centroid measurement value is obtained by performing centroid estimation on the point cloud data of the target object at the current moment, and the state predicted value is the previous moment. The centroid measurement value of the last moment is obtained by predicting the centroid measurement value of the target object, and the centroid measurement value of the target object at the last moment is obtained by performing centroid estimation on the point cloud data of the target object.

In an optional implementation, the obtaining module is configured to obtain the target covariance matrix based on the target tracking predicted value and the state measurement value when the data association times are greater than the preset value; The target tracking prediction value is obtained by performing state prediction on the target tracking result at the previous moment, and the state measurement value is obtained by performing centroid estimation on the point cloud data.

In an optional implementation, the obtaining module is specifically used for:

Based on the target tracking predicted value and the state measurement value, the target covariance matrix is obtained by the following formula:

Among them, the x _k+1|k and y _{k-1|k are} the state measurement values, and the

In an optional implementation, the acquisition module is also used to:

Obtain the covariance matrix of the measurement equation, wherein the point cloud data is collected based on the radar system, and the covariance matrix of the measurement equation represents the measurement deviation of the radar system;

The device also includes:

a matrix synthesis module for synthesizing the measurement equation covariance matrix and the target covariance matrix to obtain a synthesized target covariance matrix;

Correspondingly, the target tracking module is specifically configured to: perform target tracking on the target object based on the synthesized target covariance matrix.

In an optional implementation, the measurement equation covariance matrix is a matrix represented in a polar coordinate system, and the target covariance matrix is a matrix represented in a Cartesian coordinate system; the matrix synthesis module is specifically used for:

convert the measurement equation covariance matrix to a matrix represented in Cartesian coordinates;

Matrix addition is performed between the converted measurement equation covariance matrix and the target covariance matrix to obtain a synthesized target covariance matrix.

In an optional implementation, the matrix synthesis module is specifically used for:

Obtain the state measurement value at the current moment, which is obtained by estimating the centroid of the point cloud data;

Based on the state measurement value, a covariance transformation synthesis matrix is obtained; wherein, the covariance transformation synthesis matrix includes a plurality of elements, and some elements in the plurality of elements are generated based on the state measurement value;

Based on the covariance transformation synthesis matrix, the measurement equation covariance matrix is transformed into a matrix represented in a Cartesian coordinate system.

The converted measurement equation covariance matrix and the target covariance matrix are matrix added by the following formula:

R _all =R ₀ +A _k R _m A _k ′;

In a third aspect, the present application provides a target tracking device, comprising: a processor and a memory; the memory stores program codes; when the processor calls the program codes in the memory, the target tracking device executes the above-mentioned first aspect any of the methods.

For the steps performed by the target tracking apparatus in each possible implementation manner of the first aspect performed by the processor, reference may be made to the first aspect for details, which will not be repeated here.

In a fourth aspect, the present application provides an autonomous vehicle, which may include a processor, the processor is coupled to a memory, the memory stores program instructions, and the method of the first aspect is implemented when the program instructions stored in the memory are executed by the processor. .

For the steps performed by the automatic driving vehicle in each possible implementation manner of the processor executing the first aspect, reference may be made to the first aspect for details, and details are not repeated here.

In a fifth aspect, the present application provides a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, and when it runs on a computer, causes the computer to execute the method of the first aspect.

In a sixth aspect, the present application provides a circuit system including a processing circuit configured to perform the method of any one of the above-mentioned first aspect or various optional implementations of the first aspect.

In a seventh aspect, the present application provides a computer program that, when run on a computer, causes the computer to execute the method of the first aspect.

In an eighth aspect, the present application provides a chip system, the chip system includes a processor for supporting a server or a threshold value acquisition device to implement the functions involved in the above aspects, for example, sending or processing the above methods. data and/or information. In a possible design, the chip system further includes a memory for storing necessary program instructions and data of the server or the communication device. The chip system may be composed of chips, or may include chips and other discrete devices.

An embodiment of the present application provides a target tracking method. The method includes: acquiring point cloud data of a target object, and performing centroid estimation on the point cloud data; and, based on the target covariance matrix, perform target tracking on the target object to obtain the target tracking result. Due to the turning or occlusion of vehicles or other types of target objects, the centroid estimation of point cloud data of the same target object in different frames may experience positional disturbance. In the prior art, during target tracking, the error covariance matrix only considers The measurement error related to the point cloud measurement performance of the radar system cannot accurately quantify the error caused by the centroid estimation of the point cloud data. Therefore, the accuracy of target tracking will be reduced. In the embodiment of the present application, the target covariance matrix The error caused by the centroid estimation of the point cloud data can be relatively accurately quantified. Therefore, if the target object is tracked based on the target covariance matrix, the accuracy of the target tracking can be improved.

Description of drawings

FIG. 1 is a functional block diagram of a vehicle 100 according to an embodiment of the present invention;

2a is a schematic diagram of an application scenario provided by an embodiment of the present invention;

FIG. 2b is a schematic diagram of an application scenario provided by an embodiment of the present invention;

3 is a schematic diagram of a computer system provided by an embodiment of the present invention;

Fig. 4 is a kind of schematic diagram of point cloud data centroid estimation;

FIG. 5 is a schematic flowchart of a target tracking method provided by an embodiment of the present application;

FIG. 6 is a schematic flowchart of a target tracking method provided by an embodiment of the present application;

FIG. 7 is a schematic flowchart of a target tracking method provided by an embodiment of the present application;

FIG. 8 is a schematic diagram of a target tracking apparatus provided by an embodiment of the present application;

FIG. 9 is a schematic diagram of a computer program product provided by an embodiment of the present application;

FIG. 10 is a schematic diagram of a target tracking apparatus provided by an embodiment of the present application.

detailed description

The embodiments of the present invention will be described below with reference to the accompanying drawings in the embodiments of the present invention.

The terms "first", "second", "third" and "fourth" in the description and claims of the present application and the drawings are used to distinguish different objects, rather than to describe a specific order . Furthermore, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device comprising a series of steps or units is not limited to the listed steps or units, but optionally also includes unlisted steps or units, or optionally also includes For other steps or units inherent to these processes, methods, products or devices.

Reference herein to an "embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the present application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor a separate or alternative embodiment that is mutually exclusive of other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

The terms "component", "module", "system" and the like are used in this specification to refer to a computer-related entity, hardware, firmware, a combination of hardware and software, software, or software in execution. For example, a component may be, but is not limited to, a process running on a processor, a processor, an object, an executable, a thread of execution, a program, and/or a computer. By way of illustration, both an application running on a computing device and the computing device may be components. One or more components may reside within a process and/or thread of execution, and a component may be localized on one computer and/or distributed between 2 or more computers. In addition, these components can execute from various computer readable media having various data structures stored thereon. A component may, for example, be based on a signal having one or more data packets (eg, data from two components interacting with another component between a local system, a distributed system, and/or a network, such as the Internet interacting with other systems via signals) Communicate through local and/or remote processes.

FIG. 1 is a functional block diagram of a vehicle 100 according to an embodiment of the present invention. In one embodiment, the vehicle 100 is configured in a fully or partially autonomous driving mode. For example, the vehicle 100 can control itself while in an autonomous driving mode, and can determine the current state of the vehicle and its surroundings through human manipulation, determine the possible behavior of at least one other vehicle in the surrounding environment, and determine the other vehicles perform The confidence level corresponding to the likelihood of the possible behavior, the vehicle 100 is controlled based on the determined information. When the vehicle 100 is in an autonomous driving mode, the vehicle 100 may be placed to operate without human interaction.

Vehicle 100 may include various subsystems, such as travel system 102 , sensor system 104 , control system 106 , one or more peripherals 108 and power supply 110 , computer system 112 , and user interface 116 . Alternatively, vehicle 100 may include more or fewer subsystems, and each subsystem may include multiple elements. Additionally, each of the subsystems and elements of the vehicle 100 may be interconnected by wire or wirelessly.

The travel system 102 may include components that provide powered motion for the vehicle 100 . In one embodiment, propulsion system 102 may include engine 118 , energy source 119 , transmission 120 , and wheels/tires 121 . The engine 118 may be an internal combustion engine, an electric motor, an air compression engine, or other types of engine combinations, such as a hybrid engine consisting of a gas oil engine and an electric motor, a hybrid engine consisting of an internal combustion engine and an air compression engine. Engine 118 converts energy source 119 into mechanical energy.

Examples of energy sources 119 include gasoline, diesel, other petroleum-based fuels, propane, other compressed gas-based fuels, ethanol, solar panels, batteries, and other sources of electricity. The energy source 119 may also provide energy to other systems of the vehicle 100 .

Transmission 120 may transmit mechanical power from engine 118 to wheels 121 . Transmission 120 may include a gearbox, a differential, and a driveshaft. In one embodiment, transmission 120 may also include other devices, such as clutches. Among other things, the drive shaft may include one or more axles that may be coupled to one or more wheels 121 .

The sensor system 104 may include several sensors that sense information about the environment surrounding the vehicle 100 . For example, the sensor system 104 may include a positioning system 122 (the positioning system may be a GPS system, a Beidou system or other positioning systems), an inertial measurement unit (IMU) 124, a radar system 126, a laser rangefinder 128 and camera 130 . The sensor system 104 may also include sensors of the internal systems of the vehicle 100 being monitored (eg, an in-vehicle air quality monitor, a fuel gauge, an oil temperature gauge, etc.). Sensor data from one or more of these sensors can be used to detect objects and their corresponding characteristics (position, shape, orientation, velocity, etc.). This detection and identification is a critical function for the safe operation of the autonomous vehicle 100 .

Among them, with the development of advanced driver assistance systems (ADAS) and unmanned driving technologies, higher requirements are placed on the performance of the radar system 126 such as distance and angular resolution. The improvement of the distance and angular resolution of the vehicle-mounted radar system 126 enables the vehicle-mounted radar system 126 to detect multiple measurement points for a target object when imaging the target to form high-resolution point cloud data. The radar system in this application 126 may also be referred to as point cloud imaging radar.

The positioning system 122 may be used to estimate the geographic location of the vehicle 100 . The IMU 124 is used to sense position and orientation changes of the vehicle 100 based on inertial acceleration. In one embodiment, IMU 124 may be a combination of an accelerometer and a gyroscope.

Radar system 126 may utilize radio signals to sense objects within the surrounding environment of vehicle 100 . In some embodiments, in addition to sensing objects, radar system 126 may be used to sense the speed and/or heading of objects.

The laser rangefinder 128 may utilize laser light to sense objects in the environment in which the vehicle 100 is located. In some embodiments, the laser rangefinder 128 may include one or more laser sources, laser scanners, and one or more detectors, among other system components.

Camera 130 may be used to capture multiple images of the surrounding environment of vehicle 100 . Camera 130 may be a still camera or a video camera.

The control system 106 controls the operation of the vehicle 100 and its components. Control system 106 may include various elements including steering system 132 , throttle 134 , braking unit 136 , sensor fusion algorithms 138 , computer vision system 140 , route control system 142 , and obstacle avoidance system 144 .

The steering system 132 is operable to adjust the heading of the vehicle 100 . For example, in one embodiment it may be a steering wheel system.

The throttle 134 is used to control the operating speed of the engine 118 and thus the speed of the vehicle 100 .

The braking unit 136 is used to control the deceleration of the vehicle 100 . The braking unit 136 may use friction to slow the wheels 121 . In other embodiments, the braking unit 136 may convert the kinetic energy of the wheels 121 into electrical current. The braking unit 136 may also take other forms to slow the wheels 121 to control the speed of the vehicle 100 .

Computer vision system 140 may process and analyze images captured by camera 130 in order to identify objects and/or features in the environment surrounding vehicle 100 . The objects and/or features may include traffic signals, road boundaries and obstacles. Computer vision system 140 may use object recognition algorithms, Structure from Motion (SFM) algorithms, video tracking, and other computer vision techniques. In some embodiments, the computer vision system 140 may be used to map the environment, track objects, estimate the speed of objects, and the like.

The route control system 142 is used to determine the travel route of the vehicle 100 . In some embodiments, the route control system 142 may combine data from the sensors 138 , the GPS 122 , and one or more predetermined maps to determine a driving route for the vehicle 100 .

The obstacle avoidance system 144 is used to identify, evaluate, and avoid or otherwise traverse potential obstacles in the environment of the vehicle 100 .

Of course, in one example, the control system 106 may additionally or alternatively include components other than those shown and described. Alternatively, some of the components shown above may be reduced.

Vehicle 100 interacts with external sensors, other vehicles, other computer systems, or users through peripheral devices 108 . Peripherals 108 may include a wireless communication system 146 , an onboard computer 148 , a microphone 150 and/or a speaker 152 .

In some embodiments, peripherals 108 provide a means for a user of vehicle 100 to interact with user interface 116 . For example, the onboard computer 148 may provide information to the user of the vehicle 100 . User interface 116 may also operate on-board computer 148 to receive user input. The onboard computer 148 can be operated via a touch screen. In other cases, peripheral devices 108 may provide a means for vehicle 100 to communicate with other devices located within the vehicle. For example, microphone 150 may receive audio (eg, voice commands or other audio input) from a user of vehicle 100 . Similarly, speakers 152 may output audio to a user of vehicle 100 .

Wireless communication system 146 may wirelessly communicate with one or more devices, either directly or via a communication network. For example, wireless communication system 146 may use 3G cellular communications such as code division multiple access (CDMA), enhanced versatile disk (EVD), global system for mobile communications , GSM)/general packet radio service (GPRS), or 4G cellular communications such as LTE. Or 5G cellular communications. The wireless communication system 146 may communicate with a wireless local area network (WLAN) using WiFi. In some embodiments, the wireless communication system 146 may communicate directly with the device using an infrared link, Bluetooth, or ZigBee. Other wireless protocols, such as various vehicle communication systems, for example, wireless communication system 146 may include one or more dedicated short range communications (DSRC) devices, which may include communication between vehicles and/or roadside stations public and/or private data communications.

The power supply 110 may provide power to various components of the vehicle 100 . In one embodiment, the power source 110 may be a rechargeable lithium-ion or lead-acid battery. One or more battery packs of such a battery may be configured as a power source to provide power to various components of the vehicle 100 . In some embodiments, power source 110 and energy source 119 may be implemented together, such as in some all-electric vehicles.

Some or all of the functions of the vehicle 100 are controlled by the computer system 112 . Computer system 112 may include at least one processor 113 that executes instructions 115 stored in a non-transitory computer-readable medium such as data storage device 114 . Computer system 112 may also be multiple computing devices that control individual components or subsystems of vehicle 100 in a distributed fashion.

The processor 113 may be any conventional processor, such as a commercially available central processing unit (CPU). Alternatively, the processor may be a dedicated device such as an application specific integrated circuit (ASIC) or other hardware-based processor. Although FIG. 1 functionally illustrates the processor, memory, and other elements of the computer 110 in the same block, one of ordinary skill in the art will understand that the processor, computer, or memory may actually include a processor, a computer, or a memory that may or may not Multiple processors, computers, or memories stored within the same physical enclosure. For example, the memory may be a hard drive or other storage medium located within an enclosure other than computer 110 . Thus, reference to a processor or computer will be understood to include reference to a collection of processors or computers or memories that may or may not operate in parallel. Rather than using a single processor to perform the steps described herein, some components, such as the steering and deceleration components, may each have their own processors that only perform computations related to component-specific functions.

In this embodiment of the present application, the processor 113 may acquire the sensor system (eg, point movement data collected by the radar system 126 ) and motion model parameters, etc., and then perform target tracking based on the point cloud data and motion model parameters.

In various aspects described herein, a processor may be located remotely from the vehicle and in wireless communication with the vehicle. In other aspects, some of the processes described herein are performed on a processor disposed within the vehicle while others are performed by a remote processor, including taking steps necessary to perform a single maneuver.

In some embodiments, data storage 114 may include instructions 115 (eg, program logic) executable by processor 113 to perform various functions of vehicle 100 , including those described above. Data storage 114 may also contain additional instructions, including sending data to, receiving data from, interacting with, and/or performing data processing on one or more of propulsion system 102 , sensor system 104 , control system 106 , and peripherals 108 . control commands.

In addition to the instructions 115, the data storage device 114 may store data such as road maps, route information, vehicle location, direction, speed, and other vehicle data, among other information. Such information may be used by the vehicle 100 and the computer system 112 while the vehicle 100 is in autonomous, semi-autonomous, and/or manual modes.

A user interface 116 for providing information to or receiving information from a user of the vehicle 100 . Optionally, the user interface 116 may include one or more input/output devices within the set of peripheral devices 108 , such as a wireless communication system 146 , an onboard computer 148 , a microphone 150 and a speaker 152 .

Computer system 112 may control functions of vehicle 100 based on input received from various subsystems (eg, travel system 102 , sensor system 104 , and control system 106 ) and from user interface 116 . For example, computer system 112 may utilize input from control system 106 in order to control steering unit 132 to avoid obstacles detected by sensor system 104 and obstacle avoidance system 144 . In some embodiments, computer system 112 is operable to provide control of various aspects of vehicle 100 and its subsystems.

Alternatively, one or more of these components described above may be installed or associated with the vehicle 100 separately. For example, data storage device 114 may exist partially or completely separate from vehicle 1100 . The above-described components may be communicatively coupled together in a wired and/or wireless manner.

Optionally, the above component is just an example. In practical applications, components in each of the above modules may be added or deleted according to actual needs, and FIG. 1 should not be construed as a limitation on the embodiment of the present invention.

A self-driving car traveling on a road, such as vehicle 100 above, can recognize objects within its surroundings to determine adjustments to the current speed. The objects may be other vehicles, traffic control devices, or other types of objects. In some examples, each identified object may be considered independently, and based on the object's respective characteristics, such as its current speed, acceleration, distance from the vehicle, etc., may be used to determine the speed at which the autonomous vehicle is to adjust.

Alternatively, the autonomous vehicle vehicle 100 or a computing device associated with the autonomous vehicle 100 (eg, computer system 112, computer vision system 140, data storage device 114 of FIG. 1) may be based on the characteristics of the identified objects and the surrounding environment state (eg, traffic, rain, ice on the road, etc.) to predict the behavior of the identified objects. Optionally, each identified object is dependent on the behavior of the other, so it is also possible to predict the behavior of a single identified object by considering all identified objects together. The vehicle 100 can adjust its speed based on the predicted behavior of the identified object. In other words, the self-driving car can determine what steady state the vehicle will need to adjust to (eg, accelerate, decelerate, or stop) based on the predicted behavior of the object. In this process, other factors may also be considered to determine the speed of the vehicle 100, such as the lateral position of the vehicle 100 in the road being traveled, the curvature of the road, the proximity of static and dynamic objects, and the like.

In addition to providing instructions to adjust the speed of the self-driving car, the computing device may also provide instructions to modify the steering angle of the vehicle 100 so that the self-driving car follows a given trajectory and/or maintains contact with objects in the vicinity of the self-driving car (eg, , cars in adjacent lanes on the road) safe lateral and longitudinal distances.

The above-mentioned vehicle 100 can be a car, a truck, a motorcycle, a bus, a boat, an airplane, a helicopter, a lawn mower, a recreational vehicle, a playground vehicle, construction equipment, a tram, a golf cart, a train, a cart, etc. The embodiments of the invention are not particularly limited.

Scenario Example 1: Autonomous Driving System

According to FIG. 2 a , the computer system 101 includes a processor 103 coupled to a system bus 105 . The processor 103 may be one or more processors, each of which may include one or more processor cores. A video adapter 107, which can drive a display 109, is coupled to the system bus 105. System bus 105 is coupled to input-output (I/O) bus 113 through bus bridge 111 . I/O interface 115 is coupled to the I/O bus. I/O interface 115 communicates with various I/O devices, such as input device 117 (eg, keyboard, mouse, touch screen, etc.), media tray 121, (eg, compact disc read-only) memory, CD-ROM), multimedia interface, etc.). Transceiver 123 (which can send and/or receive radio communication signals), camera 155 (which can capture still and moving digital video images) and external USB interface 125 . Wherein, optionally, the interface connected to the I/O interface 115 may be a USB interface.

The processor 103 may be any conventional processor, including a reduced instruction set computing (reduced instruction set computing, RISC) processor, a complex instruction set computing (complex instruction set computing, CISC) processor or a combination of the above. Alternatively, the processor may be a special purpose device such as an ASIC. Optionally, the processor 103 may be a neural network processor or a combination of a neural network processor and the above-mentioned conventional processors.

Alternatively, in various embodiments herein, computer system 101 may be located remotely from the autonomous vehicle and may communicate wirelessly with the autonomous vehicle. In other aspects, some of the processes herein are performed on a processor disposed within an autonomous vehicle, others are performed by a remote processor, including taking actions required to perform a single maneuver.

Computer 101 may communicate with software deployment server 149 through network interface 129 . Illustratively, network interface 129 is a hardware network interface, such as a network card. The network 127 may be an external network, such as the Internet, or an internal network, such as an Ethernet network or a virtual private network (VPN). Optionally, the network 127 may also be a wireless network, such as a WiFi network, a cellular network, and the like.

The hard disk drive interface is coupled to the system bus 105 . The hard drive interface is connected to the hard drive. System memory 135 is coupled to system bus 105 . Data running in system memory 135 may include operating system 137 and application programs 143 of computer 101 .

The operating system includes a Shell 139 and a kernel 141 . Shell 139 is an interface between the user and the operating system's kernel. The shell is the outermost layer of the operating system. The shell manages the interaction between the user and the operating system: waiting for user input, interpreting user input to the operating system, and processing various operating system output.

Kernel 141 consists of those parts of the operating system that manage memory, files, peripherals, and system resources. The kernel 141 directly interacts with the hardware, and the operating system kernel usually runs processes, provides inter-process communication, provides CPU time slice management, interrupts, memory management, IO management, and the like.

Application 143 includes programs that control the autonomous driving of the car, for example, programs that manage the interaction of the autonomous car with obstacles on the road, programs that control the route or speed of the autonomous car, and programs that control the interaction of the autonomous car with other autonomous vehicles on the road. . Application 143 also exists on the system of deploying server 149. In one embodiment, computer system 101 may download application 143 from software deployment server 149 when application 147 needs to be executed.

Sensor 153 is associated with computer system 101 . The sensor 153 is used to detect the environment around the computer 101 . For example, the sensor 153 can detect animals, cars, obstacles and pedestrian crossings, etc. Further, the sensor 153 can also detect the environment around the above-mentioned animals, cars, obstacles and pedestrian crossings, such as: the environment around animals, for example, around animals Other animals present, weather conditions, ambient light levels, etc. Alternatively, if the computer 101 is located on a self-driving car, the sensor may be a radar system or the like.

Computer system 112 may also receive information from or transfer information to other computer systems. Alternatively, sensor data collected from the sensor system 104 of the vehicle 100 may be transferred to another computer for processing of the data. As shown in FIG. 3, data from computer system 112 may be transmitted via a network to cloud-side computer 720 for further processing. Networks and intermediate nodes may include a variety of configurations and protocols, including the Internet, the World Wide Web, an intranet, a virtual private network, a wide area network, a local area network, a private network using one or more of the company's proprietary communication protocols, Ethernet, wireless local area networks (wireless local area networks). fidelity, WiFi) and hypertext transfer protocol (HTTP), and various combinations of the foregoing. Such communication may be accomplished by any device capable of transferring data to and from other computers, such as modems and wireless interfaces.

In one example, computer 720 may include a server having multiple computers, such as a load balancing server farm, that exchange information with different nodes of the network for the purpose of receiving, processing, and transmitting data from computer system 112. The server may be configured similarly to computer system 110 , with processor 730 , memory 740 , instructions 750 and data 760 .

Example Scenario 2: Video Surveillance

With the development of smart city and safe city business, monitoring technology has become an indispensable product and technology. In the Safe City solution, the monitoring system is the basic equipment in video criminal investigation, security early warning, and traffic command. The current object tracking technology is based on the monitoring system responsible for managing multiple cameras in the area. The first step of target tracking is to determine the position of the target, and then effectively dispatch the camera to track the target according to the position of the target.

Taking the monitoring and management system 200 as an example, the system architecture to which the target tracking method provided by the present application is applicable is shown in FIG. 2b. The surveillance management system 200 may include a server 201 and a radar system 202 . The radar system 202 can collect point cloud data within the detection range of the radar system, and the server 201 can be used to manage one or more radar systems in a certain area, and can receive the point cloud data returned by the radar system 202, and then process the point cloud data. After the sum and analysis, the position information of the target is calculated and the radar system 202 is reasonably scheduled to track the target. The monitoring management system 200 may also include a display screen 203 for presenting the target tracking results in real time.

In this application, the target object may be a person, an animal, or a vehicle, and the target tracking method provided in this embodiment of the application can be used for tracking targets such as a human body, an animal, or a vehicle.

Specifically, with reference to the descriptions of FIG. 1, FIG. 2a, and FIG. 2b, in the embodiment of the present application, the vehicle in FIG. 1, the processor in the computer system in FIG. 2a, and the server in FIG. 2b can obtain the sensor system The data collected by the radar system (such as a radar system), and the instructions related to target tracking in the storage device, and the data is processed based on the acquired instructions to obtain the target tracking result. Further, the vehicle can perform automatic driving based on the target tracking result, etc. related operations.

In the embodiments of the present application, target tracking refers to analyzing the point cloud data collected by the radar system in combination with the motion model and the error covariance matrix to obtain the positions and motion states of obstacles around the vehicle. When tracking, it is first necessary to cluster the point cloud data collected by the radar system (the purpose is to distinguish the point cloud data of each object to obtain the point cloud data corresponding to each target object) and the centroid estimation (the purpose is to The centroid estimation is performed on the point cloud data of each target object, and the centroid measurement value corresponding to each target object is obtained), and further, the centroid measurement value can be obtained. The centroid measurement value of the target object can be obtained in each of the consecutive frames, and the input of the target tracking is the above-obtained centroid measurement value of each frame. In some scenarios, the centroid estimation of point cloud data of the same target object in different frames may experience position disturbance due to problems such as vehicle turning or occlusion. In the prior art, when tracking the target, the error covariance matrix only considers The measurement error related to the system's point cloud measurement performance cannot accurately quantify the error caused by the centroid estimation of the point cloud data. Therefore, it will reduce the accuracy of target tracking, and even lead to filter divergence and track break batches. Specifically, as shown in Fig. 4, the signal transmitter of the radar system (the letter "T" in Fig. 4) sends the radar signal to the target object (the vehicle in Fig. 4), and the signal receiver (the letter in Fig. 4) sends the radar signal to the target object (the vehicle in Fig. 4). "R") receives the signal reflected by the target object, and can process the signal reflected by the target object to obtain point cloud data, wherein when the target object is at vehicle position 1, the corresponding point cloud data is 401, and the target object When driving to the vehicle position 2, the corresponding point cloud data is 402. As shown in Figure 4, the deviation between the estimated centroid position and the true position of the centroid obtained based on the centroid estimation of the point cloud data 401 is small, and due to the target The object turns while driving, and the estimated position of the centroid obtained based on the centroid estimation of the point cloud data 402 has a large deviation from the actual position of the centroid.

In order to solve the above problem, an embodiment of the present application provides a target tracking method.

Referring to FIG. 5 , FIG. 5 is a schematic flowchart of a target tracking method provided by an embodiment of the present application. As shown in FIG. 5 , the target tracking method provided by the embodiment of the present application includes:

501. Acquire point cloud data of the target object, and perform centroid estimation on the point cloud data.

In this embodiment of the present application, the radar system may scan the target object to obtain point cloud data of the target object. The point cloud data is obtained by scanning the target object with the radar system, and the point cloud data can be recorded in the form of points, and each point can include coordinates and motion states.

Among them, the coordinates can represent the relative position of the point relative to the radar system. For some radar systems, the collected point cloud data is expressed in the polar coordinate system, and the coordinates included in each point can include the point between the point and the radar system. distance, and the bearing of the point;

Among them, the motion state can represent the motion speed of each point. For some radar systems, the collected point cloud data is expressed in the polar coordinate system, and the motion state included in each point can include the radial speed of the point relative to the vehicle. .

The specific expression manner of the point cloud data is not limited in this embodiment of the present application.

In this embodiment of the present application, after acquiring the point cloud data of the target object, centroid estimation may be performed on the point cloud data. Specifically, the average value of the coordinate positions corresponding to each point included in the point cloud data can be calculated based on the average value calculation method, and the average value can be used as the position of the centroid; The average value is taken as the movement rate of the centroid. The embodiment of the present application does not limit the specific operation manner of centroid estimation.

In the embodiment of the present application, the point cloud data of the target object collected by the radar system can be acquired, and the centroid estimation of the point cloud data can be performed to obtain the state measurement value. For example, the state measurement can be Z _k = [R _k , θ _k , v _rk ], where R _k can represent the distance between the radar system and the centroid, θ _k can represent the orientation of the centroid, and v _rk can represent the position of the centroid radial velocity.

It should be noted that, in an optional implementation, the point cloud data of the target object can be obtained by a device on the terminal side (for example, the processor of the vehicle), and the centroid of the point cloud data can be estimated; The device sends the point cloud data of the target object to the server on the cloud side, and the server can obtain the point cloud data of the target object and perform centroid estimation on the point cloud data.

502. Obtain a target covariance matrix, where the target covariance matrix represents a centroid position error introduced during centroid estimation.

In this embodiment of the present application, a target covariance matrix may be obtained; wherein, the target covariance matrix represents a centroid position error introduced during centroid estimation.

In this embodiment of the present application, the result of centroid estimation can be used for track initiation; wherein, track initiation refers to the process of confirming the target track before the system enters stable target tracking. The goal is to start the correct track with the highest possible probability and suppress false tracks. If the track starts incorrectly, the correct tracking of the target and information fusion cannot be achieved. If there are too many false tracks at the beginning, it will bring a large computational burden to the subsequent data comprehensive processing and affect the data comprehensive processing. Efficiency and correctness. The track initiation algorithm can include, but is not limited to, two categories: one is a sequential processing technology represented by heuristic algorithms and logical rules, and the other is a batch processing technology represented by Hough transform.

In this embodiment of the present application, a track corresponding to a target object can be determined through track initiation, and the track can include the result of centroid estimation of each frame, and the result of centroid estimation can be a centroid measurement value, which can then be based on a motion model Predict the centroid measurement value of each frame to obtain the state predicted value of the next frame, and then compare the state predicted value of the next frame with the corresponding centroid measurement value of the next frame. If the error is within the preset range, it is considered that successful association, track and record the current number of associated data, wherein the data associated with the number 0 is started, the data associated with each successful once, may increase the number of times a data association, in order to record data associated with track number N _a.

In some embodiments, different target covariance matrices can be obtained according to the difference of the _{current association times N a, which are described below:}

The present embodiments of the application, if the association number of the current record is equal to or less than N _a predetermined value N, it is considered that the current target tracking process is still at an initial stage, the filtering using respective covariance matrix is not close to the true value ( i.e. the correct value), the target track can not be performed based on the target current time history tracking result, i.e., if the association number of the current record is equal to or less than N _a predetermined value N, the target tracking has not converged, then you can use empirically The target covariance matrix set by the value is used for target tracking, wherein the target covariance matrix can represent the centroid position error introduced during centroid estimation.

In one embodiment, if the association number of the current record N _a ≤N, wherein N is a preset value, the error may be determined to represent centroid introduced when the initial value of the covariance matrix of the covariance matrix of the target centroid estimation. The target covariance matrix may be a preset covariance matrix, for example, may be set based on an empirical value.

Here is an example, if the centroid position error introduced in the estimated centroid estimation is 5 meters, and it is considered that the error in the x-direction and the error in the y-direction in the Cartesian coordinate system are not related (that is, the centroid position error in the x-direction is will not affect the centroid position error in the y direction, and the centroid position error in the y direction will not affect the centroid position error in the x direction), you can set the preset target covariance matrix as

Among them, the target covariance matrix is represented in the Cartesian coordinate system. If the centroid position error introduced in the estimated centroid estimation is 5 meters, and it is considered that the error in the x-direction and the error in the y-direction in the Cartesian coordinate system are related (that is, the centroid position error in the x-direction will affect the y-direction error). centroid position error, the centroid position error in the y direction will affect the centroid position error in the x direction), you can set the preset target covariance matrix as

Among them, the target covariance matrix is represented in the Cartesian coordinate system, and the value of x can be set based on the influence of the centroid position error in the x direction on the centroid position error in the y direction, and the value of y can be based on the centroid position in the y direction. The influence of the error on the centroid position error in the x-direction is set, which is not limited here.

In some implementations, if the current record number N _a correlation greater than a predetermined value N, it is considered that the current target tracking process in a stable stage, which filter the respective covariance matrix is used near or true value (i.e., the correct value ), in this case, the target tracking at the current moment can be performed based on the historical target tracking results. In this case, the target covariance matrix can be obtained based on the target tracking predicted value and the state measurement value; specifically, it can be Obtain the target tracking results of the target object at the last moment; and based on the motion model, perform state prediction on the target tracking results at the previous moment to obtain the target tracking prediction value at the current moment; obtain the state measurement value at the current moment, the state measurement value It is obtained by estimating the centroid of the point cloud data; based on the target tracking predicted value and the state measurement value, the target covariance matrix is obtained.

Next, we describe how to obtain the target covariance matrix based on target tracking predictions and state measurements.

In the embodiment of the present application, if the number of association times N _{a is} greater than the preset value N, the target covariance matrix may be determined according to the error between the target tracking predicted value and the state measurement value at the current moment; wherein, the calculation method of the error may be: Perform a matrix subtraction operation on the target tracking prediction value and the state measurement value. Specifically, the target tracking prediction value may include the position prediction value of the centroid, and the state measurement value may include the position measurement value of the centroid. In the Cartesian coordinate system, the position The predicted value includes the position prediction value in the x direction and the position prediction value in the y direction; the position measurement value includes the position measurement value in the x direction and the position measurement value in the y direction, and then the position prediction value in the x direction and the position measurement in the x direction can be calculated. The difference between the values, calculate the difference between the position prediction value in the y direction and the position measurement value in the y direction, you can calculate the difference between the position prediction value in the x direction and the position measurement value in the x direction or the difference The preset multiple of the value is used as a diagonal element of the target covariance matrix. A diagonal element.

In more detail, in an embodiment, the target covariance matrix can be obtained by the following formula based on the target tracking predicted value and the state measurement value:

Among them, x _k-1|k , y _k+1|k are state measurement values,

is the target tracking predicted value, and R ₁ is the target covariance matrix.

It should be noted that the above calculation process of the target covariance matrix is only an illustration, as long as the centroid position error introduced during centroid estimation can be represented, the present application does not limit the calculation method of the target covariance matrix.

503. Based on the target covariance matrix, perform target tracking on the target object to obtain a target tracking result.

Due to the turning or occlusion of vehicles or other types of target objects, the centroid estimation of point cloud data of the same target object in different frames may experience positional disturbance. In the prior art, during target tracking, the error covariance matrix only considers The measurement error related to the point cloud measurement performance of the radar system cannot accurately quantify the error caused by the centroid estimation of the point cloud data. Therefore, the accuracy of target tracking will be reduced. In the embodiment of the present application, the target covariance matrix The error caused by the centroid estimation of the point cloud data can be relatively accurately quantified. Therefore, if the target object is tracked based on the target covariance matrix, the accuracy of the target tracking can be improved.

Next, it will be described in detail how to track the target object based on the target covariance matrix.

When performing target tracking, it is necessary to carry out data according to the target tracking result at the previous moment, the centroid estimation result (state measurement value) at the current moment, and the covariance matrix (which may include the measurement equation covariance matrix and the state equation covariance matrix) and other data. The target tracking at the current moment, in which the target covariance matrix obtained above can be fused into the measurement equation covariance matrix, that is, the target covariance matrix and the measurement equation covariance matrix are synthesized to obtain the synthesized target covariance matrix, and The synthesized target covariance matrix is used as the input data for target tracking at the current moment.

In the embodiment of the present application, the target tracking is performed on the target object based on the target covariance matrix. It can be understood that the target covariance matrix is used as part of the input data to perform target tracking on the target object; , use the target covariance matrix as part of the input data.

Next, we will introduce how to combine the target covariance matrix and the measurement equation covariance matrix to obtain the combined target covariance matrix.

In the embodiment of the present application, after the target covariance matrix is obtained, the measurement equation covariance matrix may be obtained, wherein the measurement equation covariance matrix represents the measurement deviation of the radar system; the measurement equation covariance matrix and the target covariance matrix are synthesized , to get the synthesized target covariance matrix.

In an embodiment, usually due to the limitation of the radar system, the point cloud data collected by the radar system is usually the data expressed in the polar coordinate system. Therefore, the measurement equation covariance matrix related to the performance of the radar system is usually in the polar coordinate system. The matrix represented in the coordinate system, and the target covariance matrix is the matrix represented in the Cartesian coordinate system, in this case, the measurement equation covariance matrix can be converted to the matrix represented in the Cartesian coordinate system, and the The converted measurement equation covariance matrix and the target covariance matrix are matrix-added to obtain a combined target covariance matrix.

Next we describe how to convert the measurement equation covariance matrix into a matrix represented in Cartesian coordinates.

In the embodiment of the present application, a covariance transformation synthesis matrix may be constructed first, and the covariance transformation synthesis matrix may be used to convert the covariance matrix from a polar coordinate system to an expression in a Cartesian coordinate system; in one implementation, The covariance transformation synthesis matrix is related to the state measurement value at the current moment. Specifically, the state measurement value at the current moment can be Z _k = [R _k , θ _k , v _rk ], where R _k can represent the radar system and the center of mass. Distance, θ _k can represent the orientation of the centroid, and v _rk can represent the radial velocity of the centroid; based on the distance between the radar system and the centroid in the state measurements, and the orientation of the centroid, the covariance transformation can be constructed in the synthesis matrix part of the elements of the covariance transformation synthesis matrix, and use 0 and 1 to fill another part of the elements in the covariance transformation synthesis matrix; specifically, cosθ _k , -R _k cosθ _k can be used as the elements of the first row of the covariance transformation synthesis matrix, sinθ _k and R _k cos θ _k as the elements of the second row of the covariance transformation synthesis matrix.

In more detail, the covariance transformation synthesis matrix can be:

It should be noted that the above covariance transformation synthesis matrix is only an illustration. In practical applications, the dimension of the covariance transformation synthesis matrix can be flexibly adjusted, which is not limited here.

After the covariance transformation synthesis matrix is obtained, the measurement equation covariance matrix can be transformed into a matrix represented in the Cartesian coordinate system based on the covariance transformation synthesis matrix, and the transformed measurement equation covariance matrix and the target covariance matrix are processed. Matrix addition to obtain the synthesized target covariance matrix. For example, the target covariance matrix and the measurement equation covariance matrix can be synthesized by the following formula to obtain the synthesized target covariance matrix:

R _all =R ₀ +A _k R _m A _k ′;

Among them, matrix addition can be understood as matrices with the same size, and the corresponding elements are added to obtain a new matrix. For example, matrix A is

Matrix B is

Then the matrix C obtained after matrix A and matrix B are added is:

where R _all is the synthesized target covariance matrix, R ₀ is the target covariance matrix, A _k R _m A _k ′ is the transformed measurement equation covariance matrix, R _k and θ _k are the state measurement values, and A _k is Covariance transformation synthesis matrix, A _k ' is the transpose of the covariance transformation synthesis matrix.

It should be noted that the above method of synthesizing the target covariance matrix and the measurement equation covariance matrix is only an illustration. In practical applications, it is only necessary to ensure that the synthesized target covariance matrix includes the centroid position introduced in the centroid estimation. The information related to the error is sufficient, and the present application does not limit the specific matrix synthesis method.

In the embodiment of the present application, the synthesized target covariance matrix includes information related to the centroid position error introduced during centroid estimation, and the synthesized target covariance matrix relatively accurately quantifies the error caused by the centroid estimation of the point cloud data. Therefore, if Tracking the target object based on the synthesized target covariance matrix can improve the accuracy of target tracking.

In the embodiment of the present application, the target object can be tracked based on the target covariance matrix; in more detail, the target object can be tracked based on the synthesized target covariance matrix; the following describes how to track the target object based on the synthesized target Covariance matrix, to track the target object: Specifically, in this embodiment of the present application, a state equation covariance matrix P may be obtained, wherein the state equation covariance matrix P may be a matrix composed of covariances between states, The diagonal elements are the variances between the states, and the remaining elements are the covariances of the corresponding elements. Therefore, P can be a multidimensional square matrix with the same dimensions as the number of states, and P can be a symmetric square matrix. For example, state X contains two quantities, position p and velocity v. Since the covariance between the same variables is its variance, the elements on the diagonal are the variances of p and v, respectively, and the other two elements are the covariance between the two elements. The variance, equation of state covariance matrix is symmetric due to the covariance part order. When used, the state equation covariance matrix P is an iteratively updated quantity. After each round of prediction and update, P will be updated with a new value. Therefore, the initialization can be based on the estimation agreement, and the accuracy of initialization does not need to be too strict. Over several iterations, the true value will be tightened more and more.

In this embodiment of the present application, the state measurement value obtained by performing centroid estimation at the last moment may be obtained; the state measurement value may be X(k|k)=[x _k|k , y _k|k , v _xk|k , v _{yk |k} ], where x _k/k , y _k/k represent the position measurement value of the centroid at the current moment, where x _k/k represent the position measurement value of the centroid at the current moment in the x direction in the Cartesian coordinate system, y _{k/ k} represents the position measurement value of the center of mass at the current moment in the y direction in the Cartesian coordinate system, v _xk/k , v _yk/k represent the velocity measurement value of the center of mass at the current moment, where v _xk/k represents the current moment of the center of mass in Cartesian coordinates The velocity measurement in the x direction in the system, where v _yk/k represents the position measurement of the centroid at the current moment.

In the embodiment of the present application, the state at the current moment may be predicted by using the motion model and the state measurement value obtained by performing centroid estimation at the previous moment to obtain the state prediction value at the current moment. Exemplarily, F(k) represents the motion model, V(k) represents the state noise, and the state prediction value can be calculated by the following formula:

X(k+1|k)=F(k)X(k|k)+V(k);

Among them, X(k+1/k) represents the state prediction value at the current moment.

Among them, the motion model is an important part of the design of the tracking filtering algorithm. Selecting a reasonable motion model helps to accurately predict the future state or trajectory of the tracked target, which is an important condition for accurate tracking control. The motion model is mainly represented by a state space model, and the motion model can express the motion law of the target. The embodiment of the present application does not limit the specific motion model type.

In this embodiment of the present application, the prediction of the covariance matrix of the state equation at the current moment may be performed based on the state equation covariance matrix and the motion equation at the previous moment: P(k+1|k)=F(k)P(k| k)F'(k)+Q(k); where, P(k+1/k) can represent the covariance matrix of the state equation at the current moment, F(k) represents the motion model, and F'(k) can represent the matrix Transpose matrix of F(k).

Afterwards, the target tracking prediction value at the current moment can be obtained according to the measurement matrix and the target tracking result of the previous moment. Exemplarily, H(k+1) represents the measurement matrix, and W(k+1) represents the The measurement error, the target tracking prediction value at the current moment can be:

Z(k+1|k)=H(k+1)X(k+1|k)+W(k+1), where Z(k+1/k) can represent the target tracking prediction value at the current moment .

After that, the innovation innovation can be calculated based on the target tracking prediction value at the current moment and the state measurement value at the current moment; exemplarily, the innovation can be v(k+1)=Z(k+1)-Z(k+ 1|k), where v(k+1) can represent the innovation, Z(k+1/k) can represent the target tracking prediction value at the current moment, and Z(k+1) can represent the state measurement value at the current moment . It should be noted that the innovation may represent the update of the matrix related to the state information.

And based on the measurement matrix, the covariance matrix of the state equation at the current moment, and the synthesized target covariance matrix obtained above, the innovation covariance matrix is calculated. Exemplarily, the innovation covariance matrix can be calculated by the following formula:

S(k+1)=H(k+1)P(k+1|k)H'(k+1)+R _all ; where S(k+1) can represent the innovation covariance matrix, H( k+1) represents the measurement matrix, P(k+1/k) can represent the covariance matrix of the state equation at the current moment, and R _all can represent the synthesized target covariance matrix.

In this embodiment of the present application, an innovation covariance matrix may be generated based on the synthesized target covariance matrix and other covariance matrices, and then the target tracking result at the moment is calculated based on the innovation covariance matrix, where the target tracking result may be the current moment The centroid position of the target object.

In the embodiment of the present application, the target tracking result at the current moment may be calculated based on the state equation covariance matrix, the measurement matrix, the innovation covariance matrix, the innovation value, and the state predicted value at the current moment, wherein the target tracking result It can be the centroid position of the target object at the current moment.

Specifically, target tracking can be performed based on the following formula:

K(k+1)=P(k+1|k)H′(k+1)S ^-1 (k+1)

X(k+1|k+1)=X(k+1|k)+K(k+1)·v(k+1);

Among them, P(k+1/k) can represent the state equation covariance matrix at the current moment, H(k+1) can represent the measurement matrix, S(k+1) can represent the innovation covariance matrix, v(k+ 1) It can represent innovation, X(k+1/k) can represent the state prediction value at the current moment, and X(k+1/k+1) can represent the target tracking result.

It should be noted that the above-mentioned specific implementation of the target tracking for the target object based on the target covariance matrix is only an illustration. In practical applications, the target tracking process based on different filtering algorithms may be different. The adjustment of compatibility is not limited in this application.

It should be noted that, in some implementations, the point cloud data of the target object may be acquired by the terminal side (for example, a vehicle or a monitoring device), and the point cloud data of the target object may be sent to the server, and the server will perform the processing on the point cloud data. Centroid estimation; obtaining a target covariance matrix, the server can perform target tracking on the target object based on the target covariance matrix to obtain a target tracking result, and send the target tracking result to the terminal side.

It should be noted that, in some implementations, the point cloud data of the target object can be obtained by the terminal side (such as a vehicle or a monitoring device), and the centroid estimation is performed on the point cloud data; to obtain the target covariance matrix, the server can be based on For the target covariance matrix, target tracking is performed on the target object to obtain a target tracking result.

Next, an application example of the target tracking method based on the constant turning rate and velocity (CTRV) model and the cubature kalman filter (CKF) algorithm is described.

In the embodiment of the present application, the CTRV motion model and the CKF filtering algorithm are used to implement target tracking based on point cloud data, wherein the specific implementation process can be shown in Figure 6, and the method includes:

601. Parameter initialization.

In the embodiment of the present application, when the target tracking is performed, the data related to the target tracking needs to be prepared first, wherein the data related to the target tracking may include a measurement equation covariance matrix related to the performance of the radar system, which may represent the centroid The target covariance matrix of the centroid position error introduced during estimation; for details, reference may be made to the description related to step 502 in the embodiment corresponding to FIG. 5 , which will not be repeated here.

602. Set the CKF volume point set ξ according to the dimension of the state equation.

In the embodiment of the present application, the CKF volume point set is related to the number of dimensions of the state. For example, when n=3, the corresponding volume point set is:

For example, when n=2, the corresponding volume point set is:

603. Perform centroid estimation.

604. State prediction.

In this embodiment of the present application, the target tracking result at the previous moment and the CTRV motion model can be obtained, and the track state of the target object is predicted based on the target tracking result at the previous moment and the CTRV motion model.

The state prediction may include: generating a plurality of extended volume points according to the initialization parameters in step 601, the CKF volume point set obtained in step 602, and the target tracking result at the previous moment, and for each of the plurality of extended volume points Each extended volume point is predicted according to the CTRV model, and the predicted value of each volume point at the current moment is generated; and the target tracking predicted value and the covariance matrix of the state equation at the current moment are predicted according to the predicted value of each volume point at the current moment. Specifically, step 604 may include steps 6041-6043 described below.

6041. Generate an extended volume point.

In this embodiment of the present application, multiple extended volume points may be generated according to the initialization parameters in step 601 and the CKF volume point set obtained in step 602. Specifically, multiple extended volume points may be generated based on the following formula:

Among them, P(k/k) can represent the covariance matrix of the state equation at the previous moment,

represents the decomposition of the lower triangular matrix, ξ _i represents the volume point, and X(k/k) can represent the target tracking result at the previous moment.

6042. Predict each expanded volume point according to the CTRV model to obtain a predicted value corresponding to each volume point.

6043. Predict the state value at the current moment and the covariance matrix of the state equation.

In the embodiment of the present application, the state prediction value X(k+1/k) at the current moment may be obtained based on the average value of the extended volume points, and specifically, the state prediction value X(k+1/k) at the current moment may be calculated based on the following formula /k):

in,

Represents an extended volume point.

After that, the state equation covariance matrix P(k+1|k) can be predicted based on the predicted values corresponding to the multiple volume points and the state predicted value at the current moment.

Afterwards, data association can be performed based on the state prediction value at the current moment and the centroid measurement value at the current moment. In the embodiment of the present application, the result of the centroid estimation can be performed with track start, and the track start can be used to determine the corresponding target object. Track, the track can include the result of centroid estimation of each frame, and the result of centroid estimation can be the centroid measurement value, and then the centroid measurement value of each frame can be predicted based on the motion model, and the state prediction value of the next frame can be obtained , and then compare the state predicted value of the next frame with the corresponding centroid measurement value of the next frame. If the error is within the preset range, the association is considered successful, and the current track data association times are recorded, where the data association times are is 0, each data association is successful, the data may be associated with an increase in frequency and, in order to record data associated with track number N _a.

605. The measurement covariance matrix is estimated.

In the embodiment of the present application, it is necessary to obtain the measurement covariance matrix. Specifically, the synthesized target covariance matrix may be obtained based on the step of step 502 in the embodiment corresponding to FIG. 5 , wherein the estimation result of the measurement covariance matrix is That is, the target covariance matrix. For details, refer to the description in the above embodiment, and details are not repeated here.

606. Predict the state measurement value at the next moment.

In this embodiment of the present application, the state measurement value at the next moment may be predicted based on the state measurement value at the current moment and the volume point. Specifically, the prediction of the state measurement value at the next moment may include

steps

6061 and 6062 .

6061. Generate an extended volume point.

In this embodiment of the present application, the state measurement value X(k+1|k) at the current moment, the state equation covariance matrix P(k+1|k), and the CKF volume point set ξ can be obtained to generate multiple extended volume points , specifically, multiple extended volume points can be generated based on the following formula:

After that, the corresponding measurement prediction value can be generated according to the measurement equation for each extended volume point

And form the extended matrix Z _cuba (k+1|k).

6062. Predict the state measurement value, the innovation covariance matrix, and the state measurement covariance matrix at the next moment.

In the embodiment of the present application, the state measurement value, the innovation covariance matrix, and the state measurement covariance matrix at the next moment can be predicted by the following formula:

in,

Represents the measurement prediction value corresponding to the measurement equation, Z _cuba (k+1|k) can represent the expansion matrix, S(k+1) can represent the innovation covariance matrix, R _all can represent the synthesized target covariance matrix, Z (k+1) can represent the state measurement value at the current moment, and P _XZ (k+1) represents the state measurement covariance matrix.

607. Status update.

K(k+1)=P(k+1|k)H′(k+1)S ^-1 (k+1)

X(k+1|k+1)=X(k+1|k)+K(k+1)·v(k+1);

Wherein, v(k+1)=Z(k)-Z(k+1/k), and v(k+1) can represent innovation.

At this point, the point cloud target tracking based on the CTRV model and the CKF filtering algorithm can be completed, that is, the centroid position X(k+1/k+1) of the target object at the current moment is obtained.

In some scenes, due to the turning or occlusion of vehicles or other types of target objects (where the occlusion can be blocked by other obstacles in the process of going straight), the centroid estimation of the point cloud data of the same target object in different frames may appear. Disturbance, based on the application of the synthesized target covariance matrix in the CTRV combined with the CKF algorithm for target tracking, can effectively improve the target tracking accuracy in the scene of the vehicle going straight and turning; the CKF algorithm can achieve nonlinear third-order equivalent, and the filtering accuracy is better. high.

Next, a target tracking method based on the interacting multiple model-constant velocity-coordinate turn (IMM-CV-CT) model and the extended Kalman filter (EKF) algorithm is described. application example.

The specific implementation process can be shown in Figure 7, and the method includes:

701. Centroid estimation.

702. Input the centroid estimation result and initialization parameters.

703. Input interaction.

In this embodiment of the present application, the state value X ₀₁ (k|k) _{corresponding to the CV model, the state covariance matrix P 01} (k|k), and the state value corresponding to the CT model can be output according to the current transition probability and the current state value X ₀₁ (k|k) and state covariance matrix P ₀₂ (k|k).

X ₀₁ (k|k)=X ₁ (k|k)μ _1|1 (k)+X ₂ (k|k)μ _2|1 (k)

X ₀₂ (k|k)=X ₁ (k|k)μ _1|2 (k)+X ₂ (k|k)μ _2|2 (k)

P ₀₂ (k|k)=μ _1|2 (k)(P ₁ (k|k)+(X ₁ (k|k)−X ₀₂ (k|k))·(X ₁ (k|k) -X ₀₂ (k|k))')+μ _2|2 (k)(P ₂ (k|k)+(X ₂ (k|k)-X ₀₂ (k|k))·(X ₂ ( k|k)-X ₀₂ (k|k))')

Then, the track state of the target object can be predicted according to the current state X ₀₁ (k|k), X ₀₂ (k|k), CV and CT motion model. , where P ₀₁ (k/k) and P ₀₂ (k/k) can be used in the subsequent state update and covariance update.

704. Predict the track state of the target object based on CV:

X ₁ (k+1|k)=F _CV X ₀₁ (k|k),

Among them, X ₁ (k+1/k) is the prediction result obtained by predicting the track state of the target object based on the CV model.

705. Predict the track state of the target object based on the CT model:

X ₂ (k+1|k)=F _CT X ₀₂ (k|k),

Among them, X ₂ (k+1/k) is the prediction result obtained by predicting the track state of the target object based on CT.

706. Predicting state synthesis.

In the embodiment of the present application, due to the prediction result obtained by predicting the track state of the target object based on the CV model and the prediction result obtained by predicting the track state of the target object based on the CT model, it is possible to describe the different levels of the track state. Therefore, it is necessary to synthesize the prediction results, and perform subsequent data association based on the synthesized prediction results. Exemplarily, the prediction result obtained by predicting the track state of the target object based on the CV model and the prediction result obtained by predicting the track state of the target object based on the CT model can be synthesized based on the following formula:

X(k+1|k)=X ₁ (k+1|k)μ ₁ (k)+X ₂ (k+1|k)μ ₂ (k);

Among them, X(k+1/k) is the prediction result after synthesis, μ ₁ (k) is the weight value corresponding to the prediction result obtained by predicting the track state of the target object based on the CV model, and μ ₂ (k) is The weight value corresponding to the prediction result obtained by predicting the track state of the target object based on the CT model.

707. Data association.

For the specific description of step 707, reference may be made to FIG. 5 and the description related to data association in the corresponding embodiment, and details are not repeated here.

708. Determine whether the convergence condition is satisfied, and if not, adopt the preset initial covariance matrix R _m and R _{0 to} synthesize the measurement equation covariance matrix R _all for filtering.

For step 708, reference may be made to the description related to the synthetic target covariance matrix in step 502 in the foregoing embodiment, and details are not repeated here.

709. Determine whether the convergence condition is met, and if so, determine the target covariance matrix according to the error between the target tracking predicted value and the state measurement value at the current moment.

Step 709 can refer to the description related to determining the target covariance matrix according to the error between the target tracking predicted value and the state measurement value at the current moment in step 502 in the above embodiment, and details are not repeated here.

710. Synthesize a covariance matrix.

For step 710, reference may be made to the description related to the synthetic target covariance matrix in step 503 in the foregoing embodiment, and details are not repeated here.

711. Perform measurement prediction and filter output based on the CV model, and output the filtered state X ₁ (k|k).

712. Perform measurement prediction and filter output based on the CT model, and output the filtered state X ₂ (k|k).

713. Obtain the state update and covariance update of the two models respectively according to the EKF filtering algorithm.

P ₀₁ (k+1|k)=F _CV P ₀₁ (k|k)F′ _CV +Q _noise

K ₁ (k+1)=P ₀₁ (k+1|k)·H'S ₁ ^-1 (k+1|k)

S ₁ (k+1|k)=H·P ₀₁ (k+1|k)·H'+R _all

v ₁ (k+1)=Z(k)-Z ₁ (k+1|k)

X ₁ (k+1|k+1)=X ₁ (k+1|k)+K ₁ (k+1)v ₁ (k+1)

P ₁ (k+1|k+1)=P ₀₁ (k+1|k)−K ₁ (k+1)S ₁ (k+1|k)K ₁ ′(k+1);

P ₀₂ (k+1|k)=F _CT P ₀₂ (k|k)F′ _CT +Q _noise

K ₂ (k+1)=P ₀₂ (k+1|k)·H'S ₂ ^-1 (k+1|k)

S ₂ (k+1|k)=H·P ₀₂ (k+1|k)·H'+R _all

v ₂ (k+1)=Z(k)-Z ₂ (k+1|k)

X ₂ (k+1|k+1)=X ₂ (k+1|k)+K ₂ (k+1)v ₂ (k+1)

P ₂ (k+1|k+1)=P ₀₂ (k+1|k)−K ₂ (k+1)S ₂ (k+1|k)K ₂ ′(k+1)

Wherein, X ₂ (k+1/k+1) may represent the state update result, and P ₂ (k+1/k+1) may represent the covariance update result.

714. Interactive data output.

Calculate the model probability μ by the innovation v ₁ (k+1), v ₂ (k+1) of the _{corresponding model and the innovation covariance matrix S 1} (k+1|k), S ₂ (k+1|k) ₁ (k+1) and μ ₂ (k+1), and perform final state synthesis to obtain a state update X(k+1|k+1), where the above final state is the target tracking result obtained by target tracking.

X(k+1|k+1)=X ₁ (k+1|k+1)·μ ₁ (k+1)+X ₂ (k+1|k+1)·μ ₂ (k+1) ;

So far, point cloud target tracking based on IMM-CV-CT motion model and EKF filtering algorithm is completed.

Referring to FIG. 8, FIG. 8 is a schematic structural diagram of a target tracking apparatus provided by an embodiment of the present application. As shown in FIG. 8, the apparatus 800 includes:

an acquisition module 801, configured to acquire point cloud data of a target object, and perform centroid estimation on the point cloud data; acquire a target covariance matrix; wherein, the target covariance matrix represents a centroid position error introduced during centroid estimation; and,

The target tracking module 802 is configured to perform target tracking on the target object based on the target covariance matrix to obtain the target tracking result.

In an optional design, the target tracking result is a centroid state value, and the acquiring module is specifically used for:

Obtain the target tracking result of the target object at the previous moment;

In an optional design, the acquisition module is configured to use a preset covariance matrix as the target covariance matrix when the number of data associations is less than or equal to a preset value; wherein the data The number of associations represents the number of times that the difference between the centroid measurement value and the state predicted value is less than the threshold, the centroid measurement value is obtained by performing centroid estimation on the point cloud data of the target object at the current moment, and the state predicted value is the previous moment. The centroid measurement value of the last moment is obtained by performing centroid estimation on the point cloud data of the target object at the last moment.

In an optional design, the obtaining module is configured to obtain the target covariance based on the target tracking predicted value and the state measurement value when the data association times are greater than the preset value Matrix; wherein, the target tracking prediction value is obtained by performing state prediction on the target tracking result at the previous moment.

In an optional design, the acquisition module is specifically used for:

Among them, x _k+1|k , y _k+1|k are state measurement values,

In an optional design, the fetch module is also used to:

Obtain the measurement equation covariance matrix, where the point cloud data is collected based on the radar system, and the measurement equation covariance matrix represents the measurement deviation of the radar system;

The device also includes:

The matrix synthesis module is used for synthesizing the measurement equation covariance matrix and the target covariance matrix to obtain the synthesized target covariance matrix;

Correspondingly, the target tracking module is specifically used for: tracking the target object based on the synthesized target covariance matrix.

In an optional design, the covariance matrix of the measurement equation is a matrix represented in a polar coordinate system, and the target covariance matrix is a matrix represented in a Cartesian coordinate system; the matrix synthesis module is specifically used for:

Matrix addition is performed on the transformed measurement equation covariance matrix and the target covariance matrix to obtain the synthesized target covariance matrix.

In an optional design, the matrix synthesis module, specifically:

Obtain the state measurement value at the current moment, and the state measurement value is obtained by estimating the centroid of the point cloud data;

Based on the state measurement value, obtain the covariance transformation synthesis matrix;

Transform the measurement equation covariance matrix into a matrix represented in the Cartesian coordinate system based on the covariance transformation synthesis matrix.

In an optional design, the matrix synthesis module, specifically:

R _all =R ₀ +A _k R _m A _k ′;

Among them, R ₀ is the target covariance matrix, A _k R _m A _k ′ is the transformed measurement equation covariance matrix, R _k and θ _k are the state measurement values, A _k is the covariance transformation synthesis matrix, and A _k ′ is Transpose of the covariance transform composite matrix.

In some embodiments, the disclosed methods may be implemented as computer program instructions encoded in a machine-readable format on a computer-readable storage medium or on other non-transitory media or articles of manufacture. 9 schematically illustrates a conceptual partial view of an example computer program product including a computer program for executing a computer process on a computing device, arranged in accordance with at least some embodiments presented herein. In one embodiment, example computer program product 900 is provided using signal bearing medium 901 . Signal bearing medium 901 may include one or more program instructions 902 that, when executed by one or more processors, may provide the functions, or portions thereof, described above with respect to FIGS. 5-8 . Thus, for example, with reference to the embodiment shown in FIG. 3 , one or more features of blocks 302 - 309 may be undertaken by one or more instructions associated with signal bearing medium 901 . Additionally, program instructions 902 in FIG. 9 also describe example instructions.

In some examples, the signal bearing medium 901 may include a computer readable medium 903 such as, but not limited to, a hard drive, a compact disc (CD), a digital video disc (DVD), a digital tape, a memory, a read only memory (Read) -Only Memory, ROM) or random access memory (Random Access Memory, RAM) and so on. In some implementations, the signal bearing medium 901 may include a computer recordable medium 904 such as, but not limited to, memory, read/write (R/W) CDs, R/W DVDs, and the like. In some embodiments, signal bearing medium 901 may include communication medium 905, such as, but not limited to, digital and/or analog communication media (eg, fiber optic cables, waveguides, wired communication links, wireless communication links, etc.). Thus, for example, the signal bearing medium 901 may be conveyed by a wireless form of communication medium 905 (eg, a wireless communication medium conforming to the IEEE 802.11 standard or other transmission protocol). The one or more program instructions 902 may be, for example, computer-executable instructions or logic-implemented instructions. In some examples, a computing device of a computing device may be configured to respond to program instructions 902 communicated to the computing device through one or more of computer readable media 903 , computer recordable media 904 , and/or communication media 905 . , which provides various operations, functions, or actions. It should be understood that the arrangements described herein are for illustrative purposes only. Thus, those skilled in the art will understand that other arrangements and other elements (eg, machines, interfaces, functions, sequences, and groups of functions, etc.) can be used instead and that some elements may be omitted altogether depending on the desired results . Additionally, many of the described elements are functional entities that may be implemented as discrete or distributed components, or in conjunction with other components in any suitable combination and position.

FIG. 10 is a schematic structural diagram of a target tracking device provided by an embodiment of the present disclosure, which is used for executing the target tracking method provided in the above-mentioned embodiment. Referring to Figure 10, the device 1000 includes:

The transceiver 110, the memory 120 including one or more storage media, the input unit 130, the display unit 140, the sensor 150, the audio circuit 160, the processor 170 including one or more processing cores, etc. Those skilled in the art can understand that the structure of the device 1000 shown in FIG. 10 does not constitute a limitation on the device 1000, and may include more or less components than those shown, or combine some components, or arrange different components. in:

The transceiver 110 can be used to receive and transmit signals in the process of transceiving information. Through the transceiver 110, the device 1000 can communicate with other devices located in the vehicle, such as various sensors. The communication method includes but is not limited to a Bluetooth wireless communication method, a wireless fidelity (Wireless Fidelity, WiFi) wireless communication method, and the like.

The memory 120 may be used to store software programs and modules, and the processor 170 executes various functional applications and data processing by executing at least one instruction, at least one section of program, code set or instruction set stored in the memory 120 . The memory 120 mainly includes a stored program area and a stored data area, wherein the stored program area can store an operating system, at least one instruction, at least a piece of program, a code set or an instruction set, etc.; data (such as audio data), etc.

The input unit 130 may be used to receive input numerical or character information, and generate signal input related to user settings and function control. Specifically, the input unit 130 may include a touch-sensitive surface as well as other input devices. A touch-sensitive surface, also known as a touch display or trackpad, collects the user's touches on or near it and drives the corresponding connection device according to a preset program. Alternatively, the touch-sensitive surface may include two parts, a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch orientation, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into contact coordinates, and then sends it to the touch controller. To the processor 170, and can receive the command sent by the processor 170 and execute it. Additionally, touch-sensitive surfaces can be implemented using resistive, capacitive, infrared, and surface acoustic wave types. In addition to the touch-sensitive surface, the input unit 130 may also include other input devices. Specifically, other input devices may include, but are not limited to, one or more of physical keyboards, function keys (such as volume control keys, switch keys, etc.).

The display unit 140 may be used to display information input by or provided to the user and various graphical user interfaces of the device 1000, which may be composed of graphics, text, icons, videos, and any combination thereof. The display unit 140 may include a display panel, and optionally, the display panel may be configured in the form of an LCD (Liquid Crystal Display, liquid crystal display), an OLED (Organic Light-Emitting Diode, organic light-emitting diode) and the like. Further, the touch-sensitive surface may cover the display panel, and when the touch-sensitive surface detects a touch operation on or near it, it is transmitted to the processor 170 to determine the type of the touch event, and then the processor 170 displays the touch event according to the type of the touch event. The corresponding visual output is provided on the panel. Although in Figure 10 the touch-sensitive surface and the display panel are implemented as two separate components to implement the input and output functions, in some embodiments, the touch-sensitive surface and the display panel may be integrated to implement the input and output functions.

Device 1000 may also include at least one sensor 150, such as a light sensor. Specifically, the light sensor may include an ambient light sensor, wherein the ambient light sensor may adjust the brightness of the display panel according to the brightness of the ambient light.

Audio circuitry 160 , speaker 161 , and microphone 162 may provide an audio interface between the user and device 1000 . The audio circuit 160 can transmit the received audio data converted electrical signal to the speaker 161, and the speaker 161 converts it into a sound signal for output; on the other hand, the microphone 162 converts the collected sound signal into an electrical signal, which is converted by the audio circuit 160 into an electrical signal. After receiving, it is converted into audio data, and then processed by the output processor 170, and then sent to other devices such as in the vehicle via the transceiver 110, or the audio data is output to the memory 120 for further processing.

The processor 170 is the control center of the device 1000, and uses various interfaces and lines to connect various parts of the entire device 1000, by running or executing the software programs and/or modules stored in the memory 120, and calling the data stored in the memory 120. , perform various functions of the device 1000 and process data, so as to monitor the device 1000 as a whole. Optionally, the processor 170 may include one or more processing cores; preferably, the processor 170 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, etc. , the modem processor mainly deals with wireless communication. It can be understood that, the above-mentioned modulation and demodulation processor may not be integrated into the processor 170 .

The processor 170 may invoke the code stored in the memory 120 to implement the target tracking method described in FIG. 5 to FIG. 7 in the above embodiment.

Specifically, in this embodiment, the display unit of the device 1000 may be a touch screen display, and the processor 170 of the device 1000 will execute at least one instruction, at least one program, a code set or an instruction set stored in the memory 120 to implement the above embodiments target localization method.

In another exemplary embodiment, an embodiment of the present disclosure further provides a storage medium, where the storage medium stores at least one instruction, at least one piece of program, code set or instruction set, at least one instruction, at least one piece of program, code The set or instruction set is loaded and executed by the processor of the device to implement the method for locating the object in the above embodiment.

Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system, device and unit described above may refer to the corresponding process in the foregoing method embodiments, which will not be repeated here.

In the several embodiments provided in this application, it should be understood that the disclosed system, apparatus and method may be implemented in other manners. For example, the apparatus embodiments described above are only illustrative. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods, for example, multiple units or components may be combined or integrated. to another system, or some features can be ignored, or not implemented. On the other hand, the shown or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection of devices or units, and may be in electrical, mechanical or other forms.

Units described as separate components may or may not be physically separated, and components shown as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit. The above-mentioned integrated units may be implemented in the form of hardware, or may be implemented in the form of software functional units.

The integrated unit, if implemented in the form of a software functional unit and sold or used as an independent product, may be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the present application can be embodied in the form of software products in essence, or the parts that contribute to the prior art, or all or part of the technical solutions, and the computer software products are stored in a storage medium , including several instructions for causing a computer device (which may be a personal computer, a server, or other network device, etc.) to execute all or part of the steps of the method described in the embodiment of FIG. 2a of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk and other media that can store program codes .

As mentioned above, the above embodiments are only used to illustrate the technical solutions of the present application, but not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand: The technical solutions described in the embodiments are modified, or some technical features thereof are equivalently replaced; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

A target tracking method, characterized in that the method comprises:

Obtain the point cloud data of the target object, and perform centroid estimation on the point cloud data;

Obtain a target covariance matrix; wherein, the target covariance matrix represents the centroid position error introduced when performing the centroid estimation; and,

Based on the target covariance matrix, target tracking is performed on the target object to obtain a target tracking result.
The method according to claim 1, wherein the obtaining the target covariance matrix comprises:

Obtain the target tracking result of the target object at the previous moment;

Based on the motion model, state prediction is performed on the target tracking result at the previous moment to obtain the target tracking prediction value at the current moment;

Obtaining a state measurement value at the current moment, where the state measurement value is obtained by performing centroid estimation on the point cloud data;

The target covariance matrix is obtained based on the target tracking predicted value and the state measurement value.
The method according to claim 1, wherein the obtaining the target covariance matrix comprises:

In the case that the number of data associations is less than or equal to a preset value, the preset covariance matrix is used as the target covariance matrix; wherein, the number of data associations indicates that the difference between the centroid measurement value and the state predicted value is less than a threshold value number of times, the centroid measurement value is obtained by estimating the centroid of the point cloud data of the target object at the current moment, and the state prediction value is obtained by predicting the centroid measurement value at the previous moment. The measured value is obtained by estimating the centroid of the point cloud data of the target object at the previous moment.
The method according to claim 3, wherein the obtaining the target covariance matrix comprises:

When the number of data associations is greater than the preset value, the target covariance matrix is obtained based on the target tracking predicted value and the state measurement value; wherein the target tracking predicted value is the target tracking at the previous moment The result is obtained by performing state prediction, and the state measurement value is obtained by performing centroid estimation on the point cloud data.
The method according to claim 2, wherein the obtaining the target covariance matrix based on the target tracking predicted value and the state measurement value comprises:

Based on the target tracking predicted value and the state measurement value, the target covariance matrix is obtained by the following formula:

Wherein, the x k+1|k and y k+1|k are the state measurement values, and the
is the target tracking predicted value, and the R 1 is the target covariance matrix.
The method according to any one of claims 1 to 5, wherein the method further comprises:

obtaining a measurement equation covariance matrix, wherein the point cloud data is collected based on a radar system, and the measurement equation covariance matrix represents the measurement deviation of the radar system;

The measurement equation covariance matrix and the target covariance matrix are synthesized to obtain the synthesized target covariance matrix;

Correspondingly, performing target tracking on the target object based on the target covariance matrix includes:

Based on the synthesized target covariance matrix, target tracking is performed on the target object.
The method according to claim 6, wherein the measurement equation covariance matrix is a matrix represented in a polar coordinate system, and the target covariance matrix is a matrix represented in a Cartesian coordinate system; The measurement equation covariance matrix and the target covariance matrix are synthesized, including:

converting the measurement equation covariance matrix into a matrix represented in a Cartesian coordinate system;

Matrix addition is performed on the converted measurement equation covariance matrix and the target covariance matrix to obtain a combined target covariance matrix.
The method according to claim 7, wherein the converting the measurement equation covariance matrix into a matrix represented in a Cartesian coordinate system comprises:

Obtaining a state measurement value at the current moment, where the state measurement value is obtained by performing centroid estimation on the point cloud data;

Based on the state measurement value, a covariance transformation synthesis matrix is obtained; wherein the covariance transformation synthesis matrix includes a plurality of elements, and some elements in the plurality of elements are generated based on the state measurement value;

Based on the covariance transformation synthesis matrix, the measurement equation covariance matrix is transformed into a matrix represented in a Cartesian coordinate system.
The method according to claim 8, wherein, performing matrix addition on the converted measurement equation covariance matrix and the target covariance matrix, comprising:

The converted measurement equation covariance matrix and the target covariance matrix are subjected to matrix addition by the following formula:

R all =R 0 +A k R m A k ′;

Wherein, the R 0 is the target covariance matrix, the A k R m A k ′ is the converted measurement equation covariance matrix, the R k and the θ k are the state measurement values , the A k is the covariance transformation synthesis matrix, and the A k ′ is the transpose of the covariance transformation synthesis matrix.
A target tracking device, characterized in that the device comprises:

an acquisition module, used for acquiring point cloud data of the target object, and performing centroid estimation on the point cloud data; acquiring a target covariance matrix; wherein, the target covariance matrix represents the centroid position error introduced when performing the centroid estimation ;as well as,

A target tracking module, configured to perform target tracking on the target object based on the target covariance matrix to obtain a target tracking result.
The device according to claim 10, wherein the acquisition module is specifically configured to:

Obtain the target tracking result of the target object at the previous moment;

Based on the motion model, state prediction is performed on the target tracking result at the previous moment to obtain the target tracking prediction value at the current moment;

Obtaining a state measurement value at the current moment, where the state measurement value is obtained by performing centroid estimation on the point cloud data;

The target covariance matrix is obtained based on the target tracking predicted value and the state measurement value.
The device according to claim 10, wherein the acquisition module is configured to use a preset covariance matrix as the target covariance matrix when the number of data associations is less than or equal to a preset value; wherein , the number of data associations represents the number of times that the difference between the centroid measurement value and the state predicted value is less than the threshold, the centroid measurement value is obtained by performing centroid estimation on the point cloud data of the target object at the current moment, and the state predicted value is It is obtained by predicting the centroid measurement value at the last moment, and the centroid measurement value at the last moment is obtained by performing centroid estimation on the point cloud data of the target object at the last moment.
The device according to claim 12, wherein the obtaining module is configured to obtain the target based on a target tracking predicted value and a state measurement value when the data association times are greater than the preset value Covariance matrix; wherein, the target tracking prediction value is obtained by performing state prediction on the target tracking result at the previous moment, and the state measurement value is obtained by performing centroid estimation on the point cloud data.
The device according to claim 11, wherein the acquisition module is specifically configured to:

Based on the target tracking predicted value and the state measurement value, the target covariance matrix is obtained by the following formula:

Wherein, the x k+1|k and y k+1|k are the state measurement values, and the
is the target tracking predicted value, and the R 1 is the target covariance matrix.
The device according to any one of claims 10 to 14, wherein the acquiring module is further configured to:

obtaining a measurement equation covariance matrix, wherein the point cloud data is collected based on a radar system, and the measurement equation covariance matrix represents the measurement deviation of the radar system;

The device also includes:

a matrix synthesis module for synthesizing the measurement equation covariance matrix and the target covariance matrix to obtain a synthesized target covariance matrix;

Correspondingly, the target tracking module is specifically configured to: perform target tracking on the target object based on the synthesized target covariance matrix.
The device according to claim 15, wherein the measurement equation covariance matrix is a matrix represented in a polar coordinate system, and the target covariance matrix is a matrix represented in a Cartesian coordinate system; the matrix Synthesis module, specifically for:

converting the measurement equation covariance matrix into a matrix represented in a Cartesian coordinate system;

Matrix addition is performed on the converted measurement equation covariance matrix and the target covariance matrix to obtain a combined target covariance matrix.
The device according to claim 16, wherein the matrix synthesis module is specifically used for:

Obtaining a state measurement value at the current moment, where the state measurement value is obtained by performing centroid estimation on the point cloud data;

Based on the state measurement value, a covariance transformation synthesis matrix is obtained; wherein the covariance transformation synthesis matrix includes a plurality of elements, and some elements in the plurality of elements are generated based on the state measurement value;

Based on the covariance transformation synthesis matrix, the measurement equation covariance matrix is transformed into a matrix represented in a Cartesian coordinate system.
The device according to claim 17, wherein the matrix synthesis module is specifically used for:

The converted measurement equation covariance matrix and the target covariance matrix are subjected to matrix addition by the following formula:

R all =R 0 +A k R m A k ′;

Wherein, the R 0 is the target covariance matrix, the A k R m A k ′ is the converted measurement equation covariance matrix, the R k and the θ k are the state measurement values , the A k is the covariance transformation synthesis matrix, and the A k ′ is the transpose of the covariance transformation synthesis matrix.
A target tracking device, comprising: a processor and a transmission interface; when the processor calls a program code stored in a memory, the target tracking device executes the method described in any one of claims 1-9. method.
A computer program product comprising instructions, wherein the instructions, when run on a computer or processor, cause the computer or the processor to perform the method of any one of claims 1-9.
A computer-readable storage medium, comprising instructions, characterized in that, when the instructions are executed on a computer or a processor, the computer or the processor is made to execute any one of claims 1-9. Methods.