RU2814813C1 - Device and method for tracking objects - Google Patents

Abstract

FIELD: measuring equipment.

SUBSTANCE: invention relates to measurements. To this end, the object tracking device comprises a data acquisition module configured to receive data from radars mounted on a vehicle, wherein the obtained data include the coordinates of the target, the level of the reflected signal and the radial speed of the target; and a data processing and analysis module configured to cluster the obtained data and track the movement of the object by assigning a new position to the object, wherein each target in the obtained data is considered to be a point in space according to its coordinates, wherein the clustering comprises combining into clusters of points, which are from each other within the combination distance and have close radial speed, wherein the distance of combining this point with other points depends on the level of the reflected signal of that point.

EFFECT: high accuracy of determining and tracking objects and efficiency of the driver assistance system.

5 cl, 21 dwg, 1 tbl

Description

Field of technology to which the invention relates

The present invention relates to the field of sensing, and more particularly to identifying and tracking objects based on radar data and driving traffic based thereon.

State of the art

The automotive industry around the world is developing very actively. The widespread development and implementation of new technologies has led to the active development of automated solutions for vehicles - the creation of advanced driver assistance systems and fully autonomous vehicles (unmanned vehicles).

It is predicted that by 2035, self-driving cars could make up up to 25% of all cars sold worldwide. Their active use will lead to a reduction in the car park in cities by 60%, exhaust gases by 80%, and road accidents by 90%.

The process of moving unmanned or driver-assisted vehicles is similar and can be represented in the following steps, which are performed continuously in real time with varying degrees of detail (depending on the level of automation of the vehicle):

1. Localization. The vehicle determines the location of various objects: traffic lights, road markings, road signs, obstacles, etc.

2. Perception. The vehicle identifies surrounding objects and determines their speed and direction of movement.

3. Prediction. The vehicle predicts processes and events - for example, the movement of other cars.

4. Planning. Based on previous tasks, vehicle algorithms determine further actions and issue commands to control systems or recommendations to the driver.

The Advanced Driver Assistance System, or ADAS for short, includes various interaction and alert algorithms. Most manufactured vehicles are now equipped with varying levels of ADAS systems depending on the vehicle class. A key factor driving the growth of the global ADAS systems market has been government regulations in several countries. Fleets' interest in ADAS is driven by the desire to reduce operating costs and the likelihood of potential incidents.

ADAS includes various algorithms whose task is to analyze the situation on the road and warn the driver about the occurrence of a dangerous incident while driving. ADAS uses various sensors to obtain information about the surrounding world: cameras, radars and lidars.

The general principles of operation of all automated vehicles are similar and they use many technologies:

1. Photo and video cameras for detecting various objects.

2. Radars for detecting air, sea and ground objects, as well as for determining their range, speed and geometric parameters by emitting radio waves.

3. Lidars for creating point clouds, measuring distances and determining other parameters by emitting light.

4. Other sensors for monitoring the state of the internal systems of the vehicle and the surrounding space.

5. Artificial intelligence for processing information received from sensors and cameras to control a vehicle or support decision-making.

Radar data is used to solve a wide range of problems. They, firstly, can provide additional information about the state of the surface and objects along with optical data, and secondly, they can serve as a source of unique information that can be obtained either only from radar data or from ground-based measurements.

It is a known fact that the operation of the radar is less affected than all other sensors used in autonomous systems by the weather conditions under which the operation is carried out. In the process of receiving and transmitting radar signals, there is no interference when working in rain, snow or dusty conditions on the road.

When developing ADAS systems using commercially available radars implementing traditional methods, the following problems arise:

- problems of compatibility of data produced by the radar and the internal logic of the device being developed. When using ready-made solutions, expectations do not always correspond to the results obtained, especially when used under conditions that do not comply with the manufacturer’s recommendations.

- when using the radar in object tracking (MOT) mode, multiple errors in identifying objects occur, although in fact these objects are not in the path of the vehicle, are not moving, are not on the road, their trajectory does not intersect with the trajectory of this vehicle, and etc. (See the example in Fig. 1, where the radar detects an object as an obstacle even though it is not on the road and is not moving).

- when trying to combine data from two or more radars, phantom (non-existent) objects, jumping objects, bifurcating objects appear (for example, in Fig. 2A-2C, reflecting successive moments in time, object 105, as it moved near the radar, split into objects 105 and 31 , although in fact it was one walking person) and other errors in identifying and tracking objects.

- difficulty in interpreting the received signals. Commercially available radars generally work best when measuring large moving metal objects, i.e. determining the position and direction of other vehicles, and are therefore widely used in the implementation of adaptive cruise control systems. Detecting other common road users, such as pedestrians or cyclists, is a much more difficult task for radars.

Accordingly, in the prior art there is a need to create devices and methods capable of correctly identifying and tracking objects, as well as systems and methods for controlling the movement of a vehicle based on them.

The essence of the invention

According to the present invention, an object tracking device is disclosed, comprising:

a data acquisition module configured to receive data from one or more vehicle-mounted radars, the acquired data including target coordinates, reflected signal strength, and target radial velocity; And

a data processing and analysis module configured to cluster the received data and track the movement of an object by assigning a new position to the object, if the cluster obtained as a result of clustering has an association with this tracked object, while each target in the received data is considered as a point in space according to its coordinates, while clustering involves combining into clusters points that are located from each other within the combining distance and have a close radial velocity, while the distance of combining a given point with other points depends on the level of the reflected signal of this point.

In one embodiment, the target for the radar is the irradiated object that reflected the signal, the coordinates of the target include the distance to the target and azimuth relative to the radar.

In one embodiment, the points have similar radial speed if the deviation of the radial speed between them does not exceed a predetermined threshold.

In one embodiment, the radial velocity deviation threshold is 1 m/s.

In one embodiment, the clustering is based on the DBSCAN method.

In addition, according to the present invention, a method for tracking objects is disclosed, comprising the steps of:

receiving data from one or more radars mounted on the vehicle, the received data including the coordinates of the target, the level of the reflected signal and the radial speed of the target;

perform clustering of the received data based on the DBSCAN method, while each target in the received data is considered as a point in space according to its coordinates, while the clustering contains clustering of points that are located from each other within the merging distance and have a similar radial velocity, with In this case, the distance of combining a given point with other points depends on the level of the reflected signal of this point; And

track the movement of an object by assigning a new position to the object if the cluster obtained as a result of clustering has an association with this tracked object.

In one embodiment, the target for the radar is the irradiated object that reflected the signal, the coordinates of the target include the distance to the target and azimuth relative to the radar.

In one embodiment, the points have similar radial speed if the deviation of the radial speed between them does not exceed 1 m/s.

Technical result

Thanks to the above features, the present invention allows:

- improve the accuracy and efficiency of driver assistance systems;

- improve road safety;

- provide the ability to integrate simple and inexpensive radars on the market that do not have complex built-in algorithms for identifying and tracking objects into developed vehicles and ADAS systems, by processing directly “raw” data received from radars, according to the methods outlined in this document;

- provide the ability to fine-tune methods for identifying and tracking objects to adapt to the logic of the system being developed without depending on the algorithms laid down by the radar manufacturer.

Brief description of drawings

It should be understood that the figures may be presented schematically and not to scale and are intended primarily to enhance understanding of the present invention.

Detailed description

Among the existing ADAS systems are the following:

- Anti-lock Braking Systems (ABS): Standard on many modern vehicles, ABS prevents wheel locking during emergency braking by detecting differences in wheel speed and then applying the brakes to the corresponding wheels in rapid bursts until they rotate at the same speed. speed as others.

- Brake Assist (BA): Brake Assist, or Emergency Brake Assist (EBA), is designed to recognize emergency braking using data from the brake sensor and front radar or ultrasonic sensors. If the driver does not press the brake pedal hard enough, the emergency braking system, working in conjunction with the ABS system, is activated to provide additional braking force.

- Stability Program (ESP): ESP, also known as Electronic Stability Control (ESC), is designed to reduce the risk of skidding by applying the brakes to the affected wheels and reducing engine power when it detects that the vehicle is over- or under-reacting to steering inputs when cornering. collision avoidance maneuvers or when there is a sudden change in road conditions and traction. It is important that any additional equipment on the vehicle, such as a towbar, is installed and programmed correctly. When a trailer or van is attached to the vehicle, ESP automatically adjusts to take into account the additional load and weight distribution.

- Adaptive Cruise Control (ACC): Active Cruise Control (ACC), also known as autonomous cruise control, intelligent cruise control or radar cruise control, automatically adjusts the vehicle's speed to maintain a safe distance from the vehicle ahead. Most systems automatically shut off at a certain speed, but some manufacturers offer a Stop & Go or Queue Assist feature that can be used in slow-moving traffic.

- Forward Collision Warning (FCW): FCW, also known as Forward Collision Warning System or Forward Collision Early Warning System, continuously monitors the road ahead to detect objects in the vehicle's path, such as pedestrians, animals or other vehicles. In the event of an imminent collision, the system provides the driver with audio or visual signals so that he can take evasive action, that is, brake or avoid an obstacle.

- Autonomous Emergency Braking (AEB): AEB is a further development of the collision avoidance system. Instead of simply warning the driver, it applies the brakes and/or steering without the driver's input to mitigate the impact of an accident or even prevent it entirely.

- Lane Departure Warning (LDW): LDW systems are designed to prevent the vehicle from unintentionally drifting out of its lane. If the system detects that the vehicle is veering off course without any appropriate signals, it provides audiovisual signals so that the driver can correct the trajectory.

- Lane Assist (LKA): LKA or Lane Assist systems represent the next step in the development of Lane Assist systems. Instead of simply warning the driver about leaving the lane, LKA automatically corrects the vehicle's trajectory, returning the vehicle to its original position without any driver intervention. Some vehicles also have an advanced version of this system called Adaptive Lane Control. It constantly adjusts course to ensure the vehicle is always in the center of its lane.

- Lane Change Assist: Commonly referred to as Blind Spot Detection or Blind Spot Awareness, this system monitors the area around the vehicle for lane movement. If the system determines that it is unsafe to change lanes because of another vehicle moving in the lane you need to change into, especially in the driver's blind spot, it sounds an alarm.

- Intelligent Speed Adaptation (ISA): ISA helps the driver maintain a safe and/or legal speed. If the vehicle exceeds this limit, the system will either warn the driver or take action to reduce speed by reducing engine power.

- Traffic Sign Recognition (TSR): By scanning the road ahead, TSR helps drivers identify road signs ahead, such as speed limit signs or potential hazard signs. The system detects the sign and transmits its image to the instrument panel and/or infotainment system display to alert the driver.

- Adaptive headlights: Adaptive headlights always keep the light on the road thanks to information received from the steering system, as well as information about the speed and angle of the vehicle. As well as improving visibility in low light conditions, it also reduces the likelihood of dazzling oncoming drivers.

- Tire Pressure Monitoring Systems (TPMS): The TMPS system alerts the driver when one or more tires are underinflated by measuring either wheel speed or the pressure inside each tire. In addition to improving safety, the system helps reduce exhaust gases and fuel consumption.

- Night Vision Systems: Night vision systems using thermal imaging cameras can detect objects outside the area illuminated by the vehicle's headlights, warning the driver of possible collisions in low light or poor weather conditions.

- Parking Assist: One of the most common ADAS technologies, Parking Assist can range from rear view cameras that simply provide audio or visual alerts to the driver about the proximity of any obstacles, to fully autonomous systems that take care of the controls themselves. car when entering and/or leaving a parking space.

- Driver Fatigue Monitoring System: Also known as driver drowsiness detection systems, these systems can detect fatigue by observing the driver's behavior at the beginning and during each trip. If any signs of fatigue are detected, the system sends a warning signal to the driver.

The ADAS system can warn the driver of the following dangerous situations:

- Warning of critical approach to a pedestrian. Prevents collisions by providing warning signals when there is a pedestrian, bicycle or motorcycle (moving, for example, at a speed of 0 to 40 km/h) ahead.

- Dangerous proximity monitoring system. Warns the driver with signals and images on the screen when approaching too quickly.

- Warns about leaving the lane. The system provides audible and visual warnings and helps restore direction if the vehicle unexpectedly drifts out of its lane.

- Notifies the driver if the vehicle in front begins to move forward even at zero speed (at a complete stop), and the vehicle behind does not move for a specified time - for example, 2 seconds.

- Notifies the driver when a vehicle is in the detection zone. The detection range can be set, for example, to a range of 1.2 m / 2.0 m / 3.0 m (active from 0 to 30 km/h).

- Records footage before and after an accident. The DVR writes data to memory - for example, to a micro SD card every minute in a cycle.

- Recognition of speed limit signs. The system emits a warning signal if the driver is driving at a speed higher than the speed limit.

The Association of Automotive Engineers classifies ADAS systems into levels 0 through 5. Table 1 below provides a brief description of these levels.

Table 1 Level Definition Operating principle 0 Without automation Driver warning or minimal intervention 1 The driver controls The driver and car systems jointly monitor the situation, for example, adaptive cruise control (ACC, TACC) 2 The driver is not in control The automatic system can take over control completely. The driver must be prepared for immediate intervention. 3 No visual control The driver may be distracted while driving. The driver must be able to take control given sufficient notice. 4 Without human intervention The driver can do other things, such as sleep. Use only in pre-geofenced areas. 5 No steering required No driver required

Radar characteristics

Radars use a radar method based on the emission of radio waves and recording their reflections from objects.

Radars have the following technical characteristics that are important for use.

Range. Characterizes the range of distance in which the radar can detect objects (described in meters). The response time of the entire system depends on this characteristic. It is especially important if driving at speeds of more than 60 km/h is envisaged. The average range of values is 1 - 200 meters.

Horizontal field of view. Characterizes the angle (when looking at the car from top to bottom) at which the radar can detect objects. The larger this angle is, the smaller the area of the “dead zone” will be.

Vertical field of view. Characterizes the angle (when looking at the vehicle from the side) at which the radar can detect objects. An important characteristic when the vehicle moves on terrain with ups and downs. In this case, if the angle is small, the radar will not be able to recognize the obstacle ahead.

Radar resolution. The long-range 360° radar systems used in modern autonomous vehicles create a detailed map of the surrounding environment, regardless of external conditions (unlike previously used single sensors). Advanced radar solutions are integrated into intelligent infrastructure to provide a common communication channel for connected vehicles.

Protection class (IP). All types of electrical equipment have this characteristic. For a radar, this characteristic is one of the key ones due to the fact that the radar is usually installed on the front and rear parts of the vehicle - in most cases, in the area of the bumper, which is one way or another susceptible to dirt and water getting on its surface.

Identifying and tracking objects using raw radar data

The purpose of the present invention is to develop methods for processing and integrating radar data to create a new tool and implement radar and radar data into vehicle automation practice, which can significantly improve the accuracy and efficiency of advanced autonomous systems and driver assistance systems, including in urban environments .

The main function of the proposed device and method is to track observed objects and structure the processed data for subsequent transmission over various communication interfaces. In the process of processing radar data, target characteristics such as position and orientation in space, speed and object classification are calculated.

The operation of the method is divided into two main parts:

1) Clustering of signals - segmentation of “raw” radar data, primary identification of objects (candidate objects).

2) Tracking - tracking objects.

Today, there are several algorithmic approaches for solving multiple target tracking problems. Due to limitations on computing resources (in the system being developed, the use of microcontrollers with one computing core is proposed), the GNN (global nearest data association) approach was chosen as a method used within the framework of the proposed invention. Based on likelihood theory, the goal of the GNN method is to minimize the overall distance function that takes into account all assignments from tracking to detection. The GNN algorithm is the easiest to use. It has low computational cost and can lead to adequate performance for tracking sparsely distributed targets.

The GNN algorithm (shown in Fig. 3) can be formally represented in the following steps:

1. Clustering.

2. Search for associations, updating the state of the object.

3. Control of the object's life cycle.

In more detail, the proposed method contains the following steps.

Clustering

Problems at the clustering stage are related to the density of target reflected signals. Visualization of reflection signals from a target such as a dump truck (Continental ARS-408 Radar) is presented in Fig. 4 (Distance to target - 10 meters, Azimuth - 10 degrees) and Fig. 5 (Distance to target - 24 meters, Azimuth - 0 degrees). In FIG. Figure 6 shows a visualization of the reflection signals of a pedestrian type target (Distance to target - 12 meters, Azimuth - 30 degrees). In the above images you can see that the density of reflection signals from objects irradiated by the radar is non-uniform. First of all, this creates complexity during clustering. To solve this problem, it is necessary to select a clustering algorithm that will work equally well with objects of different densities. In addition, it needs to have good performance and low RAM consumption.

These requirements are generally met by DBSCAN, a density-based data clustering algorithm. Given a set of points in some space, the algorithm groups together points that are closely spaced, labeling as outliers points that are alone in low-density regions (whose nearest neighbors lie far away). In FIG. 7 is a schematic representation of the DBSCAN algorithm.

However, using this algorithm to solve the problems directly is impossible, therefore, the present invention proposes to modify the algorithm by replacing the distance constant (distance) of the association from static to dynamic, as well as adding extreme points (outliers) to the cluster.

The distance to a neighboring point is determined by the value of the electromagnetic signature (RCS). In the proposed implementation, the distance value has a linear dependence on RCS, and the Euclidean distance is used as a metric. In FIG. Figure 8 shows a graph of this dependence.

In the proposed algorithm, the dynamic distance is a parameterized value given by an integer that sets the maximum value of the distance at a predefined maximum RCS value.

Radar cross section (RCS) is a measure of how detectable an object is by radar. Therefore, it is called the electromagnetic signature of the object. A larger RCS indicates that the object is easier to detect.

The object reflects a limited amount of radar energy back to the source. Factors that influence this include:

• the material from which the target is made;

• target size relative to the wavelength of the irradiating radar signal;

• absolute target size;

• angle of incidence (the angle at which the radar beam hits a specific part of the target, which depends on the shape of the target and its orientation relative to the radar source);

• reflection angle (the angle at which the reflected beam exits part of the irradiated target depends on the angle of incidence);

• polarization of transmitted and received radiation with respect to the target orientation.

Although emitter power and distance are important for target detection, they are not factors in the RCS calculation since RCS is a property of the target's reflectivity.

The main problem at the stage of clustering two-dimensional data is the difficulty of identifying certain objects against the background of noisy data.

In FIG. 9-10 shows the cluster mode of operation of the radar (raw data) when viewed from above at the entrance to the ring and at the exit from it. The data was obtained using a radar mounted on the front grill of a passenger car. At a distance of ~25 meters from the vehicle there is another passenger car, following the vehicle like a dump truck at a distance of ~30 meters.

The images show that the signals of these vehicles are highlighted by brighter dots, but at the same time, the ambiguity between the reflection of the observed vehicles, metal fences and other foreign objects is easily visible. Accordingly, the presence of information about an object in the form of two-dimensional coordinates (x,y) and RCS value is not enough for the correct interpretation of data by any known clustering algorithms, and to solve the problem it is necessary to have additional properties in the object’s parameters.

The present invention proposes to use the radial velocity of the target as such a property.

In FIG. 11-13 show the results of an experiment in which radial velocity was added to the two-dimensional coordinates of the signals as a third dimension (Z axis). As you can see, it is now possible to identify moving vehicles.

The radar can operate on the FMCW (Frequency Modulated Continuous Wave) principle and measure distance and speed (Doppler effect) in a single measurement cycle. In another embodiment, the radar may use pulse modulation (PCM) for measurement. Other suitable speed measurement methods are possible. Different measurement principles give different accuracy. FMCW produces more accurate measurements resulting in higher reflectivity density than PCM radar.

The distance to an object can be represented both in azimuth and in the Cartesian system. However, if it is possible to accurately determine the coordinates of an object in space by azimuth and distance, then the relative speed itself does not show the direction of movement of the object, it only indicates how quickly the target is moving away or approaching the source. But having a relative (radial) velocity measurement improves the clustering process, helping to better segment signals by speed.

Thus, the DBSCAN algorithm evaluates whether a signal belongs to a cluster after sorting the signals by relative speed.

As a result of observations, it was found that a group of signals that obviously belong to the same object may have a small error in the radial velocity of up to 1 m/s. It is proposed to take this feature into account in the DBSCAN algorithm by introducing the appropriate parameter.

In the proposed version of the algorithm, the calculation of cluster centers occurs in the same cycle as the traversal of all neighbors that satisfy the search condition (search for neighbors within a radius based on the RCS of the current signal). However, if it is necessary to obtain cluster parameters such as size and spatial orientation, it is necessary to resort to additional calculations, such as calculating the covariance matrix and extracting its eigenvalues and vectors.

Covariance data, represented as a 2x2 matrix, is used to determine the size of the distribution and its spatial orientation.

where the main diagonal of the matrix represents the variances along the x and y coordinates, and the non-main diagonal represents the covariance between x and y according to the following formula:

The eigenvectors and numbers of a given covariance matrix represent the direction of the axes and their dimensions of the distribution on the plane within 1 sigma, which is further considered as the length and width of the object. A representation of the covariance matrix is shown in Fig. 14.

Search for associations, update object state, filtering

At this stage, associations of objects obtained at the clustering stage with the tracked objects are searched. By association we mean the distance from an object to the nearest neighboring cluster (Euclidean distance is taken as the distance metric).

If such associations are found, the object is assigned a new state.

A new state means assigning a new position to an object by filtering the current position of the associated cluster with the previous position of the object, as well as filtering its covariance matrix of geometric parameters using the Running Average algorithm.

The Kalman filter in this system allows you to solve the following problems: smooth out measurement noise, combine data from various sensors, and also estimate those state parameters that are not directly measured. This is achieved by describing changes in the state of the system in the form of a mathematical model and applying the Bayesian approach to it for parameters with a Gaussian noise distribution. The mathematical model is based on the assumption of rectilinear and uniform motion of the observed object, while the measured parameters are the distance to the object and the projection of its speed onto the radius vector relative to the radar. The obtained result is transformed from the global reference system into the radar coordinate system and is associated with the speed of the vehicle. To apply the Kalman filtering algorithm, the system of equations is linearized using the Jacobian matrix. The filtering itself is as follows. First, an a priori estimate of the state vector and covariance matrix is performed at a subsequent point in time based on the existing mathematical model. In the absence of measurement results, this step can be repeated many times, but this leads to increased uncertainty. To adjust the state vector, a posteriori estimation is performed, which combines measurements from the radar and vehicle speed sensors. The degree of mixing of the data is called Kalman gain and is determined by the magnitude of the observational error and the covariance matrix. Thus, the final result is obtained from a combination of mathematical prediction and measurement results taking into account the Kalman gain.

If no association is assigned to the tracked object, its new position is calculated based on the previous one. The covariance matrix of geometric parameters remains the same.

If during the check the cluster does not have an association with any of the monitored objects, then it is marked as an object with the “candidate” status. Thus, a new candidate object is identified.

Object life cycle control

This stage controls the creation and deletion of monitored objects.

All tracked objects are created from objects that have candidate status. The criterion for a candidate to transition to object status is M - the number of confirmations (object updates) for N - the number of recent data updates.

Tracks are deleted after D - the number of unconfirmed data for N - the number of last data updates.

This logic allows you to filter out unstable signals associated with various kinds of reflections and noise that are temporary in nature.

System architecture

The proposed radar data processing approaches are used as the basis of the proposed invention for processing and analyzing radar data for autonomous and driver assistance systems.

The proposed invention has the following architecture, shown in Fig. 15:

The data acquisition module collects raw data from one or more radars. The data includes data on reflected signals: the coordinates of the target (the irradiated object that reflected the signal) relative to the radar, such as distance to the target and azimuth, the level (or density) of the reflected signal, the radial speed of the target.

The data processing and analysis module carries out preliminary processing of radar data in accordance with the proposed methods and builds a map of the environment with dynamic objects displayed on it and interprets the results of radar data: performs clustering, determines the position, orientation in space, speed and classification of objects.

Data transmission module - integrates and transmits data to ADAS-type systems. Previously, the module aggregates the data into a special data structure, which contains information about the position and orientation in space, speed and classification of environmental objects. Data transmission to driver assistance systems can be carried out via the CAN bus.

The clustering stage is performed by an algorithm based on DBSCAN, which has undergone a number of changes, one of which is the point merging distance that changes depending on the reflectivity of the target.

At the object tracking stage, observations are associated with tracked targets, as well as tracks are created and deleted.

Example

The present invention has been tested in practice.

First, we used the option without taking into account radial velocity. The Continental ARS-408 radar was used (Operating mode: cluster, Maximum distance: 250 m, Sorting mode: by index, RCS threshold value: -64dB). Clustering parameters (based on DBSCAN): Maximum clustering distance radius: 7m with RCS range -64dB to 64dB, Distribution sigma value: 3, Radial velocity deviation: +-1 m/s. Track update parameters: Initial radius of association search distance: 8m. Life cycle control parameters: Number of updates that determine the life cycle of a track: 6, Number of unconfirmed updates after which the track is deleted: 2, Number of confirmed updates to initialize a track (transition from the “candidate” state to the “object” status): 4. Result The operation of this embodiment is shown in Fig. 16-17 (top view). In addition to the tracked vehicles (marked in red: small - a passenger car, large - a dump truck), there are many foreign objects in the frame.

The result of the variant with identical parameters, but with filtering by the radial velocity of clusters (in this case, all static objects are ignored) is shown in Fig. 18-19 (top view). Only the vehicles to be tracked remained in the frame.

The presented results show that the proposed method can significantly improve the accuracy of determining the position and orientation in space, speed and classification of an object.

Unmanned vehicle

The results of processing radar data using the proposed methods can be used for the following functional areas in driver assistance systems, including in an urban environment:

• Brake Assist (BA).

• Forward Collision Warning System (FCW).

• Autonomous Emergency Braking (AEB) system.

• Lane change assistance system.

• Parking assistance system.

In addition, the proposed methods can be used in an unmanned vehicle. The unmanned vehicle contains a radar, an object tracking device and a vehicle control device. The object tracking device contains the data acquisition module, data processing and analysis module, and data transmission module described above.

The current challenge is to ensure continuous and precise operation of columns of autonomous vehicles moving one after another. In this case, the driven vehicle must constantly monitor both the movement of the leading vehicle and other objects on the road. By tracking a ahead moving object, which is the lead vehicle, in the proposed manner, the follower vehicle can effectively and accurately follow the lead vehicle in a wide range of environmental conditions without encountering collisions with the lead vehicle doubling, disappearing and jumping and other problems, while spending minimal resources on tracking and freeing them up for other resource-intensive calculations, such as tracking markings, signs, traffic lights, etc.

Features of working on a microcontroller

In one embodiment, the data processing and analysis module may be implemented on a microcontroller.

The use of data processing methods on microcontrollers has some features related to the distribution of computing resources between processes. The task becomes more difficult as the requirements for computers decrease. The process of data processing on microcontrollers can be described as follows.

At the moment of receiving data from the radar, an interrupt occurs at the microcontroller hardware level, after which a transition occurs to the processing function, where the code associated with processing the received data is executed. After exiting the handler, the microcontroller transfers control to the user code.

The process of transmitting data from the radar occurs cyclically. Before sending, the radar can also send a packet with the meaning of the transmitted clusters (objects). If it is possible to set the sorting of transmitted clusters by ID, a cycle change can be determined by the value of the transmitted ID (if the current value is less than the previous one, then the radar has moved on to the next data transmission cycle).

By tracking the change of cycles in this way, you can turn off the interruption for the duration, or ignore the transition to the handler (storing messages in the internal CAN buffer) and transfer control to the functions of the proposed method.

If there is one computing core and limitations in the size of RAM (the inability to use data buffering), the quality of the method depends on the frequency of data transfer. The more time a method takes to compute, the more sparse the input data will be.

The present invention can be effectively used on the basis of a microcontroller, that is, it can reduce the cost of data processing resources and simplify the developed devices, systems and methods for autonomous driving and driver assistance.

Additional implementation features

One or more transmission blocks or devices described herein (transmitters) and one or more reception blocks or devices (receivers) may be physically implemented in the same block or device or in different blocks or devices.

A transmission device or unit in this document, for the sake of simplicity of description, may be referred to as a device or unit having the functions of not only transmitting, but also receiving data, information and/or signals. Likewise, the receiving device or unit may also include functions for transmitting data, information and/or signals.

The various illustrative blocks and modules described in connection with the disclosure herein may be implemented or executed by a general purpose processor, digital signal processor (DSP), application specific integrated circuit (ASIC), field programmable gate array (FPGA), or other programmable logic device (PLD), discrete logic gate or transistor logic, discrete hardware components, or any combination of the foregoing, designed to perform the functions described herein. The general purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices (eg, a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors plus a DSP core, or any other similar configuration).

Some of the units, individually or collectively, may constitute, for example, a computer, and include a processor that is configured to call and execute computer programs from memory to perform method steps or functions of the units in accordance with embodiments of the present invention. According to embodiments, the device may further include memory. The processor may call and execute computer programs from memory to perform the method. Memory may be a separate device, independent of the processor, or may be integrated into the processor. The memory may store code, instructions, commands and/or data for execution on a set of one or more processors of the described device. Codes, instructions, or commands may direct a processor to perform steps of a method or device function.

The functions described herein may be implemented in hardware, software executed by one or more processors, firmware, or any combination thereof. The hardware and software that implement the functions may also be physically located in different locations, including according to a distribution such that portions of the functions are implemented in different physical locations, that is, distributed processing or distributed computing may be performed.

If necessary (for example, if the volume of data and/or calculations that need to be performed on this data is large), multi-threaded data processing can be performed, which in a simple representation can be expressed in the fact that the entire set of data to be processed is divided into a set of subsets , and each processor core performs processing on its assigned subset of data.

The above memory may be volatile or non-volatile memory, or may include both volatile and non-volatile memory. One skilled in the art will also understand that when talking about memory and the storage of data, programs, codes, instructions, commands, etc., it is meant to be a machine-readable (or computer-readable, processor-readable) storage medium. Computer readable media includes both non-transitory computer storage media and communication media, which includes any transmission medium that facilitates movement of a computer program or portion thereof from one location to another. A non-transitory computer-readable storage medium may be any available medium that can be used to carry or store desired program code media in the form of instructions or data structures, and that can be accessed by a computer, processor, or other general purpose or special purpose processing device. appointments.

By way of example, and not limitation, computer-readable media may include read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electronically erasable programmable read-only memory (EEPROM), flash memory , random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM) , Enhanced Speed Synchronous Dynamic Random Access Memory (ESDRAM), Synchronous Line DRAM (SLDRAM) and Direct Access Random Access Memory (DR RAM), register, cache memory, semiconductor memories, magnetic media such as internal hard drives and removable drives, magneto-optical media and optical media such as CD-ROMs and digital versatile disks (DVDs), as well as any other storage media known in the art.

The information and signals described herein may be represented using any of a variety of different technologies. For example, the data, instructions, commands, information, signals, bits, symbols and chips that may be exemplified in the above description may be represented by voltages, currents, electromagnetic waves, magnetic fields or particles, optical fields or particles, or any combinations of the above, as applicable to the present invention.

It should be understood that while terms such as “first,” “second,” “third,” and the like may be used herein to describe various blocks, modules, networks, elements, components, regions, layers, and/or sections. ., these blocks, modules, networks, elements, components, areas, layers and/or sections should not be limited to these terms. These terms are used only to distinguish one block, module, network, element, component, area, layer or section from another block, module, network, element, component, area, layer or section. Thus, a first block, module, network, element, component, region, layer or section may be referred to as a second block, module, network, element, component, region, layer or section without departing from the scope of the present invention. As used herein, the term “and/or” includes any and all combinations of one or more of the respective items listed. Elements referred to in the singular do not exclude the plurality of elements unless specifically stated otherwise.

The functionality of an element specified in the description or claims as a single element may be implemented in practice by means of several components of the device, and conversely, the functionality of elements specified in the description or claims as several separate elements may be realized in practice by means of a single component.

Although exemplary embodiments have been described in detail and shown in the accompanying drawings, it is to be understood that such embodiments are merely illustrative and are not intended to limit the present invention, and that the invention should not be limited to the specific arrangements and structures shown and described, since Various other modifications and embodiments of the invention may be apparent to one skilled in the art based on the information set forth in the specification and knowledge of the prior art without departing from the spirit and scope of the present invention.

Claims (12)

1. An object tracking device containing: a data acquisition module configured to receive data from one or more radars mounted on the vehicle, the acquired data including target coordinates, reflected signal strength, and target radial velocity, the target for the radar being the irradiated object that reflected the signal; And a data processing and analysis module configured to cluster the received data and track the movement of an object by assigning a new position to the object, if the cluster obtained as a result of clustering has an association with this tracked object, while each target in the received data is considered as a point in space according to its coordinates, while clustering involves combining into clusters points that are located from each other within the merging distance and have a radial speed, the deviation of which between points does not exceed a predetermined threshold of 1 m/s, while the merging distance of this point with other points depends on the level of the reflected signal of this point, while a new position is assigned to the object by filtering the current position of the associated cluster with the previous position of the object, as well as filtering its covariance matrix of geometric parameters using the moving average algorithm, in this case, Kalman filtering is carried out for measurements, and a mathematical model is also used, which is based on the assumption of rectilinear and uniform motion of the observed object, while the measured coordinates of the target are the distance to the object and the projection of its speed onto the radius vector relative to the radar. 2. The device according to claim 1, in which clustering is based on the DBSCAN method. 3. The device according to claim 1, in which the filtering algorithm is as follows: first, an a priori estimate of the state vector and the covariance matrix is performed at a subsequent point in time based on the existing mathematical model; to adjust the state vector, an a posteriori estimate is performed, during which measurements from the radar are combined and vehicle speed sensors, the degree of mixing is determined by the magnitude of the observational error and the covariance matrix, resulting in the final result obtained from a combination of mathematical prediction and measurement results taking into account the Kalman gain. 4. A method for tracking objects, containing stages in which: receiving data from one or more radars mounted on the vehicle, the received data including the coordinates of the target, the level of the reflected signal and the radial speed of the target, the target for the radar being the irradiated object that reflected the signal; perform clustering of the received data based on the DBSCAN method, while each target in the received data is considered as a point in space according to its coordinates, while the clustering contains clustering of points that are located from each other within the merging distance and have a radial speed, the deviation of which between points does not exceed a preset threshold of 1 m/s, while the distance of combining a given point with other points depends on the level of the reflected signal of this point; And track the movement of an object by assigning a new position to the object if the cluster obtained as a result of clustering has an association with this tracked object, while the new position is assigned to the object by filtering the current position of the associated cluster with the previous position of the object, as well as filtering its covariance matrix of geometric parameters using the moving average algorithm, in this case, Kalman filtering is carried out for measurements, and a mathematical model is also used, which is based on the assumption of rectilinear and uniform motion of the observed object, while the measured coordinates of the target are the distance to the object and the projection of its speed onto the radius vector relative to the radar. 5. The method according to claim 4, in which the filtering algorithm is as follows: first, an a priori estimate of the state vector and the covariance matrix is performed at a subsequent point in time based on the existing mathematical model; to adjust the state vector, an a posteriori estimate is performed, during which measurements from the radar are combined and vehicle speed sensors, the degree of mixing is determined by the magnitude of the observational error and the covariance matrix, resulting in the final result obtained from a combination of mathematical prediction and measurement results taking into account the Kalman gain.