WO2020228242A1

WO2020228242A1 - Method and apparatus for tracking target object, and storage medium

Info

Publication number: WO2020228242A1
Application number: PCT/CN2019/112425
Authority: WO
Inventors: 唐得志; 赛影辉; 肖飞; 阴山慧
Original assignee: 奇瑞汽车股份有限公司
Priority date: 2019-05-13
Filing date: 2019-10-22
Publication date: 2020-11-19
Also published as: CN110077402A; CN110077402B

Abstract

A method and apparatus for tracking a target object, and a storage medium, wherein same belong to the technical field of intelligent vehicles. The method comprises: acquiring, by means of a millimeter wave radar mounted on a vehicle, radar information, in a driving direction, of the vehicle; determining, on the basis of the radar information, a target object that meets a tracking condition; and tracking the target object to control the vehicle. The tracking is targeted, thereby improving the accuracy of object tracking, and guaranteeing the driving safety of a vehicle.

Description

Target object tracking method, device and storage medium

Technical field

This application relates to the technical field of smart cars, in particular to a method, device and storage medium for tracking a target object.

Background technique

With the development of technology, cars are becoming more intelligent. Among them, the car may include an adaptive cruise system, and automatically drive under the control of the adaptive cruise system. The key to control through the adaptive cruise system is the tracking of target objects, that is, the tracking of target objects such as other cars and pedestrians in the current direction of the car.

At present, it is possible to obtain real-time target objects and object information in the driving direction through a millimeter-wave radar fixed in front of the car, and track the target objects according to the obtained object information, and control the car according to the tracking results.

However, because the car is driving, the object information obtained by millimeter wave radar may come from the target object, or it may be affected by other noise. In addition, due to the unstable operation of the millimeter-wave radar itself and the uneven return energy, false target objects may be obtained, resulting in inaccurate object tracking. At the same time, with the random bumps and swings during the driving of the car, the millimeter-wave radar measurement signal may be temporarily lost, which leads to large fluctuations in the object information, resulting in the acquisition of target objects and object information, which leads to automatic driving calculations. The amount will increase, reducing the real-time performance of automatic driving, and will also lead to the inability to accurately and timely detect dangerous target objects, which will lead to unnecessary false alarms or misexecution of the car, reduce driving comfort, and even cause safety hazards.

Summary of the invention

The embodiments of the present application provide a method, device, and storage medium for tracking a target object, which are used to solve the problem of low accuracy of target object tracking in related technologies, which leads to poor driving safety of automobiles. The technical solution is as follows:

In one aspect, a method for tracking a target object is provided, and the method includes:

Obtain the radar information of the car in the driving direction through the millimeter wave radar installed on the car;

Based on the radar information, determine target objects that meet the tracking conditions;

The target object is tracked to control the car.

In some embodiments, the determining the target object that meets the tracking condition based on the radar information includes:

Based on the radar information, determine stationary objects, false objects, empty objects, and non-dangerous objects among the objects;

The object other than the prohibited object, the fake object, the empty object, and the non-hazardous object is determined as the target object that meets the tracking condition.

In some embodiments, the radar information includes object information of objects appearing in the driving direction of the car, the lateral distance and the longitudinal distance between the car and the object, and the object information includes The relative distance between the objects, the relative speed, and the number of occurrences of the objects within the detection range of the millimeter wave radar;

The determining of stationary objects, false objects, empty objects and non-dangerous objects in the objects based on the object information includes:

Determining an object whose absolute value of the relative speed is equal to the driving speed of the car among the objects as the stationary object;

Determining an object with a relative distance equal to 0 among the objects to the car as the empty object;

Determining an object with an appearance number greater than or less than an appearance frequency threshold among the objects as the fake object;

Among the objects, an object whose lateral distance is less than the lateral distance threshold and the longitudinal distance is less than the longitudinal distance threshold is determined as the non-dangerous object.

In some embodiments, the acquiring the radar information of the car in the driving direction through the millimeter wave radar installed on the car includes:

Acquiring radar data in the driving direction through the millimeter wave radar;

Preprocessing the radar data to obtain the radar information.

In some embodiments, the preprocessing the radar data to obtain the radar information includes:

Analyze the radar data of the object according to the millimeter wave radar protocol to obtain the object information of the object;

Use the position of the millimeter wave radar as the origin to establish a world coordinate system;

When the object information includes the relative distance between the object and the car and the relative angle between the object and the car, the relative distance is decomposed in the world coordinate system to obtain the lateral distance And the longitudinal distance.

In some embodiments, the tracking the target object to control the car includes:

Projecting the target object from the world coordinate system to the image pixel coordinate system;

The target object is tracked in the image pixel coordinate system to control the automobile.

In some embodiments, the projecting the target object from the world coordinate system to the image pixel coordinate system includes:

According to the coordinate values of the target object in the world coordinate system, project the target object into the image pixel coordinate system by the following projection formula;

X ₁ ＝X*a+image.cols,

Y ₁ ＝Y*b

Wherein, X ₁ is the abscissa of the target object in the image pixel coordinate system, the Y ₁ is the ordinate of the target object in the image pixel coordinate system, and the image pixel coordinate system The median coordinate unit is pixel, the X is the lateral distance of the target object in the world coordinate system, the Y is the longitudinal distance of the target object in the world coordinate system, and the a and b Is the magnification, and the image.cols is the image width.

In some embodiments, the tracking the target object in the image pixel coordinate system to control the car includes:

Select the target object closest to the car;

Use the fourth-order Kalman filter algorithm to predict the selected target object to obtain the predicted position information;

Comparing the predicted position information with reference position information through a comparison formula, where the reference position information is the current position information of the selected target object;

When the predicted position information and the reference position information do not satisfy the comparison formula, and the number of times the comparison formula is not satisfied is greater than or equal to the inconsistency threshold, the reference position information is updated;

When the predicted position information and the reference position information satisfy the comparison formula, and the number of times the selected target object is selected is greater than or equal to the selection number threshold, it is determined that the selected target object collides with the car The duration of the collision.

In another aspect, a device for tracking a target object is provided, and the device includes:

The acquisition module is used to acquire radar information of the car in the driving direction through the millimeter wave radar installed on the car;

A determining module, configured to determine a target object that meets the tracking condition based on the radar information;

The tracking module is used to track the target object to control the car.

In some embodiments, the determining module includes:

The first determining sub-module is configured to determine stationary objects, false objects, empty objects and non-dangerous objects in the objects based on the radar information;

The second determination sub-module is used to determine objects other than the prohibited object, the fake object, the empty object, and the non-dangerous object as target objects that meet the tracking condition.

The first determining submodule is used for:

In some embodiments, the acquisition module includes:

An acquisition sub-module for acquiring radar data in the driving direction through the millimeter wave radar;

The processing sub-module is used to preprocess the radar data to obtain the radar information.

In some embodiments, the processing sub-module is used to:

In some embodiments, the tracking module includes:

Projection sub-module for projecting the target object from the world coordinate system to the image pixel coordinate system;

The tracking sub-module is used to track the target object in the image pixel coordinate system to control the car.

In some embodiments, the projection sub-module is used to:

X ₁ ＝X*a+image.cols,

Y ₁ ＝Y*b

In some embodiments, the tracking sub-module is used to:

Select the target object closest to the car;

In another aspect, a computer-readable storage medium is provided, the storage medium stores a computer program, and the computer program implements the steps of the target object tracking method provided above when the computer program is executed by a processor.

In another aspect, an automobile is provided, and the automobile includes:

processor;

A memory for storing processor executable instructions;

Wherein, the processor is configured to execute the steps of the target object tracking method provided above.

On the other hand, a computer program product containing instructions is provided, which when running on a computer, causes the computer to execute the steps of the target object tracking method provided in the first aspect.

The beneficial effects brought by the technical solutions provided by the embodiments of the present application include at least:

In the embodiments of this application, the radar information in the direction of the car can be acquired, and the target objects that meet the tracking conditions can be determined according to the acquired radar information, and then the target objects that meet the tracking conditions can be tracked, so that the tracking is targeted and improved This improves the accuracy of object tracking and ensures the driving safety of the car.

Description of the drawings

In order to more clearly describe the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed in the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained from these drawings without creative work.

FIG. 1 is a flowchart of a method for tracking a target object provided by an embodiment of the present application;

FIG. 2 is a flowchart of another target object tracking method provided by an embodiment of the present application;

FIG. 3 is a schematic diagram of a position between a car and a target object provided by an embodiment of the present application;

FIG. 4 is a schematic structural diagram of a tracking device for a target object provided by an embodiment of the present application;

FIG. 5 is a schematic structural diagram of a determining module provided by an embodiment of the present application;

FIG. 6 is a schematic structural diagram of an acquisition module provided by an embodiment of the present application;

FIG. 7 is a schematic structural diagram of a tracking module provided by an embodiment of the present application;

Fig. 8 is a schematic structural diagram of an automobile provided by an embodiment of the present application.

Detailed ways

In order to make the objectives, technical solutions, and advantages of the present application clearer, the following will further describe the embodiments of the present application in detail with reference to the accompanying drawings.

Before explaining the embodiments of the present application in detail, the application scenarios involved in the embodiments of the present application are first explained.

With the development of technology, cars are becoming more and more intelligent, and cars can drive automatically under the control of the adaptive cruise system. The key to control through the adaptive cruise system is to track the target object. At present, the target object and object information in the driving direction can be obtained in real time through the millimeter wave radar fixed in front of the car, and the target object can be tracked according to the obtained object information. Tracking, controlling the car based on the tracking results. However, because the car is driving, the object information obtained by millimeter wave radar may come from the target object, or it may be affected by other noise. In addition, due to the unstable operation of the millimeter-wave radar itself and the uneven return energy, false target objects may be obtained, resulting in inaccurate object tracking. At the same time, with the random bumps and swings during the driving of the car, the millimeter-wave radar measurement signal may be temporarily lost, which leads to large fluctuations in the object information, resulting in the acquisition of target objects and object information, which leads to automatic driving calculations. The amount will increase, reducing the real-time performance of automatic driving, and will also lead to the inability to accurately and timely detect dangerous target objects, which will lead to unnecessary false alarms or misexecution of the car, reduce driving comfort, and even cause safety hazards.

Based on such an application scenario, embodiments of the present application provide a tracking method for a target object that can improve the accuracy of object tracking.

After introducing the application scenario of the embodiment of the present application, the method for tracking the target object provided by the embodiment of the present application will be described in detail in conjunction with the accompanying drawings.

FIG. 1 is a flowchart of a method for tracking a target object provided by an embodiment of the application. Referring to FIG. 1, the method is applied to a car and includes the following steps.

Step 101: Obtain the radar information of the car in the driving direction through the millimeter wave radar installed on the car.

Step 102: Based on the radar information, determine target objects that meet the tracking conditions.

Step 103: Track the target object to control the car.

In some embodiments, based on the radar information, determining the target object that meets the tracking conditions includes:

Based on the radar information, determine the stationary objects, false objects, empty objects and non-hazardous objects in the object;

Determine the prohibited object, the fake object, the empty object, and the objects other than the non-dangerous object as target objects that meet the tracking conditions.

In some embodiments, the radar information includes object information of objects appearing in the driving direction of the car, the lateral distance and the longitudinal distance between the car and the object, and the object information includes the relative distance between the car and the object. Distance, relative speed, and the number of occurrences of the object within the detection range of the millimeter wave radar;

Based on the object information, determine the stationary objects, false objects, empty objects and non-hazardous objects in the object, including:

Determine an object whose relative speed is equal to the driving speed of the car among the objects as the stationary object;

Determine the object whose relative distance to the car is equal to 0 as the empty object;

Determine the object whose appearance times are greater than or smaller than the appearance times threshold in the object as the fake object;

In some embodiments, obtaining the radar information of the car in the driving direction through the millimeter wave radar installed on the car includes:

Obtain radar data in the driving direction through the millimeter wave radar;

The radar data is preprocessed to obtain the radar information.

In some embodiments, preprocessing the radar data to obtain the radar information includes:

According to the millimeter wave radar protocol, analyze the radar data of the object to obtain the object information of the object;

When the object information includes the relative distance between the object and the car and the relative angle between the object and the car, the relative distance is decomposed in the world coordinate system to obtain the horizontal distance and the vertical distance.

In some embodiments, tracking the target object to control the car includes:

Project the target object from the world coordinate system to the image pixel coordinate system;

Track the target object in the image pixel coordinate system to control the car.

In some embodiments, projecting the target object from the world coordinate system to the image pixel coordinate system includes:

According to the coordinate value of the target object in the world coordinate system, the target object is projected into the image pixel coordinate system by the following projection formula;

X ₁ ＝X*a+image.cols,

Y ₁ ＝Y*b

Wherein, X ₁ is the abscissa of the target object in the image pixel coordinate system, the Y ₁ is the ordinate of the target object in the image pixel coordinate system, and the coordinate unit in the image pixel coordinate system is pixel, The X is the lateral distance of the target object in the world coordinate system, the Y is the longitudinal distance of the target object in the world coordinate system, the a and b are magnifications, and the image.cols is the image width.

In some embodiments, tracking the target object in the image pixel coordinate system to control the car includes:

Select the target object closest to the car;

Comparing the predicted position information with the reference position information through a comparison formula, where the reference position information is the current position information of the selected target object;

When the predicted position information and the reference position information satisfy the comparison formula, and the number of times the selected target object is selected is greater than or equal to the selection number threshold, the collision duration for the selected target object to collide with the car is determined.

All the above-mentioned optional technical solutions can be combined in any combination to form optional embodiments of the present application, which will not be repeated in the embodiments of the present application.

FIG. 2 is a flowchart of a method for tracking a target object provided by an embodiment of the application. Referring to FIG. 2, the method includes the following steps.

Step 201: The car obtains the radar information of the car in the driving direction through the millimeter wave radar installed on the car.

As the car is driving, in order to avoid obstacles such as other cars and pedestrians in the driving direction in time to ensure the safety of the car, the car can obtain the driving direction of the car through the millimeter wave radar installed on the car On the radar information.

Since the data obtained by the millimeter wave radar is radar data, in order to facilitate information processing by the car, the car can obtain the radar data in the driving direction through the millimeter wave radar; the radar data is preprocessed to obtain the radar information.

It should be noted that the radar information includes object information of objects appearing in the driving direction of the car, the horizontal distance and the longitudinal distance between the car and the object, and the object information includes the relative distance between the car and the object, the relative speed, and the object information. The number of occurrences within the detection range of the millimeter wave radar, etc.

In some embodiments, the car preprocesses the radar data to obtain the radar information. The operation of obtaining the radar information can be: parse the radar data of the object according to the millimeter wave radar protocol to obtain the object information of the object; take the position of the millimeter wave radar as the origin , Establish a world coordinate system; when the object information includes the relative distance between the object and the car and the relative angle between the object and the car, the relative distance is decomposed into the world coordinate system to obtain the horizontal distance and the vertical distance.

It should be noted that, in the embodiments of the present application, the position of the millimeter wave radar may be used as the origin to establish the world coordinate system, or other positions may be used as the origin to establish the world coordinate system.

It should also be noted that, referring to Figure 3, the direction of the car is longitudinal Y, the direction perpendicular to the direction of travel is transverse X, and the lateral distance is the car (position O in Figure 3) and the object (position A in Figure 3) The horizontal distance (OA _{1 in} Figure 3), the longitudinal distance is the vertical distance between the car and the object (OA _{2 in} Figure 3), ∠AOA ₂ is the relative angle, and AO is the relative distance.

In some embodiments, after the car obtains the radar information, the radar information can be stored, and the car can define the structure of the radar information, and then define the array of the structure type, and finally store the radar information in the array .

Step 202: The car determines the target object that meets the tracking condition based on the radar information.

Since the car may detect some invalid objects through the millimeter wave radar, such as empty objects, false objects, stationary objects and/or non-hazardous objects, these invalid objects will not affect the driving safety of the car, and These invalid objects may cause problems for the car to track the target object. Therefore, the car needs to determine the target object from the acquired objects based on the radar information, and the target object is the object that affects the driving safety of the car. That is, the car needs to determine the target object that meets the tracking conditions based on the radar information. Based on the radar information, the operation of the car to determine the target object that meets the tracking conditions can be: based on the object information, determine the stationary object, false object, empty object and non-dangerous object in the object; object, false object, empty object and non-dangerous object will be prohibited Objects other than dangerous objects are determined as target objects that meet the tracking conditions.

As an example, the operation of determining stationary, false, empty, and non-hazardous objects in the object based on object information can be: comparing the absolute value of the relative speed in the object with the driving speed of the car (or with the millimeter wave radar The object with the same moving speed) is determined as a stationary object; the object whose relative distance between the object and the car is equal to 0 is determined as an empty object; the object whose occurrence number is greater than or less than the number of occurrence threshold is determined as a false object; Among the objects, the objects whose lateral distance is less than the lateral distance threshold and the longitudinal distance is less than the longitudinal distance threshold are determined as non-hazardous objects.

It should be noted that the threshold of the number of occurrences, the threshold of the horizontal distance and the threshold of the vertical distance can be set in advance, for example, the threshold of the number of occurrences can be 10, 20, and so on. The horizontal distance threshold can be 2 meters, 3 meters, etc., and the longitudinal distance threshold can be 3 meters, 5 meters, 10 meters, and so on.

As an example, the scanning period of a millimeter-wave radar can be 50ms (milliseconds). Since the millimeter-wave radar scans once every 50ms, the car can obtain radar information. Therefore, objects of every N period can be used as a group, and the most The proximity data association method returns a single object to a single radar information and records the number of appearances of each object; when the number of appearances of any object is less than N, any object is determined to be a false object, and N is the threshold of the number of appearances.

As an example, because the millimeter wave radar may collect multiple times on a single object to obtain multiple radar information, the radar information of a single object can be combined. That is, the car can set the distance error threshold, angle error threshold, and speed error threshold of adjacent periodic objects, and set the objects in every N cycles as a group, which will satisfy that the relative distance is less than the distance error threshold and the relative angle is less than the angle Objects with an error threshold and a relative speed less than the speed error threshold are associated with the same object, so that a single object returns to a single radar information; when the single object appears and disappears within a preset time, the single object is determined to be a false object.

In some embodiments, after determining stationary objects, fake objects, empty objects, and non-hazardous objects, stationary objects, fake objects, empty objects, and non-hazardous objects can be eliminated from the detected objects, and the remaining objects after the elimination Determined as the target object.

Step 203: The car tracks the target object.

Since the target object is an object that affects the driving safety of the car, in order to ensure the safety of the car driving, the target object needs to be tracked after the car determines the target object.

As an example, the operation of tracking the target object by the car may be: projecting the target object from the world coordinate system into the image pixel coordinate system; tracking the target object in the image pixel coordinate system.

As an example, the operation of the car to project the target object from the world coordinate system into the image pixel coordinate system may be: according to the coordinate value of the target object in the world coordinate system, project the target object into the image pixel coordinate system through a projection formula. That is, according to the coordinate value of the target object in the world coordinate system, the car can project the target object into the image pixel coordinate system through the following projection formula.

It should be noted that in the above projection formula (1), X ₁ is the abscissa of the target object in the image pixel coordinate system, Y ₁ is the ordinate of the target object in the image pixel coordinate system, and the coordinates in the image pixel coordinate system The unit is pixel, X is the horizontal distance of the target object in the world coordinate system, Y is the longitudinal distance of the target object in the world coordinate system, a and b are magnifications, and image.cols is the image width.

As an example, the car can track the target object in the image pixel coordinate system through the Kalman filter algorithm. That is, the operation of the car to track the target object in the image pixel coordinate system can be: select the target object closest to the car; use the fourth-order Kalman filter algorithm to predict the information of the selected target object to obtain the predicted position information; The predicted position information is compared with the reference position information through a comparison formula. The reference position information is the current position information of the selected target object; when the predicted position information and the reference position information do not satisfy the comparison formula, and the number of times the comparison formula is not satisfied is greater than or equal to When the threshold of the number of times of inconsistency is different, the reference position information is updated. When the predicted position information and the reference position information satisfy the comparison formula, and the number of times the selected target object is selected is greater than or equal to the selection number threshold, the collision duration of the selected target object and the car is determined.

It should be noted that when a car selects the target object closest to the car, it can use the longitudinal distance, the horizontal distance or the relative distance as a reference, and sort the target objects in the order from near to far or from far to near. Select the target object closest to the car.

It should also be noted that the threshold of the number of inconsistencies and the threshold of the number of selections can be set in advance. For example, the threshold of the number of inconsistencies can be 3, 4, etc., and the threshold of the number of selections can be 3, 4, and so on. The current position information of the selected target object can be determined by the above projection formula (1), that is, the reference position information of the selected target object can be determined by the above projection formula (1).

As an example, the car can predict the predicted position information of the selected target object through the following prediction formula.

It should be noted that in the above prediction formula (2), (x _n+1 , y _n+1 , Δx _n+1 , Δy _n+1 ) is the predicted position information, (x _n , y _n , Δx _n , Δy _n ) is the reference position information, x _n , y _n , Δx _n , Δy _n are the abscissa, ordinate, rate of change along the abscissa and along the ordinate of the selected target object in the image pixel coordinate system, respectively The rate of change.

As an example, the car can compare the predicted position information with the reference position information through the following comparison formula.

It should be noted that in the above comparison formula (3), x _t and y _t are allowable errors, and the allowable errors can be set in advance, for example, the allowable errors can be 0.1, 0.2, and so on.

As an example, updating the reference position information by the car may refer to reselecting the target object closest to the car, and determining the current position information of the newly selected target object as the reference position information.

It should be noted that when the predicted position information and the reference position information do not meet the comparison formula, the number of times the comparison formula is not met can be increased first, and then it is determined whether the number of times the comparison formula is not met is greater than or equal to the threshold of the number of inconsistencies, and when the comparison is not met When the number of formulas is less than the threshold of the number of inconsistencies, the fourth-order Kalman filter algorithm is used to predict the information of the selected target object again. When the predicted position information and the reference position information satisfy the comparison formula, the number of times the selected target object is selected can be increased first, and then it is determined whether the number of times the selected target object is selected is greater than or equal to the selection times threshold, and when the selected target object is selected When the number of selections is less than the selection threshold, the information prediction of the selected target object is performed again through the fourth-order Kalman filter algorithm.

Step 204: The car controls the car according to the tracking result of the target object.

It can be seen from the above step 203 that the car can include the collision duration of the selected target object and the car according to the tracking result of the target object. Therefore, the car can control the car according to the collision duration of the target object and the car.

As an example, when the collision duration is less than or equal to the collision duration threshold, the car is controlled to perform braking control, and when the collision duration is greater than the collision duration, the car can update the reference position information.

It should be noted that the collision duration threshold can be set in advance, for example, the collision duration threshold can be 20 seconds, 30 seconds, 1 minute, and so on.

In the embodiment of this application, the car can obtain radar information in the direction of the car, and eliminate empty objects, false objects, non-hazardous objects, and prohibited objects according to the obtained radar information, thereby determining target objects that meet the tracking conditions, and then Target objects that meet the tracking conditions are tracked, so that the tracking is targeted, avoiding the tracking of invalid objects and the interference of invalid objects to the target object, improving the accuracy of object tracking, and ensuring the driving safety of the car.

After explaining the target object tracking method provided in the embodiment of the present application, next, the target object tracking device provided in the embodiment of the present application will be introduced.

Fig. 4 is a block diagram of a device for tracking a target object provided by an embodiment of the present disclosure. Referring to Fig. 4, the device can be implemented by software, hardware or a combination of both. The device includes: an acquisition module 401, a determination module 402, and a tracking module 403.

The obtaining module 401 is configured to obtain radar information of the car in the driving direction through the millimeter wave radar installed on the car;

The determining module 402 is configured to determine a target object that meets the tracking condition based on the radar information;

The tracking module 403 is used to track the target object to control the car.

In some embodiments, referring to FIG. 5, the determining module 402 includes:

The first determining sub-module 4021 is configured to determine stationary objects, false objects, empty objects and non-dangerous objects in the objects based on the radar information;

The second determining sub-module 4022 is configured to determine objects other than the prohibited object, the false object, the empty object, and the non-dangerous object as target objects that meet the tracking condition.

The first determining submodule 4021 is used to:

In some embodiments, referring to FIG. 6, the acquiring module 401 includes:

An obtaining sub-module 4011 is configured to obtain radar data in the driving direction through the millimeter wave radar;

The processing sub-module 4012 is used to preprocess the radar data to obtain the radar information.

In some embodiments, the processing submodule 4012 is used to:

In some embodiments, referring to FIG. 7, the tracking module 403 includes:

The projection sub-module 4031 is used to project the target object from the world coordinate system to the image pixel coordinate system;

The tracking sub-module 4032 is used to track the target object in the image pixel coordinate system to control the car.

In some embodiments, the projection sub-module 4031 is used to:

X ₁ ＝X*a+image.cols,

Y ₁ ＝Y*b

In some embodiments, the tracking sub-module 4032 is used to:

Select the target object closest to the car;

To sum up, in the embodiments of this application, the car can obtain radar information in the direction of the car, and eliminate empty objects, false objects, non-hazardous objects, and prohibited objects according to the obtained radar information, so as to determine those that meet the tracking conditions The target object is then tracked to the target object that meets the tracking conditions, so that the tracking is targeted, avoiding the tracking of invalid objects and the interference of invalid objects to the target object, improving the accuracy of object tracking and ensuring the car’s performance Driving safety.

It should be noted that when the target object tracking device provided in the above embodiment tracks the target object, only the division of the above functional modules is used as an example for illustration. In actual applications, the above functions can be allocated to different functional modules as needed. Complete, that is, divide the internal structure of the device into different functional modules to complete all or part of the functions described above. In addition, the target object tracking device provided in the above-mentioned embodiment and the target object tracking method embodiment belong to the same concept. For the specific implementation process, please refer to the method embodiment, which will not be repeated here.

FIG. 8 shows a structural block diagram of a car 800 provided by an exemplary embodiment of the present application.

Generally, the car 800 includes a processor 801 and a memory 802.

The processor 801 may include one or more processing cores, such as a 4-core processor, an 8-core processor, and so on. The processor 801 can adopt at least one hardware form among DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array, Programmable Logic Array). achieve. The processor 801 may also include a main processor and a coprocessor. The main processor is a processor used to process data in the awake state, also called a CPU (Central Processing Unit, central processing unit); the coprocessor is A low-power processor used to process data in the standby state. In some embodiments, the processor 801 may be integrated with a GPU (Graphics Processing Unit, image processor), and the GPU is used to render and draw content that needs to be displayed on the display screen. In some embodiments, the processor 801 may further include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

The memory 802 may include one or more computer-readable storage media, which may be non-transitory. The memory 802 may also include high-speed random access memory and non-volatile memory, such as one or more magnetic disk storage devices and flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 802 is used to store at least one instruction, and the at least one instruction is used to be executed by the processor 801 to implement the target object provided in the method embodiment of the present application. Tracking method.

In some embodiments, the car 800 optionally further includes: a peripheral device interface 803 and at least one peripheral device. The processor 801, the memory 802, and the peripheral device interface 803 may be connected by a bus or a signal line. Each peripheral device can be connected to the peripheral device interface 803 through a bus, a signal line or a circuit board. Specifically, the peripheral device includes at least one of a radio frequency circuit 804, a touch display screen 805, a camera 806, an audio circuit 807, a positioning component 808, and a power supply 809.

The peripheral device interface 803 may be used to connect at least one peripheral device related to I/O (Input/Output) to the processor 801 and the memory 802. In some embodiments, the processor 801, the memory 802, and the peripheral device interface 803 are integrated on the same chip or circuit board; in some other embodiments, any one of the processor 801, the memory 802 and the peripheral device interface 803 or The two can be implemented on separate chips or circuit boards, which are not limited in this embodiment.

The radio frequency circuit 804 is used to receive and transmit RF (Radio Frequency, radio frequency) signals, also called electromagnetic signals. The radio frequency circuit 804 communicates with a communication network and other communication devices through electromagnetic signals. The radio frequency circuit 804 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals into electrical signals. Optionally, the radio frequency circuit 804 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, and so on. The radio frequency circuit 804 can communicate with other terminals through at least one wireless communication protocol. The wireless communication protocol includes but is not limited to: metropolitan area network, various generations of mobile communication networks (2G, 3G, 4G, and 5G), wireless local area network and/or WiFi (Wireless Fidelity, wireless fidelity) network. In some embodiments, the radio frequency circuit 804 may also include NFC (Near Field Communication) related circuits, which is not limited in this application.

The display screen 805 is used to display UI (User Interface). The UI can include graphics, text, icons, videos, and any combination thereof. When the display screen 805 is a touch display screen, the display screen 805 also has the ability to collect touch signals on or above the surface of the display screen 805. The touch signal can be input to the processor 801 as a control signal for processing. At this time, the display screen 805 may also be used to provide virtual buttons and/or virtual keyboards, also called soft buttons and/or soft keyboards. In some embodiments, there may be one display screen 805, which is provided with the front panel of the car 800; in other embodiments, there may be at least two display screens 805, which are respectively arranged on different surfaces of the car 800 or in a folding design; In still other embodiments, the display screen 805 may be a flexible display screen, which is arranged on a curved surface or a folding surface of the automobile 800. Furthermore, the display screen 805 can also be set as a non-rectangular irregular pattern, that is, a special-shaped screen. The display screen 805 may be made of materials such as LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode, organic light emitting diode).

The camera assembly 806 is used to capture images or videos. Optionally, the camera assembly 806 includes a front camera and a rear camera. Generally, the front camera is set on the front panel of the terminal, and the rear camera is set on the back of the terminal. In some embodiments, there are at least two rear cameras, each of which is a main camera, a depth-of-field camera, a wide-angle camera, and a telephoto camera, so as to realize the fusion of the main camera and the depth-of-field camera to realize the background blur function, the main camera Integrate with the wide-angle camera to realize panoramic shooting and VR (Virtual Reality) shooting function or other fusion shooting functions.

The audio circuit 807 may include a microphone and a speaker. The microphone is used to collect sound waves of the user and the environment, and convert the sound waves into electrical signals and input them to the processor 801 for processing, or input to the radio frequency circuit 804 to implement voice communication. For the purpose of stereo collection or noise reduction, there may be multiple microphones, which are respectively set in different parts of the car 800. The microphone can also be an array microphone or an omnidirectional acquisition microphone. The speaker is used to convert the electrical signal from the processor 801 or the radio frequency circuit 804 into sound waves. The speaker can be a traditional thin-film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, not only can the electrical signal be converted into human audible sound waves, but also the electrical signal can be converted into human inaudible sound waves for distance measurement and other purposes. In some embodiments, the audio circuit 807 may also include a headphone jack.

The positioning component 808 is used to locate the current geographic location of the car 800 to implement navigation or LBS (Location Based Service, location-based service). The positioning component 808 may be a positioning component based on the GPS (Global Positioning System, Global Positioning System) of the United States, the Beidou system of China, the Granus system of Russia, or the Galileo system of the European Union.

The power supply 809 is used to supply power to various components in the car 800. The power source 809 may be alternating current, direct current, disposable batteries, or rechargeable batteries. When the power source 809 includes a rechargeable battery, the rechargeable battery may support wired charging or wireless charging. The rechargeable battery can also be used to support fast charging technology.

In some embodiments, the car 800 further includes one or more sensors 810.

That is, the embodiments of the present application not only provide a car, but also include a processor and a memory for storing executable instructions of the processor, where the processor is configured to execute the steps in the embodiments shown in FIGS. 1 and 2 In addition, the embodiment of the present application also provides a computer-readable storage medium in which a computer program is stored. When the computer program is executed by a processor, the computer program in the embodiment shown in FIG. 1 and FIG. The tracking method of the target object.

Those skilled in the art can understand that the structure shown in FIG. 8 does not constitute a limitation on the automobile 800, and may include more or fewer components than shown, or combine certain components, or adopt different component arrangements.

Those of ordinary skill in the art can understand that all or part of the steps in the foregoing embodiments can be implemented by hardware, or by a program instructing relevant hardware to be completed. The program can be stored in a computer-readable storage medium. The storage medium mentioned can be a read-only memory, a magnetic disk or an optical disk, etc.

The above descriptions are only preferred embodiments of this application and are not intended to limit this application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this application shall be included in the protection of this application. Within range.

Claims

A method for tracking a target object, characterized in that the method includes:

Obtain the radar information of the car in the driving direction through the millimeter wave radar installed on the car;

Based on the radar information, determine target objects that meet the tracking conditions;

The target object is tracked to control the car.
The method according to claim 1, wherein the determining the target object that meets the tracking condition based on the radar information comprises:

Based on the radar information, determine stationary objects, false objects, empty objects, and non-dangerous objects among the objects;

The object other than the prohibited object, the fake object, the empty object, and the non-hazardous object is determined as the target object that meets the tracking condition.
The method according to claim 2, wherein the radar information includes object information of objects appearing in the driving direction of the car, the horizontal distance and the longitudinal distance between the car and the object, and the The object information includes the relative distance between the car and the object, the relative speed, and the number of occurrences of the object within the detection range of the millimeter wave radar;

The determining of stationary objects, false objects, empty objects and non-dangerous objects in the objects based on the object information includes:

Determining an object whose absolute value of the relative speed is equal to the driving speed of the car among the objects as the stationary object;

Determining an object with a relative distance equal to 0 among the objects to the car as the empty object;

Determining an object with an appearance number greater than or less than an appearance frequency threshold among the objects as the fake object;

Among the objects, an object whose lateral distance is less than the lateral distance threshold and the longitudinal distance is less than the longitudinal distance threshold is determined as the non-dangerous object.
The method according to claim 1, wherein the acquiring radar information of the car in the driving direction through a millimeter wave radar installed on the car comprises:

Acquiring radar data in the driving direction through the millimeter wave radar;

Preprocessing the radar data to obtain the radar information.
The method of claim 4, wherein said preprocessing said radar data to obtain said radar information comprises:

Analyze the radar data of the object according to the millimeter wave radar protocol to obtain the object information of the object;

Use the position of the millimeter wave radar as the origin to establish a world coordinate system;

When the object information includes the relative distance between the object and the car and the relative angle between the object and the car, the relative distance is decomposed in the world coordinate system to obtain the lateral distance And the longitudinal distance.
The method of claim 1, wherein the tracking the target object to control the car comprises:

Projecting the target object from the world coordinate system to the image pixel coordinate system;

The target object is tracked in the image pixel coordinate system to control the automobile.
The method according to claim 6, wherein the projecting the target object from the world coordinate system to the image pixel coordinate system comprises:

According to the coordinate values of the target object in the world coordinate system, project the target object into the image pixel coordinate system by the following projection formula;

X 1 ＝X*a+image.cols,

Y 1 ＝Y*b

Wherein, X 1 is the abscissa of the target object in the image pixel coordinate system, the Y 1 is the ordinate of the target object in the image pixel coordinate system, and the image pixel coordinate system The median coordinate unit is pixel, the X is the lateral distance of the target object in the world coordinate system, the Y is the longitudinal distance of the target object in the world coordinate system, and the a and b Is the magnification, and the image.cols is the image width.
The method according to claim 6, wherein the tracking the target object in the image pixel coordinate system to control the car comprises:

Select the target object closest to the car;

Use the fourth-order Kalman filter algorithm to predict the selected target object to obtain the predicted position information;

Comparing the predicted position information with reference position information through a comparison formula, where the reference position information is the current position information of the selected target object;

When the predicted position information and the reference position information do not satisfy the comparison formula, and the number of times the comparison formula is not satisfied is greater than or equal to the inconsistency threshold, the reference position information is updated;

When the predicted position information and the reference position information satisfy the comparison formula, and the number of times the selected target object is selected is greater than or equal to the selection number threshold, it is determined that the selected target object collides with the car The duration of the collision.
A tracking device for a target object, characterized in that the device comprises:

The acquisition module is used to acquire radar information of the car in the driving direction through the millimeter wave radar installed on the car;

A determining module, configured to determine a target object that meets the tracking condition based on the radar information;

The tracking module is used to track the target object to control the car.
A computer-readable storage medium, characterized in that a computer program is stored in the storage medium, and when the computer program is executed by a processor, the method according to any one of claims 1-8 is implemented.