CN114608556A

CN114608556A - Data processing method and device, electronic equipment and storage medium

Info

Publication number: CN114608556A
Application number: CN202210199789.2A
Authority: CN
Inventors: 杨松铭; 刘德春; 林元则; 韦健林; 刘庆勃
Original assignee: Zhejiang Geely Holding Group Co Ltd; Zhejiang Geely New Energy Commercial Vehicle Group Co Ltd; Geely Sichuan Commercial Vehicle Co Ltd; Zhejiang Remote Commercial Vehicle R&D Co Ltd
Current assignee: Zhejiang Geely Holding Group Co Ltd; Zhejiang Geely New Energy Commercial Vehicle Group Co Ltd; Geely Sichuan Commercial Vehicle Co Ltd; Zhejiang Remote Commercial Vehicle R&D Co Ltd
Priority date: 2022-03-01
Filing date: 2022-03-01
Publication date: 2022-06-10

Abstract

The application provides a data processing method, a data processing device, electronic equipment and a storage medium, wherein target information around a vehicle is collected through a forward millimeter wave radar of the vehicle, environmental information around the vehicle is collected through a forward-looking camera of the vehicle, a first target is determined according to the target information collected by the forward millimeter wave radar, a second target is determined according to the first target and the environmental information collected by the forward-looking camera, and the second target is determined as a detection result of the forward millimeter wave radar to perform multi-sensor data fusion. The first target is filtered by combining the environmental information sensed by the forward-looking camera to obtain the second target, the number of the targets sensed by the forward millimeter wave radar is reduced, the second target is used as a detection result of the forward millimeter wave radar to perform multi-sensor data fusion, the possibility of multi-correlation and error correlation in the fusion process is reduced, the complexity and robustness of a fusion algorithm are improved, the algorithm performance is improved, the requirement on hardware performance is reduced, and the method has great practical application value.

Description

Data processing method and device, electronic equipment and storage medium

Technical Field

The present application relates to the field of intelligent driving technologies, and in particular, to a data processing method and apparatus, an electronic device, and a storage medium.

Background

With the development of automotive electronics and intelligence, various sensor technologies are widely used. In the field of automobile safety, a millimeter wave radar is an important sensor for sensing surrounding targets, and is mainly applied to the active safety fields of automobile blind area monitoring, automatic emergency braking, adaptive cruise, forward collision early warning and the like. However, in a real environment, a large amount of noise is often accompanied by millimeter wave signals, so that the millimeter wave radar can receive not only reflected signals from a target, but also reflected signals from the environment and from an unnecessary target, and a target detection result of the millimeter wave radar has a high false detection rate.

Meanwhile, the sensing accuracy is improved by a multi-sensor fusion method in the current fields of assistant driving and automatic driving, so that the final targets of intelligent driving and safe driving are achieved through self-checking and self-learning of an assistant driving system. Thus, in the multi-sensor fusion process, the number of targets sensed by each sensor directly determines the complexity and performance of the fusion algorithm. More targets means more chip processing power and memory requirements, and also results in an increased likelihood of multiple and false associations during the fusion process.

Therefore, a solution is needed to process the target detection result required by the multi-sensor fusion process to overcome the problem caused by too many input targets of the fusion algorithm.

Disclosure of Invention

The application provides a data processing method, a data processing device, electronic equipment and a storage medium, and aims to provide a data processing method for reducing targets detected by a millimeter wave radar and solving the problem caused by too many input targets in a multi-sensor fusion process.

In a first aspect, the present application provides a data processing method, including:

acquiring target information around a vehicle through a forward millimeter wave radar of the vehicle, and acquiring environment information around the vehicle through a forward-looking camera of the vehicle;

determining a first target according to target information acquired by the forward millimeter wave radar;

and determining a second target according to the first target and the environmental information acquired by the forward-looking camera, and determining the second target as a detection result of the forward millimeter wave radar to perform multi-sensor data fusion.

In one possible design, the target information collected by the forward millimeter wave radar includes a speed, a lateral distance, and a longitudinal distance of each target;

the environmental information collected by the front-view camera comprises lane line data, and the lane line data comprises left lane line data and right lane line data.

In one possible design, the determining a second target based on the first target and the environmental information collected by the forward-looking camera includes:

judging whether the forward-looking camera shoots a lane line image or not according to the environment information;

if so, mapping the first target to a lane corresponding to the lane image according to lane line data in the environment information, and determining the second target according to a mapping result;

if not, mapping the first target to a virtual straight lane according to lane line data in the environment information, and determining the second target according to a mapping result, wherein the virtual straight lane is determined according to a motor vehicle lane width standard.

In one possible design, the determining the second target according to the mapping result includes:

acquiring a first preset number of first targets which are mapped to a lane where the vehicle is located and are closest to the vehicle;

acquiring a second preset number of first targets which are mapped into a left lane of the lane and are closest to the vehicle;

acquiring a third preset number of first targets which are mapped into a right lane of the lane and are closest to the vehicle;

and determining the first targets with the first preset number, the second first targets with the second preset number and the third first targets with the third preset number as the second targets.

In one possible design, the determining a first target according to the target information collected by the forward millimeter wave radar includes:

and clustering the speed, the transverse distance and the longitudinal distance of each target to determine the first target.

In one possible design, the clustering the speed, the lateral distance, and the longitudinal distance of each target to determine the first target includes:

sequencing all targets from small to large according to respective speeds to obtain a first column, and performing first clustering processing on the first target in the first column based on a speed difference value to obtain a first clustering result;

performing second clustering processing according to the first clustering result and the transverse distance of the target corresponding to the first clustering result to obtain a second clustering result;

performing second clustering treatment according to the second clustering result and the longitudinal distance of the target corresponding to the second clustering result to obtain a third clustering result;

determining a target of a first column of each row in the sequence representing the third cluster result as the first target.

In a possible design, the performing a first clustering process based on a speed difference value from a first target in the first column to obtain a first clustering result includes:

acquiring a speed difference value between respective speeds of a first target and an adjacent next target in the first column;

judging whether the speed difference value is smaller than or equal to a preset speed difference value or not;

if yes, determining that the next target and the first target in the first column are first approximate targets, and arranging the first approximate targets side by side;

if not, determining that the next target and the first target in the first column are first non-approximate targets, and arranging the first non-approximate targets in rows;

continuously judging whether the speed difference value between the respective speeds of the next target and the next target in the first column is less than or equal to the preset speed difference value or not;

and repeating the steps according to the judgment result until the comparison between the speed difference value corresponding to the last target in the first column and the preset speed difference value is completed, and determining the target in the first sequence obtained after the comparison and the arrangement as the first clustering result.

In a possible design, the performing, according to the first clustering result and the lateral distance of the target corresponding to the first clustering result, a second clustering process to obtain a second clustering result includes:

sequencing the targets of each row in the first sequence from small arrival according to respective transverse distance to obtain a second column corresponding to each row in the first sequence;

for each second column, acquiring a transverse distance difference between respective transverse distances of a first target and a next target in the second column;

judging whether the transverse distance difference is smaller than or equal to a preset transverse distance difference or not;

if yes, determining that the next target and the first target in the second column are second approximate targets, wherein the second approximate targets are arranged side by side;

if not, determining a second non-approximate target of the next target and the first target in the second column, wherein the second non-approximate targets are arranged in rows;

continuously judging whether the transverse distance difference value between the respective transverse distances of the next target and the next target in the second column is smaller than or equal to the preset transverse distance difference value;

and repeating the steps according to the judgment result until the comparison between the transverse distance difference value corresponding to the last target in each second column and the preset transverse distance difference value is completed, and determining the target in the second sequence obtained after the comparison and the arrangement as the second clustering result.

In a possible design, the performing the second clustering process according to the second clustering result and the longitudinal distance of the target corresponding to the second clustering result to obtain a third clustering result includes:

sequencing the targets of each row in the second sequence from small arrival according to respective longitudinal distance to obtain a third column corresponding to each row in the second sequence;

for each third column, acquiring a longitudinal distance difference between respective longitudinal distances of a first target and a next target in the third column;

judging whether the longitudinal distance difference is smaller than or equal to a preset longitudinal distance difference or not;

if yes, determining that the next target and the first target in the third column are the same targets, and arranging the same targets side by side;

if not, determining that the next target and the first target in the third column are different targets, and arranging the different targets in rows;

continuously judging whether the longitudinal distance difference value between the respective longitudinal distances of the next target and the next target in the third column is less than or equal to the preset longitudinal distance difference value or not;

and repeating the steps according to the judgment result until the comparison between the longitudinal distance difference value corresponding to the last target in each third column and the preset longitudinal distance difference value is completed, and determining the target in the sequence obtained after the comparison and arrangement is finished as the third clustering result.

In a second aspect, the present application provides a data processing apparatus comprising:

the acquisition module is used for acquiring target information around the vehicle through a forward millimeter wave radar of the vehicle and acquiring environment information around the vehicle through a forward-looking camera of the vehicle;

the first processing module is used for determining a first target according to the target information acquired by the forward millimeter wave radar;

and the second processing module is used for determining a second target according to the first target and the environmental information acquired by the forward-looking camera, and determining the second target as a detection result of the forward millimeter wave radar to perform multi-sensor data fusion.

In a third aspect, the present application provides an electronic device, comprising: a processor, and a memory communicatively coupled to the processor;

the memory stores computer-executable instructions;

the processor executes computer-executable instructions stored by the memory to implement any one of the possible data processing methods as provided by the first aspect.

In a fourth aspect, the present application provides a computer-readable storage medium having stored therein computer-executable instructions for implementing any one of the possible data processing methods as provided in the first aspect when executed by a processor.

In a fifth aspect, the present application provides a computer program product comprising computer executable instructions for implementing any one of the possible data processing methods provided in the first aspect when executed by a processor.

The application provides a data processing method, a data processing device, electronic equipment and a storage medium, wherein target information around a vehicle is collected through a forward millimeter wave radar of the vehicle, environmental information around the vehicle is collected through a forward-looking camera of the vehicle, a first target is determined according to the target information collected by the forward millimeter wave radar, a second target is determined according to the first target and the environmental information collected by the forward-looking camera, and the obtained second target is determined as a detection result of the forward millimeter wave radar so as to perform multi-sensor data fusion. The first target determined according to the target information sensed by the forward millimeter wave radar is filtered by combining the environmental information sensed by the forward looking camera to obtain the second target, the number of the targets sensed by the forward millimeter wave radar is reduced, the second target is used as the detection result of the forward millimeter wave radar to perform multi-sensor data fusion, the possibility of multi-correlation and error correlation in the fusion process can be reduced, the complexity and robustness of the fusion algorithm are improved, the algorithm performance is improved, the requirement on the hardware performance is reduced, and the method has great practical application value.

Drawings

Fig. 1 is a schematic view of an application scenario provided in an embodiment of the present application;

fig. 2 is a schematic flowchart of a data processing method according to an embodiment of the present application;

fig. 3 is a schematic flow chart of another data processing method according to an embodiment of the present application;

fig. 4 is a schematic flowchart of another data processing method according to an embodiment of the present application;

fig. 5 is a schematic flowchart of another data processing method according to an embodiment of the present application;

fig. 6 is a schematic flowchart of another data processing method according to an embodiment of the present application;

fig. 7 is a schematic flowchart of another data processing method according to an embodiment of the present application;

fig. 8 is a schematic structural diagram of a data processing apparatus according to an embodiment of the present application;

fig. 9 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

Detailed Description

Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. The following description refers to the accompanying drawings in which the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Rather, they are merely examples of methods and apparatus consistent with certain aspects of the present application, as detailed in the appended claims.

The terms "first," "second," "third," "fourth," and the like in the description and in the claims of the present application and in the above-described drawings (if any) are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the application described herein are, for example, capable of operation in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

In the field of automobile safety, a millimeter wave radar is an important sensor for sensing surrounding targets, and is mainly applied to the active safety fields of automobile blind area monitoring, automatic emergency braking, adaptive cruise, forward collision early warning and the like. However, in a real environment, a large amount of noise is often accompanied by millimeter wave signals, so that the millimeter wave radar can receive not only reflected signals from a target, but also reflected signals from the environment and interferents, and a target detection result of the millimeter wave radar has a high false detection rate. Meanwhile, the sensing accuracy is improved by adopting a multi-sensor fusion method in the current fields of assistant driving and automatic driving, so that the final aims of intelligent driving and safe driving are achieved through self-checking and self-learning of an assistant driving system. Thus, in a multi-sensor fusion process, the number of targets sensed by the sensors directly determines the complexity and performance of the fusion algorithm. More targets means more chip processing power and memory requirements, and also results in an increased likelihood of multiple and false associations during the fusion process. Therefore, a solution is needed to process the target detection result required by the multi-sensor fusion process to overcome the problem caused by too many input targets of the fusion algorithm.

Aiming at the problems in the prior art, the data processing method provided by the application aims at filtering the targets sensed by the millimeter wave radar, so that the number of the targets detected by the millimeter wave radar is reduced, and the problem caused by too many targets output by a fusion algorithm is solved. The inventive concept of the data processing method provided by the application is as follows: the method comprises the steps of combining a perception result of a forward-looking camera, namely environment information around a vehicle collected by the forward-looking camera, processing a first target determined by the forward-looking millimeter wave radar according to the collected target information to determine a second target, achieving the purpose of filtering the first target to obtain the second target, and inputting the second target obtained after filtering into a fusion algorithm of a multi-sensor fusion process as a detection result of the forward-looking millimeter wave radar, so that the number of targets perceived by the forward-looking millimeter wave radar is reduced, the possibility of multi-correlation and false correlation is reduced, the complexity and robustness of the fusion algorithm are improved, the algorithm performance is improved, the requirement on hardware performance is reduced, and the method has great practical application value.

An exemplary application scenario of the embodiments of the present application is described below.

Fig. 1 is a schematic view of an application scenario provided in an embodiment of the present application, and as shown in fig. 1, a forward Driver Assistance System (Advanced driving Assistance System) function of a vehicle 10 is implemented by using multi-sensor fusion to improve a perception accuracy, so as to achieve the purposes of intelligent driving and safe driving. The millimeter wave radar is an important sensor, but because a millimeter wave signal in a real environment often accompanies a large amount of noise, a target detection result of the millimeter wave radar has a high false detection rate, and the millimeter wave radar is, for example, a forward millimeter wave radar provided on the vehicle 10.

The electronic device 20 is configured to execute the data processing method provided in the embodiment of the present application, and in combination with the environmental information around the vehicle 10 collected by the forward-looking camera arranged in the vehicle 10, the forward millimeter wave radar arranged in the vehicle 10 filters a first target determined according to the target information collected by the forward-looking millimeter wave radar to obtain a second target, and the second target obtained after filtering is used as a detection result of the forward-looking millimeter wave radar and is input to a fusion algorithm in a multi-sensor fusion process, so that the number of targets sensed by the forward-looking millimeter wave radar is reduced.

It should be noted that the electronic device 20 may be a computer, a server, or a server cluster, and the embodiment of the present application is not limited thereto. The electronic device 20 in fig. 1 is illustrated as a computer.

It should be noted that the above application scenarios are only exemplary, and the data processing method, apparatus, device, and storage medium provided in the embodiments of the present application include, but are not limited to, the above application scenarios.

Fig. 2 is a schematic flowchart of a data processing method according to an embodiment of the present disclosure. As shown in fig. 2, a data processing method provided in an embodiment of the present application includes:

s101: the method comprises the steps of collecting target information around a vehicle through a forward millimeter wave radar of the vehicle, and collecting environment information around the vehicle through a forward-looking camera of the vehicle.

The vehicle is provided with a forward millimeter wave radar and a forward-looking camera, and after the whole vehicle is electrified, the forward millimeter wave radar can be used for collecting target information around the vehicle, and the forward-looking camera can be used for collecting environment information around the vehicle.

In one possible design, the target information collected by the forward millimeter wave radar may include, but is not limited to, the speed of each target detected by the forward millimeter wave radar, the lateral distance of each target, the longitudinal distance of each target, and the like.

The environmental information collected by the forward-looking camera may include, but is not limited to, lane line data, for example, the lane line data may include left lane line data and right lane line data. The left lane line data is data representing a lane line corresponding to a left lane adjacent to the vehicle. Accordingly, the right lane line data refers to data representing a right lane corresponding to a lane line adjacent to the vehicle. When the vehicle is located on a multi-lane road, the left lane line data may include data of a lane line of a next left lane to the left of the left lane adjacent to the vehicle, i.e., left lane line data. The right lane line data may further include data of a lane line of a next left lane to the right of the right lane adjacent to the vehicle, that is, right and right lane line data.

The lane line data may be represented using a lane line equation, such as may be represented as (c)₀,c₁,c₂,c₃) Wherein c is₀Representing the lateral distance of the vehicle from the lane line represented by the lane line equation, c₁Is the heading angle of the vehicle, c₂Is the curvature of the lane line, c₃Is the rate of change of the curvature. The lane line data may characterize the real environment of the vehicle travel process, such as the lane, the travelable area, and so on.

S102: and determining a first target according to the target information acquired by the forward millimeter wave radar.

And determining a plurality of targets corresponding to the target information, namely a first target, based on the target information acquired by the forward millimeter wave radar.

In one possible design, the speed of each target, the lateral distance of each target, and the longitudinal distance of each target may be clustered to obtain each target, i.e., the first target.

For example, in the clustering process, clustering may be performed first based on the speed of each target, clustering may be performed on the obtained corresponding clustering result based on the lateral distance of each target, clustering may be further performed on the obtained corresponding clustering result based on the longitudinal distance of each target, and finally each target corresponding to the target information, that is, the first target, may be determined.

Since the millimeter wave signal is often accompanied by a large amount of noise, a certain false detection rate may exist in the first target determined according to the target information acquired by the forward millimeter wave radar. In view of this, the first target may be filtered in combination with the environmental information collected by the forward-looking camera, so as to reduce the number of targets detected by the forward millimeter wave radar.

S103: and determining a second target according to the first target and the environment information acquired by the forward-looking camera, and determining the second target as a detection result of the forward millimeter wave radar to perform data fusion of the multiple sensors.

And filtering the first target by combining the environmental information acquired by the forward-looking camera on the basis of acquiring the first target to obtain a second target. The number of the first targets can be greatly reduced in the filtering process, the number of targets sensed by the forward millimeter wave radar is reduced by combining a real application scene of a vehicle, and the virtual detection rate of the forward millimeter wave radar is reduced. And then the second target is determined as the detection result of the forward millimeter wave radar and is input into the fusion algorithm of the multi-sensor for multi-sensor data fusion, so that the possibility of multi-correlation and error correlation caused by more input targets is reduced, the complexity and robustness of the fusion algorithm are improved, the algorithm performance is improved, the requirement on the hardware performance is reduced, and the method has great practical application value.

In addition, the data processing method provided by the embodiment of the application acquires target information and environmental information around a vehicle respectively based on the forward millimeter wave radar and the forward-looking camera, determines a first target according to the target information acquired by the forward millimeter wave radar through clustering, and filters the first target by combining the environmental information acquired by the forward-looking camera to obtain a second target. For the vehicle with the driving assisting function, a hardware structure is not required to be added, so that the hardware cost and the mechanical complexity are not increased. And the test verification cost in the test stage is low, and the method can be realized only by changing and verifying the program on the software level on the basis of the existing scheme, and has wider use value.

In one possible design, the possible implementation manner of determining the second target in step S103 in combination with the environmental information collected by the front-view camera is shown in fig. 3. Fig. 3 is a schematic flow chart of another data processing method according to an embodiment of the present application. As shown in fig. 3, the embodiment of the present application includes:

s1031: and judging whether the forward-looking camera shoots the lane line image or not according to the environment information.

And judging whether the forward-looking camera shoots the lane line image or not in the environment information collected by the forward-looking camera. It can be understood that if a lane line is drawn on a road where a vehicle is located, a lane line image captured by the forward-looking camera is inevitably included in the environmental information collected by the forward-looking camera, and conversely, if a lane line is not drawn on the road, a lane line image captured by the forward-looking camera is inevitably absent in the environmental information collected by the forward-looking camera. In other words, the purpose of this step is to determine whether a lane line is drawn on the road on which the vehicle is located, based on the environmental information. If yes, step S1032 is executed, otherwise, if no, step S1033 is executed.

S1032: if so, mapping the first target to a lane corresponding to the lane image according to the lane data in the environment information, and determining the second target according to the mapping result.

If the forward-looking camera shoots the lane line image, it is indicated that a lane line is drawn on a road on which the vehicle runs, and the first target is mapped to a lane corresponding to the lane line image according to the lane line data in the environment information. The first target is a plurality of targets obtained through clustering, and the first target is a general name of the plurality of targets, and the number of the first targets is a plurality of targets.

When mapping the first target into the corresponding lane, for example, the relationship between the lateral distance of the target and the lane line data may be used to determine the relationship between the first target and the lane, for example, some targets in the plurality of targets may be in the own lane where the vehicle is located, or may be in the left lane or the right lane of the vehicle. The manner used for determining the positional relationship between the multiple targets and the vehicle's own lane, left lane, right lane, or even the left lane of the left lane, i.e., left lane, and the right lane of the right lane, i.e., right lane, is the mapping, and the embodiment of the present application is not limited to the specific manner used for implementing the mapping.

The mapping result is obtained through mapping, and the mapping result is the lane where the multiple targets, that is, the first target, are located, some targets may be in the own lane where the vehicle is located, and some targets are in the left lane, the right lane, the left lane, the right lane and the like of the vehicle. And determining a second target from the first targets according to the mapping result.

S1033: if not, the first target is mapped to the virtual straight lane according to the lane line data in the environment information, and the second target is determined according to the mapping result.

And if the forward-looking camera does not shoot the lane line image, which indicates that the lane line is not drawn on the driving road of the vehicle, mapping the first target to the virtual straight lane according to the lane line data in the environment information. As described above, the first target is a plurality of targets obtained by clustering, and the first target is a generic name of the plurality of targets, and the number of the first targets is plural.

This step differs from step S1032 in that the lane to which the first target is to be mapped is a virtual straight lane, which may be determined according to a motor vehicle lane width criterion. For a virtual straight lane, the vehicle heading angle, the curvature of the lane line, and the rate of change of curvature, i.e., c, in the lane line data₁、c₂And c₃Each value is 0, namely c₁＝c₂＝c₃0. Therefore, when the first target is mapped into the virtual straight lane according to the lane line data in the environment information, only the transverse distance, namely c, from the vehicle to the lane line of the virtual straight lane is used₀The transverse distance between the target and the target can be determinedAnd determining the position relation between the targets and the virtual straight lane so as to determine the position relation between the targets and the vehicle lane, the left lane, the right lane and even the left lane of the left lane, namely the left lane, and the right lane of the right lane, namely the right lane.

And obtaining a mapping result through mapping, wherein the mapping result is the lane where the multiple targets, namely the first target, are located, some targets may be in the lane where the vehicle is located, and some targets are in the left lane, the right lane, the left lane, the right lane and the like of the vehicle, and then determining a second target from the first target according to the mapping result.

It can be understood that the local lane, the left lane, the right lane, and even the left lane and the right lane in the step are all virtual straight lanes determined according to the lane width standard of the motor vehicle. In addition, the embodiment of the present application does not limit the specific manner used for implementing the mapping in this step.

In addition, the virtual straight lane may be determined according to a motor vehicle lane width standard, for example, the lane width standard of each motor vehicle of the urban road is 3.5 meters, and the virtual straight lane lines may be drawn such that the width between adjacent drawn virtual straight lane lines is 3.5 meters. It should be noted that the lane width standards of motor vehicles on different roads may be different, and the specific values of the lane width standards of motor vehicles are not limited in the embodiment of the present application.

In one possible design, whether or not the forward-looking camera captures an image of the lane line, determining a possible implementation of the second objective based on the mapping result includes:

the method comprises the steps of obtaining a first preset number of first targets which are mapped in a lane where a vehicle is located and are closest to the vehicle, obtaining a second preset number of first targets which are mapped in a left lane of the lane and are closest to the vehicle, obtaining a third preset number of first targets which are mapped in a right lane of the lane and are closest to the vehicle, and determining the obtained first preset number of first targets, the second preset number of first targets and the third preset number of first targets as second targets, so that the purpose of filtering the first targets by combining a real application scene of the vehicle to obtain the second targets is achieved. The specific values of the first preset number, the second preset number and the third preset number may be set according to actual conditions, and the three values may be the same or different, for example, may be one or two or zero, and the embodiment of the present application does not limit this.

For example, two first targets which are mapped to the lane where the vehicle is located and are the most nearest to the vehicle in the left lane and the right lane are respectively obtained, two first targets in the three lanes are determined as second targets, and the obtained second targets are determined as detection results of the forward millimeter wave radar, so that the number of targets sensed by the forward millimeter wave radar is greatly reduced.

According to the data processing method provided by the embodiment of the application, after the first target is obtained, the first target determined according to the target information sensed by the forward millimeter wave radar is filtered by combining the environment information sensed by the forward-looking camera, the second target is obtained, the number of the targets sensed by the forward millimeter wave radar is reduced, the second target is used as the detection result of the forward millimeter wave radar to perform multi-sensor data fusion, the possibility of multi-correlation and error correlation in the fusion process can be reduced, the complexity and robustness of a fusion algorithm are improved, the algorithm performance is improved, the requirement on the hardware performance is reduced, and the method has great practical application value.

On the basis of the above embodiment, a possible implementation manner of determining the first target by clustering the speed, the lateral distance, and the longitudinal distance of each target is shown in fig. 4. Fig. 4 is a schematic flowchart of another data processing method according to an embodiment of the present application. As shown in fig. 4, the embodiment of the present application includes:

s201: and sequencing all the targets from small to large according to respective speeds to obtain a first column, and performing first clustering processing on the first target in the first column based on a speed difference value to obtain a first clustering result.

For the speed of each target acquired by the forward millimeter wave radar, all the targets are sorted from small to large according to the respective speed to obtain a column, and the column is defined as a first column, for example, if the forward millimeter wave radar acquires the respective speeds of M targets, the first column has M rows, the first row is the target with the minimum speed, and the mth row is the target with the maximum speed. And then, carrying out first clustering processing on the first target in the first column based on the speed difference to obtain a first clustering result. The first clustering result is a corresponding target obtained by clustering based on the speed of each target.

S202: and performing second clustering processing according to the first clustering result and the transverse distance of the target corresponding to the first clustering result to obtain a second clustering result.

S203: and performing second clustering treatment according to the second clustering result and the longitudinal distance of the target corresponding to the second clustering result to obtain a third clustering result.

And on the basis of the first clustering result, continuing to perform second clustering processing based on the transverse distance of the target corresponding to the first clustering result to obtain a second clustering result. And further performing third clustering processing based on the longitudinal distance of the target corresponding to the second clustering result on the basis of the second clustering result to obtain a third clustering processing result.

S204: the target of the first column of each row in the sequence representing the third cluster result is determined as the first target.

Because the first clustering processing is performed based on the first column, the clustering result obtained by each clustering processing represents the target obtained after clustering in a sequence mode. Therefore, after the third clustering result is obtained, the objects in the first column of each line in the sequence representing the third clustering result are determined as the first objects obtained by clustering the speed, the horizontal distance, and the vertical distance of each object represented by the object information. The first target is a generic name of targets obtained by clustering, and the number of the first targets is multiple.

The data processing method provided by the embodiment of the application performs clustering processing on the speed of each target, the transverse distance of each target and the longitudinal distance of each target included in target information acquired by the forward millimeter wave radar so as to determine the target corresponding to the target information, namely the first target. The clustering process is to cluster target information representing the same target to determine the same target represented by the target information, and the targets determined by the clustering are collectively referred to as a first target.

In one possible design, a possible implementation of step S201 is shown in fig. 5. Fig. 5 is a schematic flowchart of another data processing method according to an embodiment of the present application. As shown in fig. 5, the embodiment of the present application includes:

s2011: and acquiring the speed difference between the respective speeds of the first target and the next adjacent target in the first column.

And acquiring a speed difference value, namely a speed difference value, between the first target in the first column and the next adjacent target.

S2012: and judging whether the speed difference value is smaller than or equal to a preset speed difference value.

And comparing the obtained speed difference value with a preset speed difference value, and judging whether the speed difference value is smaller than or equal to the preset speed difference value, wherein the specific value of the preset speed difference value is set according to the actual working condition.

S2013: if yes, determining that the next target and the first target in the first column are first approximate targets.

Wherein the first approximation targets are arranged side by side.

S2014: if not, determining that the next target and the first target in the first column are the first non-approximate target.

Wherein the first non-approximate targets are arranged in rows.

After comparison, if the speed difference is less than or equal to the preset speed difference, it may be temporarily determined that the next target and the first target may be the same target, that is, the next target and the first target in the first column are defined as first approximate targets, and the next target and the first target are arranged side by side. If the first column is a sequence of M rows and 1 columns, the first column is a sequence of M-1 rows and 2 columns after the step.

On the contrary, if the speed difference is greater than the preset speed difference, the next target and the first target are not the same target, that is, the next target and the first target in the first column are defined as first non-approximate targets, and the current arrangement of the next target and the first target in the first column is maintained, that is, the first non-approximate targets are arranged in rows.

S2015: and continuously judging whether the speed difference value between the respective speeds of the next target and the next target in the first column is smaller than or equal to the preset speed difference value.

S2016: and repeating the steps according to the judgment result until the comparison between the speed difference value corresponding to the last target in the first column and the preset speed difference value is completed, and determining the target in the first sequence obtained after the comparison and the arrangement as a first clustering result.

Starting from the first target in the first column, the steps S2011 to S2014 are performed one by one, that is, after the first target in the first column and the next target of the first target are determined and arranged, whether the speed difference between the next target and the next target in the first column is less than or equal to the preset speed difference is continuously determined, if yes, the next target and the next target are first approximate targets, and the targets can be arranged side by side. If not, the next target and the next target are the first non-approximate target and are arranged in rows. The steps are repeated until the comparison between the speed difference value between the last target in the first column and a target before the last target and the preset speed difference value is completed, and the sequence obtained after the comparison and the arrangement is finished, namely the target in the first sequence is determined as a first clustering result.

According to the data processing method provided by the embodiment of the application, the first clustering processing is carried out based on the speed difference value between the speeds of the targets in the target information to obtain the first clustering result, the clustering is completed based on the speed of each target collected by the forward millimeter wave radar, and the first clustering result is represented by the first sequence so as to further carry out the second clustering processing on the basis of the first sequence.

In one possible design, a possible implementation of step S202 is shown in fig. 6. Fig. 6 is a schematic flowchart of another data processing method according to an embodiment of the present application. As shown in fig. 6, the embodiment of the present application includes:

s2021: and sequencing the targets of each row in the first sequence from small arrival according to the respective transverse distance to obtain a second column corresponding to each row in the first sequence.

And aiming at the targets of each row in the first sequence, sequencing the targets from small to large according to the respective transverse distances of the targets, so as to obtain a second column corresponding to each row in the first sequence, namely, each row corresponds to one second column. The number of rows in the second column is the number of objects in the row in the first sequence that results in the second column.

S2022: and acquiring the transverse distance difference between the respective transverse distances of the first target and the next target in each second column.

S2023: and judging whether the transverse distance difference is smaller than or equal to a preset transverse distance difference.

S2024: if yes, determining that the next target and the first target in the second column are second approximate targets.

Wherein the second approximation targets are arranged side by side.

S2025: if not, determining that the next target and the first target in the second column are second non-approximate targets.

Wherein the second non-approximate targets are arranged in rows.

S2026: and continuously judging whether the transverse distance difference between the respective transverse distances of the next target and the next target in the second column is smaller than or equal to the preset transverse distance difference.

S2027: and repeating the steps according to the judgment result until the comparison between the transverse distance difference value corresponding to the last target in each second column and the preset transverse distance difference value is completed, and determining the targets in the second sequence obtained after the comparison and the arrangement are finished as a second clustering result.

The implementation manner of performing the second clustering process based on the lateral distance in the above steps is similar to the implementation manner of performing the first clustering process based on the velocity in the embodiment shown in fig. 5, except that the number of the second columns in the embodiment of the present application is multiple, and the second clustering process is performed for each second column by using the comparison and arrangement similar to those shown in fig. 5.

For example, for each second column, if the difference in lateral distance between two targets in the second column is less than or equal to the preset lateral distance difference, the two targets are considered as second approximate targets and are arranged side by side. On the contrary, if the difference value of the transverse distances is larger than the preset difference value of the transverse distances, the two targets are not considered to be the same target, namely the two targets are second non-approximate targets and are arranged in rows. And for each second column, comparing the transverse distance difference value corresponding to the last target from the first target in the second column to the preset transverse distance difference value, determining the sequence spliced by the comparison end and the sequence corresponding to the arranged second columns as a second sequence, and determining the target in the second sequence as a second clustering result.

According to the data processing method provided by the embodiment of the application, on the basis of a first clustering result obtained by clustering speed, second clustering processing is carried out on the basis of a transverse distance difference value between transverse distances of targets in target information to obtain a second clustering result, clustering is completed on the basis of the transverse distances of the targets acquired by the forward millimeter wave radar, and the second clustering result is expressed by a second sequence so as to be convenient for further carrying out third clustering processing on the basis of the second sequence.

In one possible design, a possible implementation of step S203 is shown in fig. 7. Fig. 7 is a schematic flowchart of another data processing method according to an embodiment of the present application. As shown in fig. 7, the embodiment of the present application includes:

s2031: and sequencing the targets of each row in the second sequence from small arrival according to the respective longitudinal distance to obtain a third column corresponding to each row in the second sequence.

And aiming at the targets of each row in the second sequence, sorting the targets from small to large according to the respective longitudinal distances of the targets, so as to obtain a third column corresponding to each row in the second sequence, namely, each row corresponds to one third column. The number of rows in the third column is the number of objects in the row in the second sequence from which the third column is derived.

S2032: and acquiring the longitudinal distance difference between the respective longitudinal distances of the first target and the next target in the third column aiming at each third column.

S2033: and judging whether the longitudinal distance difference is smaller than or equal to a preset longitudinal distance difference.

S2034: if yes, determining that the next target and the first target in the third column are the same target.

Wherein the same targets are arranged side by side;

s2035: if not, determining that the next target in the third column is different from the first target.

Wherein the different targets are arranged in rows.

S2036: and continuously judging whether the longitudinal distance difference between the respective longitudinal distances of the next target and the next target in the third column is smaller than or equal to the preset longitudinal distance difference.

S2037: and repeating the steps according to the judgment result until the comparison between the longitudinal distance difference value corresponding to the last target in each third column and the preset longitudinal distance difference value is completed, and determining the targets in the sequence obtained after the comparison and the arrangement are finished as a third clustering result.

The implementation manner of performing the third clustering based on the longitudinal distance in the above steps is similar to the implementation manner of performing the first clustering based on the speed in the embodiment shown in fig. 5, except that the number of the third columns in the embodiment of the present application is multiple, and the third clustering is performed for each third column by using the comparison and arrangement similar to those shown in fig. 5.

For example, for each third column, if the difference in longitudinal distance between the longitudinal distances between two targets in the third column is less than or equal to the preset longitudinal distance difference, the two targets are considered as the same target and are arranged side by side. And otherwise, if the longitudinal distance difference is larger than the preset longitudinal distance difference, the two targets are considered as different targets and are arranged in rows. And for each third column, comparing the longitudinal distance difference value corresponding to the last target from the first target in the third column to the preset longitudinal distance difference value, determining the sequence spliced by the comparison end and the sequence corresponding to each arranged third column as a third sequence, and determining the target in the third sequence as a third clustering result.

According to the data processing method provided by the embodiment of the application, on the basis of a second clustering result obtained by clustering transverse distances, third clustering processing is carried out based on a longitudinal distance difference value between longitudinal distances of targets in target information to obtain a third clustering result, clustering is completed based on the longitudinal distances of the targets collected by the forward millimeter wave radar, the third clustering result is expressed by a third sequence, and therefore targets in a first column of each line in the third sequence are determined to be first targets, clustering processing is completed, and the first targets are obtained.

Fig. 8 is a schematic structural diagram of a data processing apparatus according to an embodiment of the present application. As shown in fig. 8, a data processing apparatus 400 provided in an embodiment of the present application includes:

the acquisition module 401 is used for acquiring target information around the vehicle through a forward millimeter wave radar of the vehicle and acquiring environment information around the vehicle through a forward-looking camera of the vehicle;

the first processing module 402 is configured to determine a first target according to target information acquired by the forward millimeter wave radar;

the second processing module 403 is configured to determine a second target according to the first target and the environment information acquired by the forward-looking camera, and determine the second target as a detection result of the forward millimeter wave radar to perform data fusion of multiple sensors.

In one possible design, the target information collected by the forward millimeter wave radar includes the speed, the lateral distance, and the longitudinal distance of each target;

In one possible design, the second processing module 403 is specifically configured to:

if so, mapping the first target to a lane corresponding to the lane image according to lane data in the environment information, and determining a second target according to a mapping result;

if not, the first target is mapped to the virtual straight lane according to lane line data in the environment information, the second target is determined according to the mapping result, and the virtual straight lane is determined according to the motor vehicle lane width standard.

In one possible design, the second processing module 403 is further specifically configured to:

acquiring a first preset number of first targets which are mapped to a road where a vehicle is located and are closest to the vehicle;

and determining a first preset number of first targets, a second preset number of first targets and a third preset number of first targets as second targets. In one possible design, the first processing module 402 includes:

and the clustering and processing module is used for clustering the speed, the transverse distance and the longitudinal distance of each target to determine a first target.

In one possible design, the clustering and processing module includes:

the first clustering module is used for sequencing all targets from small to large according to respective speeds to obtain a first column, and performing first clustering processing on the first target in the first column based on a speed difference value to obtain a first clustering result;

the second clustering module is used for carrying out second clustering processing according to the first clustering result and the transverse distance of the target corresponding to the first clustering result to obtain a second clustering result;

the third clustering module is used for carrying out second clustering treatment according to the second clustering result and the longitudinal distance of the target corresponding to the second clustering result to obtain a third clustering result;

and the processing submodule is used for determining the target of the first column of each row in the sequence representing the third clustering result as the first target.

In one possible design, the first clustering module is specifically configured to:

acquiring a speed difference value between respective speeds of a first target and an adjacent next target in a first column;

if yes, determining that the next target and the first target in the first column are first approximate targets which are arranged side by side;

if not, determining that the next target and the first target in the first column are first non-approximate targets which are arranged in rows;

continuously judging whether the speed difference value between the respective speeds of the next target and the next target in the first column is less than or equal to a preset speed difference value or not;

and repeating the steps according to the judgment result until the comparison between the speed difference value corresponding to the last target in the first column and the preset speed difference value is completed, and determining the target in the first sequence obtained after the comparison and the arrangement as a first clustering result.

In one possible design, the second clustering module is specifically configured to:

sequencing the targets of each row in the first sequence from small arrival according to the respective transverse distance to obtain a second column corresponding to each row in the first sequence;

aiming at each second column, acquiring a transverse distance difference value between the respective transverse distances of a first target and a next target in the second column;

if yes, determining that the next target and the first target in the second column are second approximate targets which are arranged side by side;

if not, determining that the next target and the first target in the second column are second non-approximate targets which are arranged in rows;

continuously judging whether the transverse distance difference between the respective transverse distances of the next target and the next target in the second column is smaller than or equal to the preset transverse distance difference;

and repeating the steps according to the judgment result until the comparison between the transverse distance difference value corresponding to the last target in each second column and the preset transverse distance difference value is completed, and determining the targets in the second sequence obtained after the comparison and arrangement are finished as a second clustering result.

In one possible design, the third generic module is specifically configured to:

aiming at each third column, acquiring a longitudinal distance difference value between respective longitudinal distances of a first target and a next target in the third column;

if not, determining that the next target in the third column is different from the first target, and arranging the different targets in rows;

continuously judging whether the longitudinal distance difference between the respective longitudinal distances of the next target and the next target in the third column is less than or equal to the preset longitudinal distance difference;

and repeating the steps according to the judgment result until the comparison between the longitudinal distance difference value corresponding to the last target in each third column and the preset longitudinal distance difference value is completed, and determining the target in the sequence obtained after the comparison and the arrangement is finished as a third clustering result.

The data processing apparatus provided in the embodiment of the present application may perform corresponding steps of the data processing method in the foregoing method embodiment, and the implementation principle and the technical effect are similar, which are not described herein again.

Fig. 9 is a schematic structural diagram of an electronic device according to an embodiment of the present application. As shown in fig. 9, the electronic device 500 may include: a processor 501, and a memory 502 communicatively coupled to the processor 501.

The memory 502 is used for storing programs. In particular, the program may include program code comprising computer-executable instructions.

Memory 502 may comprise high-speed RAM memory, and may also include non-volatile memory (MoM-volatile memory), such as at least one disk memory.

The processor 501 is used to execute computer-executable instructions stored by the memory 502 to implement a data processing method.

The processor 501 may be a Central Processing Unit (CPU), an Application Specific Integrated Circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

Alternatively, the memory 502 may be separate or integrated with the processor 501. When the memory 502 is a device independent of the processor 501, the electronic device 500 may further include:

a bus 503 for connecting the processor 501 and the memory 502. The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, an Extended ISA (EISA) bus, or the like. Buses may be classified as address buses, data buses, control buses, etc., but do not represent only one bus or type of bus.

Alternatively, in a specific implementation, if the memory 502 and the processor 501 are integrated into a chip, the memory 502 and the processor 501 may complete communication through an internal interface.

The present application also provides a computer-readable storage medium, which may include: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and in particular, the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used for the data processing method in the foregoing embodiments.

The present application also provides a computer program product comprising computer executable instructions, which when executed by a processor implement the data processing method in the above embodiments.

Other embodiments of the present application will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention following, in general, the principles of the application and including such departures from the present disclosure as come within known or customary practice within the art to which the invention pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the application being indicated by the following claims.

It will be understood that the present application is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the application is limited only by the appended claims.

Claims

1. A data processing method, comprising:

collecting target information around a vehicle through a forward millimeter wave radar of the vehicle, and collecting environmental information around the vehicle through a forward-looking camera of the vehicle;

2. The data processing method of claim 1, wherein the target information collected by the forward millimeter wave radar includes a speed, a lateral distance, and a longitudinal distance of each target;

3. The data processing method of claim 2, wherein determining a second target based on the first target and environmental information collected by the forward looking camera comprises:

if yes, mapping the first target to a lane corresponding to the lane line image according to lane line data in the environment information, and determining the second target according to a mapping result;

4. The data processing method of claim 3, wherein determining the second target according to the mapping result comprises:

5. The data processing method according to any one of claims 1 to 4, wherein the determining a first target according to the target information collected by the forward millimeter wave radar comprises:

6. The data processing method of claim 5, wherein the clustering the velocity, the lateral distance, and the longitudinal distance of each target to determine the first target comprises:

7. The data processing method of claim 6, wherein the performing a first clustering process based on the speed difference from a first target in the first column to obtain a first clustering result comprises:

8. The data processing method according to claim 7, wherein performing second clustering processing according to the first clustering result and the lateral distance of the target corresponding to the first clustering result to obtain a second clustering result comprises:

9. The data processing method according to claim 8, wherein the performing the second clustering process according to the second clustering result and the longitudinal distance of the target corresponding to the second clustering result to obtain a third clustering result comprises:

if so, determining that the next target and the first target in the third column are the same target, and arranging the same targets side by side;

10. A data processing apparatus, comprising:

the acquisition module is used for acquiring target information around the vehicle through a forward millimeter wave radar of the vehicle and acquiring environmental information around the vehicle through a forward-looking camera of the vehicle;

11. An electronic device, comprising: a processor, and a memory communicatively coupled to the processor;

the memory stores computer-executable instructions;

the processor executes computer-executable instructions stored by the memory to implement the data processing method of any of claims 1 to 9.

12. A computer-readable storage medium having computer-executable instructions stored thereon, which when executed by a processor, are configured to implement the data processing method of any one of claims 1 to 9.