CN116091556A - Road edge tracking method, device, equipment and storage medium - Google Patents

Abstract

The application discloses a roadside tracking method, device, equipment and storage medium. The road edge tracking method includes obtaining the predicted road edge point set and the detected road edge point set at the current moment; grouping the detected road edge point set based on the predicted road edge point set to obtain the grouping result of the detected road edge point set; obtaining the current moment Whether there is a curb recognition result with incomplete curbs on the road the vehicle is driving; based on the curb recognition result and the grouping result of the detected curb point set, the predicted curb point set and the detected curb point set are fused to obtain the fusion road Point set along the road: Carry out curve fitting on the point set along the fusion road, and obtain the fitting result of the road edge at the current moment. Through the above-mentioned implementation, the predicted roadside point set and the detected roadside point set can be fused according to whether the detected roadside points are transformed or not, and whether the roadside is blocked or missing, so that the roadside in complex road conditions can also be stable. , Accurately tracked out, good robustness.

Description

A roadside tracking method, device, equipment and storage medium

technical field

The present application relates to the technical field of traffic, and in particular to a roadside tracking method, device, equipment and storage medium.

Background technique

The perception of the surrounding environment during vehicle driving is the basis for realizing vehicle intelligent assisted driving and unmanned driving. Road edge detection technology is an important link to realize vehicle intelligent path planning and decision-making control. ) and Lane Departure Warning (Lane Departure Warning, LDW) and other assisted driving basis.

The existing road edge detection technology usually obtains the current road scene photo of the road where the vehicle is located, and detects the road edge from the current road scene photo. However, this road edge detection strategy cannot be adjusted according to real-time road edge conditions, resulting in low accuracy of road edge detection.

Contents of the invention

In order to solve the above-mentioned technical problems existing in the prior art, the present application provides a roadside tracking method, device, equipment and storage medium.

In order to solve the above problems, the present application provides a roadside tracking method, the roadside tracking method includes: obtaining the predicted roadside point set and the detected roadside point set at the current moment; based on the predicted roadside point set, the detected The roadside point set is grouped, and the grouping result of the detected roadside point set is obtained; whether there is an incomplete roadside identification result on the road that the vehicle is running at the current moment is obtained; based on the roadside identification result and the detection For the grouping result of the roadside point set, the predicted roadside point set and the detected roadside point set are fused to obtain the fused roadside point set; curve fitting is performed on the fused roadside point set to obtain the current Curb fitting result at time.

In order to solve the above problems, the application provides a curb fitting device, which includes: an acquisition module, a grouping module, a fusion module and a fitting module; the acquisition module is used to obtain the predicted curb at the current moment A point set and a detected roadside point set; the grouping module is used to group the detected roadside point set based on the predicted roadside point set, and obtain the grouping result of the detected roadside point set; the acquiring The module is used to obtain the curb recognition result of whether there is an incomplete curb on the road the vehicle is driving at the current moment; the fusion module is used to classify the performing fusion processing on the predicted roadside point set and the detected roadside point set to obtain a fused roadside point set; the fitting module is used to perform curve fitting on the fused roadside point set to obtain the current moment Fitting results.

In order to solve the above problems, the present application provides a roadside fitting device, the roadside fitting device includes: a processor and a memory, a computer program is stored in the memory, and the processor is used to execute the computer program to realize the above method.

In order to solve the above problems, the present application provides a computer-readable storage medium on which program instructions are stored, and when the program instructions are executed by a processor, the above method is implemented.

Compared with the prior art, the curb tracking method of the present application includes: obtaining the predicted curb point set and the detected curb point set at the current moment; grouping the detected curb point set based on the predicted curb point set to obtain the detected curb point set The grouping result of the roadside point set; obtain the roadside recognition result of whether the road that the vehicle is driving at the current moment has an incomplete roadside; based on the roadside recognition result and the grouping result of the detected roadside point set, predict the roadside point set and detect The roadside point set is fused to obtain the fused roadside point set; curve fitting is performed on the fused roadside point set to obtain the roadside fitting result at the current moment. Through the above-mentioned embodiment, at the same time refer to the curb identification result and the grouping result of the detected curb point set to perform fusion processing on the predicted curb point set and the detected curb point set, and then perform curve fitting on the fused curb point, Therefore, according to whether the identified detected roadside points are transformed or not, and whether the roadside is blocked or missing, the predicted roadside point set and the detected roadside point set can be fused, so that the roadside in complex road conditions can also be stably and accurately detected. Tracking out, good robustness.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the accompanying drawings required in the embodiments. Obviously, the accompanying drawings in the following description are only some of the present application. Embodiments, for those of ordinary skill in the art, other drawings can also be obtained based on these drawings without any creative effort.

Fig. 1 is a schematic flow chart of an embodiment of the roadside tracking method provided by the present application;

Fig. 2 is a schematic flow chart of an embodiment of step S102 in Fig. 1;

FIG. 3 is a schematic flow diagram of an embodiment of obtaining the predicted roadside point set at the current moment provided by the present application;

Fig. 4 is a schematic structural diagram of an embodiment of a roadside fitting device provided by the present application;

FIG. 5 is a schematic structural view of an embodiment of a roadside fitting device provided by the present application;

Fig. 6 is a schematic structural diagram of an embodiment of a computer storage medium provided by the present application.

Detailed ways

The application will be described in further detail below in conjunction with the accompanying drawings and embodiments. In particular, the following examples are only used to illustrate the present application, but not to limit the scope of the present application. Likewise, the following embodiments are only some of the embodiments of the present application but not all of them, and all other embodiments obtained by those skilled in the art without creative efforts fall within the protection scope of the present application.

Reference herein to an "embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the present application. The occurrences of this phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. It is understood explicitly and implicitly by those skilled in the art that the embodiments described herein can be combined with other embodiments.

In the description of this application, it should be noted that, unless otherwise clearly stipulated and limited, the terms "installation", "setting", "connection" and "connection" should be understood in a broad sense, for example, it can be a fixed connection, it can be a Detachable connection, or integral connection; it can be mechanically connected or electrically connected; it can be directly connected or connected at intervals through an intermediate medium. For those skilled in the art, the above-mentioned specific meanings belonging to the present application can be connected in specific situations.

The perception of the surrounding environment during vehicle driving is the basis for realizing vehicle intelligent assisted driving and unmanned driving. Road edge detection technology is an important link to realize vehicle intelligent path planning and decision-making control. ) and Lane Departure Warning (Lane Departure Warning, LDW) and other assisted driving basis.

Existing roadside tracking methods for complex road conditions may mostly use a single strategy of fusing detection results and prediction results. When the road is blocked or missing, it is difficult to accurately track the shape and position of the roadside. For example, in some roads there will be vehicle congestion and vehicles blocking the roadside, and lidar cannot detect the roadside points of the blocked part, while the traditional roadside tracking method uses a single strategy of fusing detection results and prediction results , cannot make adjustments according to the real-time roadside conditions, and it is difficult to accurately track the shape and position of the roadside; Points will cause errors in roadside fitting. The traditional roadside tracking method adopts a single fusion detection and prediction strategy, which cannot adjust the tracking strategy according to the real-time roadside conditions, and it is difficult to accurately track the shape and position of the roadside; some roads will Including multiple roads, such as ramps, viaduct entrances and exits, etc., the traditional roadside tracking method does not have matching rules for roadside points, and there will be problems with tracking results jumping, which will affect the output of the actual roadside.

In order to solve a series of technical problems existing in the prior art, the present application provides a roadside tracking method, see Fig. 1, Fig. 1 is a schematic flow chart of an embodiment of the roadside tracking method provided by the present application, specifically It includes the following steps S101 to S105.

Step S101: Obtain the predicted roadside point set and detected roadside point set at the current moment.

During the normal driving process of a vehicle on a road with a curb, in order to accurately identify the actual situation of the curb, the predicted curb point set and the detected curb point set at the current moment can be obtained, so as to pass the predicted curb point set and detection The curb point set handles the output curb condition. The predicted roadside point set can be calculated according to the relevant data according to a specific algorithm to predict the roadside condition of the vehicle at the current moment. The output road edge condition is obtained by processing. The detection of the roadside point set can be obtained by obtaining the information of the roadside in real time during the driving process of the vehicle, and then analyzing and processing the information of the roadside. processing to obtain the detected roadside point set; or the original point cloud can be detected by lidar technology, and then the original point cloud can be processed to obtain the detected roadside point set. For example, the vehicle can be equipped with a laser, and the laser emits The pulse laser hits the surrounding trees, roads, cars, pedestrians, etc., causing scattering, and part of the light waves will be reflected to the receiver of the lidar. According to the calculation of the principle of laser ranging, the distance from the laser radar to the target point can be obtained. The pulse laser continuously scans the target object, and the data of all target points on the target object can be obtained. After imaging processing with this data, the accurate 3D stereoscopic image, and then analyze the image, and finally get the detection roadside point set.

Step S102: Grouping the detected roadside point set based on the predicted roadside point set to obtain a grouping result of the detected roadside point set.

The detected roadside points are usually scattered points without roadside attribute characteristics. The predicted roadside point set at the current moment is calculated according to the relevant algorithm. Therefore, when the roadside does not change significantly, the predicted roadside point set It is more accurate. At this time, the predicted roadside point set can be used to group and divide the detected roadside point set. Exemplarily, the predicted roadside point set usually includes the left roadside predicted point set and the right roadside predicted point set, and the detected roadside point set can be divided into Compare the roadside points with the predicted roadside points, and then divide the detected roadside point set into left roadside detection point set, right roadside detection point set, new roadside detection point set, and finally get the grouping of detected roadside point sets The results include whether a nascent curb set exists.

Step S103: Obtain the roadside identification result of whether the road the vehicle is driving at the current moment has an incomplete roadside.

During the normal driving process of the vehicle on the road, by acquiring the image of the periphery of the vehicle, and then analyzing the image, it is finally determined whether there is an incomplete roadside recognition result on the road the vehicle is driving at the current moment. Exemplarily, the vehicle can be equipped with a camera sensor, which can acquire roadside images on both sides of the vehicle in real time, and then input the roadside image into the trained neural network model, and the neural network model outputs whether the roadside is blocked or whether For the missing road edge recognition result, when the road edge is blocked or missing, the recognition result of incomplete road edge is output, otherwise the recognition result of no incomplete road edge is output. Wherein, the neural network model can adopt Efficientnet or Resnet network model. In other embodiments, the neural network model can also be other network models, as long as it can output the roadside recognition result of whether there is an incomplete roadside on the road.

Step S104: Based on the curb recognition result and the grouping result of the detected curb point set, perform fusion processing on the predicted curb point set and the detected curb point set to obtain a fused curb point set.

The curb recognition results include the recognition results of the road with incomplete curbs and the recognition results of the absence of incomplete curbs. The grouping results of the detected curb point sets include the grouping results of the new road edge point set and the non-existent new road edge point set. . When the curb recognition result at the current moment and the grouping result of the detected curb point set are determined, the predicted curb point set and the detected curb point set can be fused to obtain a fused curb point set. Among them, the fusion processing of the predicted roadside point set and the detected roadside point set may include only outputting the predicted roadside point set or only the detected roadside point set as the fusion roadside point, or setting the predicted roadside point set and Detect the respective weights of the roadside point sets, and then use the fusion algorithm to fuse the predicted roadside point set and the detected roadside point set, and finally obtain the fused roadside point set.

Step S105: Curve fitting is performed on the fused roadside point set to obtain the roadside fitting result at the current moment.

After the fusion roadside points are obtained, curve fitting can be performed on the fusion roadside points based on the curve fitting algorithm, and finally the roadside fitting result at the current moment can be obtained. Exemplarily, curve fitting can be performed on the fusion roadside points by RANSAC algorithm, least squares method or other curve fitting algorithms, and when the fitted curve model does not meet the requirements, the fusion roadside point set can be processed in the same way Perform multiple fittings to finally get the roadside fitting result.

Through the above-mentioned embodiment, at the same time refer to the curb identification result and the grouping result of the detected curb point set to perform fusion processing on the predicted curb point set and the detected curb point set, and then perform curve fitting on the fused curb point, Therefore, according to whether the identified detected roadside points are transformed or not, and whether the roadside is blocked or missing, the predicted roadside point set and the detected roadside point set can be fused, so that the roadside in complex road conditions can also be stably and accurately detected. Tracking out, good robustness.

In one embodiment, based on the curb recognition result and the grouping result of the detected curb point set, performing fusion processing on the predicted curb point set and the detected curb point set (step S104) includes: based on the curb recognition result and the detected curb point set According to the grouping result of the edge point set, the respective fusion weights of the predicted road edge point set and the detected road edge point set are determined; the predicted road edge point set and the detected road edge point set are fused according to their respective fusion weights.

Specifically, the predicted roadside point set and the detected roadside point set can be fused according to the following calculation method:

f(x)=Eg(x)+Fc(x)

Among them, g(x) represents the prediction result of the predicted roadside point set, c(x) represents the detection result of the detected roadside point set, E represents the prediction weight coefficient of the predicted roadside point set, and F represents the value of the detected roadside point set The detection weight coefficient, f(x) represents the fusion result of the fusion prediction roadside point set and the detection roadside point set.

The weight coefficient of the predicted roadside point set and the weight coefficient of the detected roadside point set are determined by the grouping result and the roadside recognition result, so that the fusion result is more in line with the actual situation. For example, when the roadside recognition result and the detected roadside If it is determined that the predicted roadside point set is more in line with the real situation in the grouping results of the point set, the value of the prediction weight coefficient can be set to be greater than the value of the detection weight coefficient; otherwise, the value of the prediction weight coefficient can be set to be smaller than that of the detection weight coefficient value. Specifically, see the following examples:

In one embodiment, based on the curb identification result and the grouping result of the detected curb point set, the respective fusion weights of the predicted curb point set and the detected curb point set are determined, including: if the grouping result of the detected curb point set If there is no new road edge point set and there is no road edge incompleteness in the road edge recognition result, it is determined that the difference between the fusion weight of the predicted road edge point set and the fusion weight of the detected road edge point set is less than or equal to the preset threshold.

It indicates that the current roadside is not blocked or missing, and it is also detected that the vehicle will continue to drive on the current road and will not enter the new road. In this embodiment, when determining the importance function, it is judged that the predicted result and the detected result can be believed at the same time, so the difference between the fusion weight of the predicted roadside point set and the fusion weight of the detected roadside point set can be set to be less than Or equal to the preset threshold, so as to simultaneously fuse the predicted roadside point set and the detected roadside point set. For example, the weight coefficient of the predicted roadside point set and the weight coefficient of the detected roadside point set are set to 0.5, so that the difference between the two is 0, so that the predicted roadside point set and the detected roadside point set are fused as the final Fusion results.

In one embodiment, based on the curb identification result and the grouping result of the detected curb point set, the respective fusion weights of the predicted curb point set and the detected curb point set are determined, including: if the grouping result of the detected curb point set If there is no new road edge point set and there is an incomplete road edge in the road edge recognition result, it is determined that the fusion weight of the predicted road edge point set is greater than the fusion weight of the detected road edge point set.

Indicates that the current roadside is at least partially blocked or at least partially missing, and the blocked or missing part of the roadside cannot be detected. At the same time, it is also detected that the vehicle will continue to drive on the current road and will not enter the new road. In this case, the traditional tracking algorithm will fuse the detection and prediction results, but the output curb will deviate from the actual curb. However, in this embodiment, due to the lack or occlusion of the road, the occluded or missing part of the roadside cannot be detected, resulting in a discrepancy between the detected roadside point set and the actual situation. When determining the importance function, it is judged that the prediction should be more trusted Therefore, the fusion weight of the predicted roadside point set can be set to be greater than the fusion weight of the detected roadside point set. For example, the weight coefficient of the predicted roadside point set is set to 1, and the weight coefficient of the detected roadside point set is set to 0, so that the predicted roadside point set is directly regarded as the final fusion result.

In one embodiment, based on the curb identification result and the grouping result of the detected curb point set, the respective fusion weights of the predicted curb point set and the detected curb point set are determined, including: if the grouping result of the detected curb point set If there is a new road edge point set and there is no road edge incompleteness in the road edge recognition result, it is determined that the fusion weight of the predicted road edge point set is smaller than the fusion weight of the detected road edge point set.

It indicates that the current roadside is not blocked or missing, and at the same time, it is detected that the vehicle will not continue to drive on the current road, but enter the new road. In this embodiment, since the predicted roadside point set is usually the roadside point when the vehicle continues to drive on the original road, when the vehicle needs to enter the new road, the predicted roadside point set of the predicted roadside point set does not match the actual situation. Then when determining the importance function, it is judged that the detection result should be believed more, so the fusion weight of the detected roadside point set can be set to be greater than the fusion weight of the predicted roadside point set, for example, the weight coefficient of the detected roadside point set is set to 1. The weight coefficient of the predicted roadside point set is set to 0, so that the detected roadside point set is directly regarded as the final fusion result.

In one embodiment, based on the curb identification result and the grouping result of the detected curb point set, the respective fusion weights of the predicted curb point set and the detected curb point set are determined, including: if the grouping result of the detected curb point set If there is a new road edge point set and there is an incomplete road edge in the road edge recognition result, it is determined that the fusion weight of the predicted road edge point set is smaller than the fusion weight of the detected road edge point set.

Indicates that the current road edge is at least partially occluded or at least partially missing, and the occluded or missing part of the road edge cannot be detected, and a new road edge point set is also detected. In this embodiment, due to the lack or occlusion of the road, the occluded or missing part of the roadside cannot be detected, resulting in the detection of the roadside point set does not match the actual situation. When determining the importance function, it is judged that the detection can be more reliable As a result, the fusion weight of the predicted roadside point set can be set to be smaller than the fusion weight of the detected roadside point set, for example, the weight coefficient of the predicted roadside point set is set to 0, and the weight coefficient of the detected roadside point set is set to 1 , so that the detected roadside point set is directly regarded as the final fusion result.

In the above-mentioned embodiment, it is mainly introduced that the predicted roadside point set and the detected roadside point set can be fused according to the grouping result of the detected roadside point set. Therefore, how to group the detected roadside point set is very important, see Fig. 2, Fig. 2 is a schematic flowchart of an embodiment of step S102 in Fig. 1, specifically, it includes the following steps S201 to S203.

Step S201: Select a target roadside point whose distance value has not been calculated from the detected roadside point set and calculate the distance from the left roadside predicted point set in the predicted roadside point set to obtain the left roadside distance value, and set it with the predicted roadside point set Calculate the distance of the right-road point set to get the distance value of the right-road, and repeat the current step until all detected road-side points are traversed.

There are many detected roadside points in the concentration of detected roadside points, and more detected roadside points need to be grouped to make it clear that some detected roadside points are the left roadside points of the current road, and some detected roadside points are Right curb points of the current road, some detected curb points are new curb points. In this embodiment, a point whose distance value has not been calculated can be selected from several detected curb points as the target curb point, and the distance between the target curb point and the predicted point of the left curb can be calculated to obtain the distance value of the left curb , and at the same time calculate the distance with the right-curb point set to obtain the distance value of the right-curb, so as to determine whether the target curb point belongs to the point on the left-curb or the point on the right-curb according to the size of the distance value. After obtaining the left-curb distance value and right-curb distance value of a detected curb point, the left-curb distance value and right-curb distance value of another detected curb point can be calculated in the same way, so as to finally obtain each The left curb distance value and the right curb distance value of each detected curb point.

Specifically, calculating the distance value of the detected roadside point may specifically include: calculating the distance value between the selected target roadside point and all left roadside prediction points in the left roadside prediction point set, and calculating the average value of all distance values As the distance value along the left road; calculate the distance value between the selected target roadside point and all the predicted points along the right roadside in the set of predicted points along the right roadside, and calculate the average value of all the distance values as the distance value along the right roadside. That is to say, first calculate the Euclidean distance between the target roadside point and each predicted point along the left road, then sum the calculated Euclidean distances and calculate the average value, and use the average value as the distance value along the left road. Use the same method to obtain the distance value to the right road edge. The same steps are repeated until all detected roadside points are traversed to obtain the left roadside distance value and right roadside distance value of each detected roadside point.

Step S202: Selecting the smaller one from the left curb distance value and the right curb distance value of the detected curb point to compare with the preset distance threshold.

After obtaining the distance value of the left roadside and the right roadside of each detected roadside point, it is necessary to group the detected roadside points according to the distance value. In this embodiment, it is necessary to select the one with the smaller value from the left and right roadside distance values of all detected roadside points to compare with the preset distance threshold, wherein the preset distance threshold can be determined according to the actual situation. Situation setting. Exemplarily, one of the target curb points is first selected from a plurality of detected curb points for processing. For example, the value of the left curb distance value of the target curb point is greater than the value of the right curb distance value, then the right curb distance Value is compared with the preset distance to get the comparison result, and then another detected roadside point that has not been compared with the preset distance threshold is selected from the detected roadside point set, and the left roadside distance of the selected detected roadside point is selected value and the distance value of the right road edge, choose the smaller one to compare with the preset distance threshold to obtain the comparison result, and repeat the steps to traverse all detected roadside points until the preset distance between each detected roadside point and the preset distance is obtained Threshold comparison result.

Step S203: Based on the distance comparison result, it is determined whether there is a new road edge point set in the grouping result of the detected road edge point set.

After obtaining the comparison result between each detected roadside point and the preset distance threshold, each detected roadside point can be grouped. Specifically, when the comparison result is less than or equal to the preset distance threshold, the detected roadside point can be grouped. The edge point is defined as the edge point of the original road. Specifically, the detected edge point can also be determined as the left edge point or the right edge point of the road. Exemplarily, the right edge distance of the detected edge point can be selected. When the value is compared with the preset distance threshold value, the detected roadside point is a point on the right roadside; when the left roadside distance value of the selected detection roadside point is compared with the preset distance threshold value, the detected roadside point is The point on the left curb. When the comparison result is greater than the preset distance threshold, the detected roadside point can be defined as an undetermined roadside point. When the number of undetermined roadside points is large and enough to form a road, it can be determined that there is a new roadside point point set.

Specifically, based on the distance comparison result, it is determined whether there is a new roadside point set in the grouping result of the detection roadside point set (step S203), including:

If the number of undetermined roadside points whose distance value is greater than the preset distance threshold in the detected roadside points is greater than the preset number threshold, then the minimum value of the ordinate value of all undetermined roadside points in the vehicle coordinate system is compared with the preset ordinate value threshold, if the minimum value of the ordinate values of all undetermined roadside points is less than the preset ordinate threshold, then it is determined that there is a new roadside point set in the grouping result of the detected roadside point set, otherwise it is determined that the detected roadside point set The new curb point set does not exist in the grouping result.

The preset quantity threshold can be set according to actual conditions, and is not limited here. In this embodiment, the vehicle coordinate system can be established with the vehicle as the origin. Specifically, the direction the vehicle is advancing can be defined as the positive direction of the x-axis in the vehicle coordinate system, and the left side of the vehicle can be defined as the vehicle coordinate system. The positive direction of the y-axis. When the number of undetermined roadside points whose distance value is greater than the preset distance threshold in the detected roadside points is greater than the preset number threshold, the coordinate values of all undetermined roadside points in the vehicle coordinate system can be recorded. When all undetermined roadside points When the ordinate values in are less than the preset ordinate threshold, all undetermined curb points are defined as the new curb point set, and the new curb point set can be defined as the left curb point set of the new curb or the right curb point set, and then the curb located on the other side of the vehicle in the original curb point set is defined as the left curb or right curb of the new curb, for example, when the new curb is located on the left side of the vehicle , then the Xinsheng road edge set can be used as the left edge of the Xinsheng road, and the right edge set of the original road can be used as the right edge set of the Xinsheng road, and finally the Xinxin road edge set and the original road can be used The right-curb point set of is fitted to form a new curb. When it is not satisfied that the minimum values of the ordinate values in all the undetermined roadside points are less than the preset ordinate value threshold, it is determined that there is no new roadside point set.

It should be noted that, in the above-mentioned embodiment, the step: comparing the minimum value of the ordinate values of all undetermined roadside points in the vehicle coordinate system with the preset ordinate threshold value is established on the basis of the vehicle coordinate system with the vehicle as the origin, The forward direction of the vehicle is defined as the positive direction of the x-axis in the vehicle coordinate system, and the left side of the vehicle is defined as the positive direction of the y-axis in the vehicle coordinate system. In other embodiments, when the definition of the vehicle coordinate system is different, in the step: comparing the minimum value of the ordinate values of all undetermined roadside points in the vehicle coordinate system with the preset ordinate threshold value, the change will also occur, for example Specifically, when the coordinate system is established with the vehicle as the origin, the forward direction of the vehicle is defined as the positive direction of the y-axis in the vehicle coordinate system, and the left side of the vehicle is defined as the positive direction of the x-axis in the vehicle coordinate system. At this time, the above steps can be changed to: compare the minimum value of the abscissa value of all undetermined roadside points in the vehicle coordinate system with the preset abscissa threshold, if the minimum value of the abscissa value of all undetermined roadside points is less than If the ordinate threshold is preset, it is determined that there is a new roadside point set in the grouping result of the detected roadside point set, otherwise it is determined that there is no new roadside point set in the grouping result of the detected roadside point set.

In the above-mentioned embodiment, it is mainly introduced that the grouping of the detected roadside point set can be determined according to the predicted roadside point set. Therefore, how to determine the predicted roadside point set is very important. See Figure 3, which is provided by the present application A schematic flowchart of an embodiment of obtaining the predicted roadside point set at the current moment, specifically, includes the following steps S301 to S302.

Step S301: Determine the lateral displacement, longitudinal displacement and rotation angle of the vehicle from the previous moment to the current moment based on the vehicle motion information at the previous moment.

For two consecutive moments of the previous moment and the current moment, each moment can be separated by seconds. For example, if the previous moment is 19:30:29, the current moment can be 19:30:30, of course The specific time interval between the previous moment and the current moment can also be other values, which is not limited here. The vehicle motion information can include the vehicle's speed and rotation angle information, etc., so that the lateral displacement and longitudinal displacement of the vehicle from the previous moment to the current moment can be calculated according to the following formula:

Among them, Δx and Δy respectively represent the longitudinal displacement and lateral displacement of the vehicle from time k-1 to time k (under the premise that the front of the vehicle is the positive direction of the x-axis and the left is the positive direction of the y-axis), v represents the time k-1 T(k-1, k) represents the time elapsed from time k-1 to time k, and θ represents the rotation angle of the vehicle around the vertical direction.

Step S302: Based on the lateral displacement, longitudinal displacement, rotation angle, the translation transformation matrix of the vehicle coordinate system, the rotation transformation matrix of the vehicle coordinate system, and the coordinate values of the fused roadside points in the vehicle coordinate system at the previous moment, determine the current moment Set of predicted curb points.

Before performing this step, the translation transformation matrix of the vehicle coordinate system and the rotation transformation matrix of the vehicle coordinate system can be obtained first. The coordinate value of the fusion roadside point in the vehicle coordinate system at the previous moment can be implemented by using any of the above In the same way as the example, obtain the calculated fusion roadside point set. Then, the coordinate value of the fused roadside point in the vehicle coordinate system at the previous moment, and the rotation angle, lateral displacement and longitudinal displacement obtained in the previous step are combined to calculate the predicted roadside point set at the current moment. Specifically, the predicted roadside point set at the current moment can be calculated according to the following formula:

Among them, x _k and y _k represent the ordinate and abscissa of the roadside point in the vehicle coordinate system at time k respectively; A represents the rotation transformation matrix of the vehicle coordinate system; B represents the translation transformation matrix of the vehicle coordinate system; Δx and Δy respectively Represents the longitudinal displacement and lateral displacement of the vehicle from time k-1 to time k, and θ represents the rotation angle of the vehicle around the vertical direction.

In one embodiment, the step of performing curve fitting on the fused curb point set to obtain the curb fitting result at the current moment (step S105) includes: selecting several fitted curb points from the fused curb points; Fit the curb points to perform curve fitting to obtain the fitted curve model; match other fused curb points with the fitted curve model; if the number of fused curb points within the preset range is greater than or equal to the preset number threshold, the fitting curve model will be used as the roadside fitting result; if the number of fused roadside points whose matching degree is within the preset range is less than the preset number threshold, then return to the step to select some simulated roadside points from the fused roadside points. Merge along the road.

The fused set of curb points may include a set of fused left kerb points and a fused set of right kerb points. Curve fitting may be performed on the fused set of left kerb points respectively, and then curve fitting may be performed on the set of fused right kerb points. Illustratively, taking fitting and fusion of the left-road point set as an example, a number of roadside points that need to be curve-fitted are selected from the fusion left-road point set, and then other fused left-road point sets are compared with the fitting curve model. Matching to verify whether the fitting curve model meets the requirements. Specifically, when the number of matching degrees between other fused left road points and the fitting curve model is within a preset range, the fitting curve model can be considered to meet the requirements. It is only necessary to output the fitted curve model as the roadside fitting result; when the matching degree of other fused left roadside points and the fitted curve model is less than the preset range, the fitted curve model can be regarded as If the requirements are not met, several roadside points can be reselected from the fused left roadside point set to fit a new fitting curve model, and this step can be repeated until a fitting curve model meeting the requirements is obtained.

Through the above-mentioned embodiment, at the same time refer to the curb identification result and the grouping result of the detected curb point set to perform fusion processing on the predicted curb point set and the detected curb point set, and then perform curve fitting on the fused curb point, Therefore, according to whether the identified detected roadside points are transformed or not, and whether the roadside is blocked or missing, the predicted roadside point set and the detected roadside point set can be fused, so that the roadside in complex road conditions can also be stably and accurately detected. Tracking out, good robustness.

The roadside tracking method in this embodiment can be applied to a roadside fitting device. The roadside fitting device in this application can be a server, a mobile device, or a system in which a server and a mobile device cooperate with each other. Correspondingly, various parts included in the mobile device, such as various units, subunits, modules, and submodules, may all be set in the server, or all of them may be set in the mobile device, or they may be set in the server and the mobile device separately.

Further, the above server may be hardware or software. When the server is hardware, it can be implemented as a distributed server cluster composed of multiple servers, or as a single server. When the server is software, it can be implemented as multiple software or software modules, such as software or software modules used to provide a distributed server, or as a single software or software module, which is not specifically limited here.

In order to implement the road edge tracking method in the above embodiments, the present application provides a road edge fitting device. Referring to FIG. 4 , FIG. 4 is a structural schematic diagram of an embodiment of a roadside fitting device provided by the present application.

Specifically, the roadside fitting device 40 may include an acquisition module 41 , a grouping module 42 , a fusion module 43 and a fitting module 44 .

The acquiring module 41 is used to acquire the set of predicted roadside points and the set of detected roadside points at the current moment.

The grouping module 42 is configured to perform grouping processing on the detected roadside point set based on the predicted roadside point set, and obtain a grouping result of the detected roadside point set.

The acquiring module 41 is used to acquire the curb recognition result of whether there is an incomplete curb on the road the vehicle is driving at the current moment.

The fusion module 43 is configured to perform fusion processing on the predicted curb point set and the detected curb point set based on the curb recognition result and the grouping result of the detected curb point set to obtain a fused curb point set.

The fitting module 44 is used to perform curve fitting on the fused roadside point set to obtain the roadside fitting result at the current moment.

Wherein, in one embodiment of the present application, each module in the curb fitting device 40 shown in FIG. The same operation can be realized without affecting the realization of the technical effects of the embodiments of the present application by splitting into multiple functionally smaller subunits. The above-mentioned modules are divided based on logical functions. In practical applications, the functions of one module can also be realized by multiple units, or the functions of multiple modules can be realized by one unit. In other embodiments of the present application, the roadside fitting device 40 may also include other units. In practical applications, these functions may also be implemented with the assistance of other units, and may be implemented cooperatively by multiple units.

The above method is applied to the roadside fitting device. Please refer to FIG. 5 for details. FIG. 5 is a schematic structural diagram of an embodiment of a roadside fitting device provided in the present application. The roadside fitting device 50 of this embodiment includes a processor 51 and a memory 52 . Wherein, a computer program is stored in the memory 52, and the processor 51 is used to execute the computer program to realize the above road edge tracking method.

Wherein, the processor 51 may be an integrated circuit chip, which has a signal processing capability. The processor 51 can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components . A general-purpose processor may be a microprocessor, or the processor may be any conventional processor, or the like.

For the roadside tracking method of the above-mentioned embodiment, it can be presented in the form of a computer program. This application proposes a computer storage medium carrying a computer program. Please refer to FIG. 6, which is an example of an embodiment of a computer storage medium provided by this application. Schematic diagram of the structure, the computer storage medium 60 of this embodiment includes a computer program 61 that can be executed to implement the above road edge tracking method.

In this embodiment, the computer storage medium 60 can be a medium that can store program instructions such as a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, Or it can also be a server that stores the program instructions, and the server can send the stored program instructions to other devices to run, or can also run the stored program instructions by itself.

In addition, if the above-mentioned functions are implemented in the form of software functions and sold or used as independent products, they can be stored in a storage medium that can be read by a mobile terminal, that is, the application also provides a storage device that stores program data, so The above program data can be executed to implement the methods of the above embodiments, and the storage device can be, for example, a U disk, an optical disk, a server, and the like. That is to say, the present application may be embodied in the form of a software product, which includes several instructions for enabling an intelligent terminal to execute all or part of the steps of the method described in each embodiment.

In the description of this application, reference to the terms "one embodiment," "some embodiments," "example," "specific examples," or "some examples" means that specific features described in connection with that embodiment or example , structure, material or characteristic is included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms are not necessarily directed to the same embodiment or example. Furthermore, the described specific features, structures, materials or characteristics may be combined in any suitable manner in any one or more embodiments or examples. In addition, those skilled in the art can combine and combine different embodiments or examples and features of different embodiments or examples described in this specification without conflicting with each other.

In addition, the terms "first" and "second" are used for descriptive purposes only, and cannot be interpreted as indicating or implying relative importance or implicitly specifying the quantity of indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include at least one of these features. In the description of the present application, "plurality" means at least two, such as two, three, etc., unless otherwise specifically defined.

Any process or method descriptions in flowcharts or otherwise described herein may be understood to represent modules, segments or portions of code comprising one or more executable instructions for implementing specific logical functions or steps of the process , and the scope of preferred embodiments of the present application includes additional implementations in which functions may be performed out of the order shown or discussed, including in substantially simultaneous fashion or in reverse order depending on the functions involved, which shall It should be understood by those skilled in the art to which the embodiments of the present application belong.

The logic and/or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a sequenced listing of executable instructions for implementing logical functions, can be embodied in any computer-readable medium, For the use of instruction execution systems, devices or equipment (which may be personal computers, servers, network equipment or other systems that can fetch instructions from instruction execution systems, devices or devices and execute instructions), or in combination with these instruction execution systems, devices or devices And use. For the purposes of this specification, a "computer-readable medium" may be any device that can contain, store, communicate, propagate or transmit a program for use in or in conjunction with an instruction execution system, device or device. More specific examples (non-exhaustive list) of computer-readable media include the following: electrical connection with one or more wires (electronic device), portable computer disk case (magnetic device), random access memory (RAM), Read Only Memory (ROM), Erasable and Editable Read Only Memory (EPROM or Flash Memory), Fiber Optic Devices, and Portable Compact Disc Read Only Memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program can be printed, as it may be possible, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or other suitable processing if necessary. The program is processed electronically and stored in computer memory.

The above is only the implementation of the application, and does not limit the patent scope of the application. Any equivalent structure or equivalent process conversion made by using the specification and drawings of the application, or directly or indirectly used in other related technologies fields, are all included in the scope of patent protection of this application in the same way.

Claims (14)

1. a roadside tracking method, is characterized in that, described roadside tracking method comprises: Obtain the predicted roadside point set and detected roadside point set at the current moment; performing grouping processing on the detected roadside point set based on the predicted roadside point set, to obtain a grouping result of the detected roadside point set; Obtain the roadside recognition result of whether there is an incomplete roadside on the road the vehicle is driving at the current moment; Based on the curb identification result and the grouping result of the detected curb point set, performing fusion processing on the predicted curb point set and the detected curb point set to obtain a fused curb point set; Curve fitting is performed on the fused roadside point set to obtain a roadside fitting result at the current moment. 2. The roadside tracking method according to claim 1, characterized in that, based on the grouping result of the roadside recognition result and the detected roadside point set, the predicted roadside point set and the Detect roadside point sets for fusion processing, including: determining respective fusion weights of the predicted curb point set and the detected curb point set based on the curb identification result and the grouping result of the detected curb point set; Fusion processing is performed on the predicted roadside point set and the detected roadside point set according to the respective fusion weights. 3. The roadside tracking method according to claim 2, characterized in that, based on the grouping result of the roadside identification result and the detected roadside point set, the predicted roadside point set and the roadside point set are determined. Detect the respective fusion weights of the roadside point sets, including: If there is no new road edge point set in the grouping result of the detected road edge point set and there is no road edge incompleteness in the road edge identification result, then determine the fusion weight of the predicted road edge point set and the detected road edge point set. The difference of fusion weights along the point set is less than or equal to a preset threshold. 4. The roadside tracking method according to claim 2, characterized in that, the grouping result based on the roadside recognition result and the detected roadside point set determines the prediction roadside point set and the Detect the respective fusion weights of the roadside point sets, including: If there is no new road edge point set in the grouping result of the detected road edge point set and there is an incomplete road edge in the road edge recognition result, it is determined that the fusion weight of the predicted road edge point set is greater than the detected road edge point set Fusion weights for point sets. 5. The roadside tracking method according to claim 2, characterized in that, based on the grouping result of the roadside recognition result and the detected roadside point set, the predicted roadside point set and the roadside point set are determined. Detect the respective fusion weights of the roadside point sets, including: If there is a new road edge point set in the grouping result of the detected road edge point set and there is no road edge incompleteness in the road edge identification result, then it is determined that the fusion weight of the predicted road edge point set is smaller than the detected road edge point set Fusion weights for point sets. 6. The roadside tracking method according to claim 2, characterized in that, based on the grouping result of the roadside recognition result and the detected roadside point set, the predicted roadside point set and the roadside point set are determined. Detect the respective fusion weights of the roadside point sets, including: If there is a new road edge point set in the grouping result of the detected road edge point set and there is an incomplete road edge in the road edge recognition result, then determine that the fusion weight of the predicted road edge point set is smaller than the detected road edge point Set of fusion weights. 7. The roadside tracking method according to any one of claims 1-6, wherein the grouping process is performed on the detected roadside point set based on the predicted roadside point set to obtain the detected roadside point set. Grouped results along point sets, including: Select a target roadside point from the detected roadside point set without calculating the distance value and calculate the distance from the left roadside predicted point set in the predicted roadside point set to obtain the left roadside distance value, and Calculate the distance of the right along point set in the set of along points to obtain the distance value of the right along the road, and repeat the current step until all detected road along points are traversed; Selecting the smaller one from the distance value of the left roadside and the distance value of the right roadside of the detected roadside point to compare with a preset distance threshold; Based on the distance comparison result, it is determined whether there is a new roadside point set in the grouping result of the detected roadside point set. 8. The roadside tracking method according to claim 7, wherein the determination based on the distance comparison result whether there is a new roadside point set in the grouping result of the detected roadside point set comprises: If the number of undetermined roadside points whose distance value is greater than the preset distance threshold in the detected roadside points is greater than the preset number threshold, then the minimum value of the ordinate value of all undetermined roadside points in the vehicle coordinate system is compared with the preset The ordinate threshold is compared; If the minimum value of the ordinate values of all undetermined roadside points is less than the preset ordinate threshold value, it is determined that there is a new roadside point set in the grouping result of the detected roadside point set, otherwise it is determined that the detected roadside point set The new curb point set does not exist in the grouping result. 9. The roadside tracking method according to claim 7, characterized in that, selecting a target roadside point whose distance value has not been calculated from the set of detected roadside points and the left side of the set of predicted roadside points Calculate the distance from the roadside prediction point set to obtain the distance value of the left roadside, and calculate the distance from the right roadside point set in the predicted roadside point set to obtain the right roadside distance value, including: Calculate the distance value between the selected target roadside point and all left roadside prediction points in the left roadside prediction point set, and calculate the average value of all distance values as the left roadside distance value; Calculate the distance value between the selected target roadside point and all the right roadside prediction points in the right roadside prediction point set, and calculate the average value of all the distance values as the right roadside distance value. 10. The roadside tracking method according to any one of claims 1-6, wherein said obtaining the predicted roadside point set at the current moment comprises: Determine the lateral displacement, longitudinal displacement and rotation angle of the vehicle from the previous moment to the current moment based on the vehicle motion information at the previous moment; Based on the lateral displacement, the longitudinal displacement, the rotation angle, the translation transformation matrix of the vehicle coordinate system, the rotation transformation matrix of the vehicle coordinate system, and the coordinate values of the fused roadside point in the vehicle coordinate system at the previous moment, determine The set of predicted roadside points at the current moment. 11. according to the roadside tracking method described in any one of claim 1-6, it is characterized in that, described fusion roadside point set is carried out curve fitting, obtains the roadside fitting result of current moment, comprises: selecting some fitting roadside points from the collection of fusion roadside points; Carry out curve fitting with described several fitting roadside points, obtain fitting curve model; Matching other fusion roadside points with the fitting curve model; If the number of fused roadside points whose matching degree is within the preset range is greater than or equal to the preset number threshold, the fitting curve model is used as the roadside fitting result; If the number of fused roadside points whose matching degree is within the preset range is less than the preset number threshold, return to the step of selecting several fitting roadside points from the fused roadside points. 12. A roadside fitting device, comprising: an acquisition module, a grouping module, a fusion module and a fitting module; The obtaining module is used to obtain the predicted roadside point set and the detected roadside point set at the current moment; The grouping module is used to group the detected roadside point set based on the predicted roadside point set, and obtain the grouping result of the detected roadside point set; The acquiring module is used to acquire the curb recognition result of whether there is an incomplete curb on the road the vehicle is driving at the current moment; The fusion module is configured to perform fusion processing on the predicted curb point set and the detected curb point set based on the curb recognition result and the grouping result of the detected curb point set to obtain the fused curb point set; The fitting module is used for performing curve fitting on the fused roadside point set to obtain a roadside fitting result at the current moment. 13. A roadside fitting device, characterized in that it comprises: a processor and a memory, a computer program is stored in the memory, and the processor is used to execute the computer program to realize any one of claims 1 to 11 method described in the item. 14. A computer-readable storage medium on which program instructions are stored, wherein the method according to any one of claims 1 to 11 is implemented when the program instructions are executed by a processor.

Images