CN114394088A

CN114394088A - Parking tracking track generation method and device, electronic equipment and storage medium

Info

Publication number: CN114394088A
Application number: CN202111621759.8A
Authority: CN
Inventors: 李雪
Original assignee: Beijing Yihang Yuanzhi Technology Co Ltd
Current assignee: Beijing Yihang Yuanzhi Technology Co Ltd
Priority date: 2021-12-28
Filing date: 2021-12-28
Publication date: 2022-04-26
Anticipated expiration: 2041-12-28
Also published as: CN114394088B

Abstract

The present disclosure provides a parking tracking trajectory generation method, including: synchronously positioning and drawing to generate an actual driving track and a point cloud map of the vehicle; acquiring parking space information and road characteristic information in the image data based on the acquired image data; judging whether the actual driving track of the vehicle contains an irregular track or not based on the road characteristic information and/or the parking space information; if the actual running track of the vehicle contains an irregular track, the actual running track of the vehicle is corrected based on the road characteristic information and/or the parking space information to generate a vehicle compliant running track; generating a virtual parking space based on the final parking position of the vehicle in the actual driving track of the vehicle to obtain the corrected final parking position of the vehicle; and acquiring a parking tracking track at least based on the vehicle compliant running track and the corrected final parking position of the vehicle. The disclosure also provides a parking tracking track generation device, an electronic device and a readable storage medium.

Description

Parking tracking track generation method and device, electronic equipment and storage medium

Technical Field

The present disclosure relates to the field of automatic driving technologies, and in particular, to a method and an apparatus for generating a parking tracking trajectory, an electronic device, and a storage medium.

Background

The tracking driving of the automobile is a mode of automatic driving (including unmanned driving), the tracking driving refers to the automatic tracking driving of the automobile according to a preset tracking track, and whether the tracking track is correct or not and the quality of the automatic driving is determined to a great extent accurately.

The tracking driving of the automobile comprises automatic parking, the automatic parking needs to generate a parking tracking track based on a vehicle historical driving track, namely an actual driving track of the vehicle, so that automatic parking is carried out based on the parking tracking track in the subsequent automatic parking process. However, the generation of the parking tracking trajectory in the prior art still has some accuracy problems, which affect the reliability of the parking tracking trajectory.

Disclosure of Invention

In order to solve at least one of the above technical problems, the present disclosure provides a parking tracking trajectory generation method, apparatus, electronic device, and storage medium.

According to an aspect of the present disclosure, there is provided a parking track generation method including:

s102, synchronously positioning and mapping (SLAM) based on image data acquired by a vehicle camera device and vehicle running data acquired by a vehicle wheel speed mileage device, and generating an actual vehicle running track and a point cloud map;

s104, acquiring parking space information and road characteristic information in image data based on the acquired image data;

s106, judging whether the actual running track of the vehicle contains an irregular track or not based on the road characteristic information and/or the parking space information;

s108, if the actual running track of the vehicle contains an irregular track, correcting the actual running track of the vehicle (including track translation and fitting correction) based on the road characteristic information and/or the parking space information to generate a vehicle compliant running track (the correcting process also includes track deletion, namely if the correcting processes such as track translation and fitting correction fail, prompt information is generated to prompt a failure reason and delete the corresponding actual running track of the vehicle);

s110, generating a virtual parking space based on the final parking position of the vehicle in the actual driving track of the vehicle, comparing the virtual parking space with the parking space information acquired based on the acquired image data to generate parking space correction information, correcting the final parking position of the vehicle based on the parking space correction information, and acquiring the corrected final parking position of the vehicle;

and S112, acquiring a parking tracking track (namely the suggested parking tracking track) at least based on the vehicle compliant running track and the corrected final parking position of the vehicle.

According to at least one embodiment of the present disclosure, a method for generating a parking tracking track, S104, acquiring parking space information and road characteristic information in image data based on the acquired image data, includes:

s1042, carrying out semantic segmentation processing on the acquired image data, and endowing each pixel in the image data with a semantic category label;

s1044, acquiring an image area formed by pixel points with the same semantic category label and adjacent positions at least based on the semantic category label of each pixel in the image data so as to acquire road characteristic information (road boundary line, lane line, traffic identification (non-motor lane identification, sidewalk identification, bus lane identification, and the like)) in the image data; and at least carrying out parking space angle line characteristic extraction on the image data to acquire parking space information in the image data.

According to the parking tracking trajectory generation method according to at least one embodiment of the present disclosure, step S104 further includes:

s1046, projecting image data acquired by the vehicle camera device to obtain a top view, wherein a coordinate system of the top view is positioned on a vehicle bottom plane;

and S1048, generating a road topological graph containing parking space information and road characteristic information based on the top view, the parking space information and the road characteristic information in the acquired image data.

According to at least one embodiment of the present disclosure, the step S106 of determining whether the actual driving trajectory of the vehicle includes an irregular trajectory based on the road characteristic information and/or the parking space information includes:

judging whether the actual running track of the vehicle compacts the lane line or not based on the road characteristic information;

judging whether a reverse track exists in the actual running track of the vehicle or not based on the road characteristic information;

judging whether the actual running track of the vehicle has a running track of a non-motor lane, a sidewalk and/or a bus lane or not based on the road characteristic information; and/or

And judging whether the actual running track of the vehicle presses other parking spaces or not based on the parking space information.

According to at least one embodiment of the present disclosure, a method for generating a parking tracking track, which determines whether a lane line is compacted by an actual driving track of a vehicle based on road characteristic information, includes:

and respectively carrying out curve fitting (preferably cubic curve fitting or curve fitting more than three times) on the actual driving track of the vehicle and the solid line lane line in the road characteristic information for multiple times, calculating whether the curve of the actual driving track of the vehicle and the curve of the solid line lane line have an intersection, and if the intersection exists, judging as the compaction line lane line.

According to at least one embodiment of the present disclosure, in the parking track generation method, if the actual driving track of the vehicle includes an irregular track, S108, performing a correction process on the actual driving track of the vehicle based on the road characteristic information and/or the parking space information to generate a vehicle compliant driving track includes:

based on the positions of the cross points, taking the cross point in the front of the time sequence as a line pressing starting point, and taking the cross point in the back of the time sequence as a line pressing ending point, and obtaining a line pressing area between the line pressing starting point and the line pressing ending point;

deleting the actual running track of the vehicle in the line pressing area, and fitting the whole vehicle running track to generate a vehicle compliant running track, wherein the road boundary line and/or the lane line in the road characteristic information is used as constraint in the fitting process.

According to at least one embodiment of the present disclosure, a method for generating a parking tracking track, which determines whether a reverse track exists in an actual running track of a vehicle based on road characteristic information, includes:

sampling track points of the actual running track of the vehicle to determine the running direction of each track sampling point in the actual running track of the vehicle;

and comparing the running direction of each sampling point with the direction specified by the road mark arrow in the road characteristic information to judge whether the actual running track of the vehicle has a retrograde motion track.

sequentially detecting the road direction of a reference lane near the current lane of the retrograde track section according to the distance, and acquiring a reference lane, the road direction of which is consistent with the driving direction of a driving track outside the retrograde track section of the actual driving track of the vehicle and has the closest distance with the current lane of the retrograde track section;

and translating the retrograde trajectory segment to the center line of the reference lane.

According to at least one embodiment of the present disclosure, a method for generating a parking tracking track, which determines whether a driving track of a non-motor lane, a sidewalk and/or a bus lane exists in an actual driving track of a vehicle based on road characteristic information, includes:

sampling track points of the actual running track of the vehicle to determine the positions of each track sampling point in the actual running track of the vehicle;

and judging whether each track sampling point violates the driving condition or not based on the lane types in the road characteristic information of the positions of the track sampling points, wherein the lane types at least comprise non-motor lanes, sidewalks and/or bus lanes.

sequentially detecting the road types of reference lanes near the current lane of the violation track section according to the distance, and acquiring the reference lane which is the motor lane and has the closest distance with the current lane of the violation track section;

and translating the violation track segment to the center line of the reference lane.

According to at least one embodiment of the present disclosure, a method for generating a parking tracking trajectory, which determines whether an actual driving trajectory of a vehicle presses another parking space based on parking space information, includes:

and performing curve fitting (preferably cubic curve fitting or curve fitting of more than three times) on the actual running track of the vehicle for multiple times, comparing the curve of the actual running track of the vehicle after curve fitting with the area ranges of other parking spaces around the vehicle in the running process of the vehicle, and if overlapping exists, judging that the actual running track of the vehicle presses other parking spaces.

and acquiring a compliance lane closest to the pressed vehicle location, and fitting by taking a road boundary line corresponding to the compliance lane and a lane line of the compliance lane as constraint conditions between the overlap starting point and the overlap ending point to generate a vehicle compliance driving track comprising a fitting track on the compliance lane.

According to at least one embodiment of the present disclosure, a parking tracking track generation method, in which a virtual parking space is generated based on a final parking position of a vehicle in an actual driving track of the vehicle, the virtual parking space is compared with parking space information acquired based on collected image data to generate parking space correction information, the final parking position of the vehicle is corrected based on the parking space correction information, and a corrected final parking position of the vehicle is obtained, includes:

recording the final parking position of the vehicle, and generating a virtual parking space with the same size as the actual parking space by taking the rear axle of the vehicle as a reference so that the vehicle is positioned in the middle of the virtual parking space;

and comparing the virtual parking space with the adjacent actual parking space, calculating an overlapping area, and correcting the final parking position of the vehicle to the adjacent actual parking space if the area/ratio of the overlapping area is greater than or equal to a preset threshold value.

According to at least one embodiment of the present disclosure, a method for generating a parking track, in which a parking track is obtained based on at least the vehicle compliant travel track and the corrected final parking position of the vehicle, in S112, includes:

and extracting a vehicle compliant running track within a preset distance range of the corrected final parking position of the vehicle as the parking tracking track based on the corrected final parking position of the vehicle.

According to the parking track generation method of at least one embodiment of the present disclosure, S112, obtaining a parking track based on at least the vehicle compliant travel track and the corrected final parking position of the vehicle, further includes:

and generating a parking area (an area range frame capable of being directly parked in the parking space) based on the corrected final parking position of the vehicle, and if only one track section appears in the parking area, judging the parking tracking track as a reasonable parking track.

According to the parking tracking trajectory generation method of at least one embodiment of the present disclosure, if two or more trajectory sections are present in the parking area, the parking tracking trajectory is corrected:

sampling track points on each track segment;

sequentially judging whether each track sampling point can be directly parked into the corrected final parking position of the vehicle according to the time sequence;

and taking the foremost track sampling point in the time sequence as a parking point, deleting the track point of the actual running track of the vehicle after the parking point time sequence as the corrected parking tracking track.

A parking track generation method according to at least one embodiment of the present disclosure further includes:

and S114, optimizing the path of the generated parking tracking track based on the automobile kinematic model, and optimizing the speed of the parking tracking track after path optimization to obtain the optimized parking tracking track.

According to another aspect of the present disclosure, there is provided a parking track generation device including:

the synchronous positioning and mapping module carries out synchronous positioning and mapping (SLAM) based on image data acquired by a vehicle camera device and vehicle running data acquired by a vehicle wheel speed mileage device to generate a vehicle actual running track and a point cloud map;

the information extraction module acquires parking space information and road characteristic information in the image data based on the acquired image data;

the first judgment module judges whether the actual driving track of the vehicle contains an irregular track or not based on the road characteristic information and/or the parking space information;

a first correction module, configured to, if the actual vehicle driving trajectory includes an irregular trajectory, perform correction processing (including trajectory translation and fitting correction) on the actual vehicle driving trajectory based on the road characteristic information and/or the parking space information to generate a vehicle compliant driving trajectory (the correction processing further includes trajectory deletion, that is, if the correction processing such as trajectory translation and fitting correction fails, generate prompt information to prompt a failure reason and delete a corresponding actual vehicle driving trajectory);

the parking position obtaining module generates a virtual parking space based on a final parking position of a vehicle in an actual driving track of the vehicle, compares the virtual parking space with parking space information obtained based on the acquired image data to generate parking space correction information, and corrects the final parking position of the vehicle based on the parking space correction information to obtain a corrected final parking position of the vehicle;

and the parking tracking track generating module is used for acquiring a parking tracking track at least based on the vehicle compliant running track and the corrected final parking position of the vehicle.

A parking track generation device according to at least one embodiment of the present disclosure further includes:

and the tracking track optimization module is used for optimizing the path of the generated parking tracking track based on the automobile kinematics model and optimizing the speed of the parking tracking track after path optimization to obtain the optimized parking tracking track.

According to yet another aspect of the present disclosure, there is provided an electronic device including: a memory storing execution instructions; and a processor executing execution instructions stored by the memory to cause the processor to perform any of the methods described above.

According to yet another aspect of the present disclosure, there is provided a readable storage medium having stored therein execution instructions for implementing any of the above methods when executed by a processor.

Drawings

The accompanying drawings, which are included to provide a further understanding of the disclosure and are incorporated in and constitute a part of this specification, illustrate exemplary embodiments of the disclosure and together with the description serve to explain the principles of the disclosure.

Fig. 1 is a flowchart illustrating a parking track generation method according to an embodiment of the present disclosure.

Fig. 2 is a schematic flow chart illustrating a method for acquiring parking space information and road characteristic information in a parking tracking track generation method according to an embodiment of the present disclosure.

Fig. 3 is a flowchart illustrating a method for acquiring parking space information and road characteristic information in a parking track generation method according to still another embodiment of the present disclosure.

Fig. 4 shows a case where the lane lines of the compaction line are compacted in the actual running track of the vehicle.

Fig. 5 shows a situation where the actual travel trajectory of the vehicle is pressed against another parking space.

Fig. 6 is a schematic diagram of a vehicle final parking position after obtaining a correction based on a vehicle final parking position in an actual driving trajectory of the vehicle according to an embodiment of the present disclosure.

Fig. 7 to 8 show the presence of two track segments and the process of correcting the parking tracking track.

Fig. 9 is an overall process diagram of the parking tracking track generation method according to the present disclosure.

Fig. 10 is a flowchart illustrating a parking track generation method according to still another embodiment of the present disclosure.

Fig. 11 is a block diagram schematically illustrating a configuration of a parking track generation device implemented by hardware using a processing system according to an embodiment of the present disclosure.

Description of the reference numerals

1000 parking tracking track generating device

1002 synchronous positioning and mapping module

1004 information extraction module

1006 first judging module

1008 first correction module

1010 parking position acquisition module

1012 parking tracking track generation module

1014 tracking trajectory optimization module

1100 bus

1200 processor

1300 memory

1400 and other circuits.

Detailed Description

The present disclosure will be described in further detail with reference to the drawings and embodiments. It is to be understood that the specific embodiments described herein are for purposes of illustration only and are not to be construed as limitations of the present disclosure. It should be further noted that, for the convenience of description, only the portions relevant to the present disclosure are shown in the drawings.

It should be noted that the embodiments and features of the embodiments in the present disclosure may be combined with each other without conflict. Technical solutions of the present disclosure will be described in detail below with reference to the accompanying drawings in conjunction with embodiments.

Unless otherwise indicated, the illustrated exemplary embodiments/examples are to be understood as providing exemplary features of various details of some ways in which the technical concepts of the present disclosure may be practiced. Accordingly, unless otherwise indicated, features of the various embodiments may be additionally combined, separated, interchanged, and/or rearranged without departing from the technical concept of the present disclosure.

The use of cross-hatching and/or shading in the drawings is generally used to clarify the boundaries between adjacent components. As such, unless otherwise noted, the presence or absence of cross-hatching or shading does not convey or indicate any preference or requirement for a particular material, material property, size, proportion, commonality between the illustrated components and/or any other characteristic, attribute, property, etc., of a component. Further, in the drawings, the size and relative sizes of components may be exaggerated for clarity and/or descriptive purposes. While example embodiments may be practiced differently, the specific process sequence may be performed in a different order than that described. For example, two processes described consecutively may be performed substantially simultaneously or in reverse order to that described. In addition, like reference numerals denote like parts.

When an element is referred to as being "on" or "on," "connected to" or "coupled to" another element, it can be directly on, connected or coupled to the other element or intervening elements may be present. However, when an element is referred to as being "directly on," "directly connected to" or "directly coupled to" another element, there are no intervening elements present. For purposes of this disclosure, the term "connected" may refer to physically, electrically, etc., and may or may not have intermediate components.

The terminology used herein is for the purpose of describing particular embodiments and is not intended to be limiting. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. Furthermore, when the terms "comprises" and/or "comprising" and variations thereof are used in this specification, the presence of stated features, integers, steps, operations, elements, components and/or groups thereof are stated but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and/or groups thereof. It is also noted that, as used herein, the terms "substantially," "about," and other similar terms are used as approximate terms and not as degree terms, and as such, are used to interpret inherent deviations in measured values, calculated values, and/or provided values that would be recognized by one of ordinary skill in the art.

The parking tracking track generation method, the parking tracking track generation device, the electronic apparatus, and the readable storage medium according to the present disclosure are described in detail below with reference to fig. 1 to 11.

Referring to fig. 1, a parking track generation method S100 according to an embodiment of the present disclosure includes:

s104, acquiring parking space information and road characteristic information in the image data based on the acquired image data;

s106, judging whether the actual driving track of the vehicle contains an irregular track or not based on the road characteristic information and/or the parking space information;

s108, if the actual running track of the vehicle contains an irregular track, correcting the actual running track of the vehicle (including track translation and fitting correction) based on the road characteristic information and/or the parking space information to generate a vehicle compliant running track (the correcting process also includes track deletion, namely if the correcting processes such as track translation and fitting correction fail, prompt information is generated to prompt the reason of the failure and delete the corresponding actual running track of the vehicle);

s110, generating a virtual parking space based on the final parking position of the vehicle in the actual driving track of the vehicle, comparing the virtual parking space with parking space information acquired based on the acquired image data to generate parking space correction information, correcting the final parking position of the vehicle based on the parking space correction information, and acquiring the corrected final parking position of the vehicle;

In the method for generating the parking tracking track, when a manually driven automobile runs and parks on a road, the surrounding environment information of the automobile and the running state information of the automobile (namely the running data of the automobile) are captured preferably by four fisheye cameras (automobile camera devices) and a wheel speed odometer which are arranged at the lower edges of a front rear bumper and a left rear-view mirror and a right rear-view mirror of the automobile, a point cloud map and the actual running track of the automobile are built in real time and stored based on synchronous positioning and mapping (SLAM), and information (comprising the running path/route information of the automobile and the like) required by SLAM repositioning and tracking when the map is built is determined in the map is preferred.

In the driving process, feature point extraction is carried out based on a fisheye camera image, preferably a BRIEF descriptor is calculated based on an Oriented FAST key point of the image, ORB feature points are extracted, a feature point cloud map is constructed by using the feature points, and the method comprises the following steps:

map initialization: and adding the two frames of images into the map, and generating a new feature point in the map for each pair of matched ORB feature points in the two frames, wherein the information comprises the positions of the key points and the attributes of the descriptors.

And (3) updating the map: and when the current frame image and the last key frame move for a certain distance, adding the current frame image into the map. And when a new ORB feature is added into the map, sequentially updating key points and descriptors of the map feature according to the matching relationship.

Map optimization: and optimizing by using the pose of the last frame and the local map thereof and the position of the feature point which can be observed by the local map as variables and the minimum reprojection error as an objective function and preferably by using Bundle Adjustment to obtain more accurate positions of the pose of the key frame and the feature point of the map.

Meanwhile, the length (or track information) of the motion of the vehicle body and the course angle information are sensed based on the wheel speed odometer, and the sensed length (or track information) is used as the track constraint of the camera to optimize the track generated by the fisheye camera.

Preferably, the first constraint module constrains the camera motion increment with the same motion increment generated from the time i to the time j by the wheel speed odometry data so as to constrain the vehicle translation vector; the second constraint module takes the reading difference of the wheel speed odometer or the heading of the vehicle body as the constraint of the rotation increment so as to constrain the vehicle rotation vector.

And calculating to obtain SIM (3) mapping of the two tracks, namely integral translation rotation and scale mapping of the two tracks, and mapping the map track obtained by the wheel speed odometer to the map track of the fisheye camera. The method solves the problem that the scale of the monocular camera is uncertain, and can also prevent the accumulation of the positioning errors and the scale drift of the camera, so that the constructed feature point cloud map is more accurate.

For the parking tracking track generation method S100 according to the above embodiment, preferably, the step S104 of acquiring parking space information and road characteristic information in the image data based on the acquired image data includes:

s1042, carrying out semantic segmentation processing on the acquired image data, and endowing each pixel in the image data with a semantic class label (the semantic class label preferably comprises a road edge, a lane line, a traffic identification (a non-motor lane identification, a sidewalk identification, a bus lane identification and the like)) to the image data;

s1044, acquiring an image area formed by pixel points with the same semantic category label and adjacent positions at least based on the semantic category label of each pixel in the image data so as to acquire road characteristic information (road boundary line, lane line, traffic identification (non-motor lane identification, sidewalk identification, bus lane identification, and the like)) in the image data; and at least extracting the parking space angle line characteristics of the image data to acquire parking space information in the image data.

Fig. 2 shows a flow of a method for acquiring parking space information and road characteristic information in a parking track generation method according to an embodiment of the present disclosure.

In the semantic segmentation processing in the embodiment, preferably, the image data acquired by the fisheye camera is subjected to semantic segmentation processing by using a deep learning method, and each pixel in the image is subjected to semantic category labeling processing, and is preferably divided into semantic category labels such as a road edge, a lane line, various traffic identification lines, a parking space line, and the like, so that each pixel of each output image includes the semantic category label. The image semantic segmentation process described above can be implemented using a full convolutional neural network (FCN) that replaces the fully-connected layer of a traditional convolutional network with a convolutional layer, a UNet network based on an encoder-decoder structure, or a deep lab network using pyramid hole pooling and a fully-connected Conditional Random Field (CRF).

For the image region in step S1042, which is composed of pixels with the same semantic category label and adjacent positions, based on at least the semantic category label of each pixel in the image data, to obtain road characteristic information (lane line, traffic identifier (non-motor lane identifier, sidewalk identifier, bus lane identifier, etc.)) in the image data, preferably, a Connected Component Analysis (CCA) is used to divide the pixel set in the semantic segmentation image into image regions composed of pixels with the same semantic category label and adjacent positions, to determine the lane line and the road identifier (i.e., road characteristic information) in the image data, i.e., perform clustering.

Preferably, the clustering result is reclassified by using an SVM algorithm or a random tree, the lane lines are reclassified as yellow dotted lines, white dotted lines, yellow solid lines, white solid lines, double yellow solid lines, double white solid lines, etc., and the road signs are reclassified as arrows (preferably determining the designated directions thereof, such as straight, straight and turning, turning around), non-motor lane signs, sidewalk signs, and bus lane signs.

For the parking space angle feature extraction at least performed on the image data in step S1042 to obtain the parking space information in the image data, preferably, at least one target parking space candidate frame is generated based on the parking space angle feature, preferably, the fast RCNN network is used to perform feature extraction on the target parking space candidate frame, and an energy loss function of the target parking space candidate frame is calculated in combination with the obtained semantic category label (preferably, also in combination with the position prior information and the target shape prior information) to obtain accurate parking space information (i.e., the target parking space).

According to the parking tracking trajectory generation method S100 of the preferred embodiment of the present disclosure, the step S104 further includes:

and S1048, generating a road topological graph containing the parking space information and the road characteristic information based on the top view, the parking space information and the road characteristic information in the acquired image data.

Fig. 3 shows a flow of a method for acquiring parking space information and road characteristic information in a parking track generation method according to still another embodiment of the present disclosure.

In step S1046, preferably, based on the internal reference and the calibrated external reference of the camera device, the image data acquired by the camera device is projected to the top view to perform mapping between the original image data acquired by the camera device and the pixels of the top view, that is, mapping from a camera coordinate system to a top view coordinate system.

Illustratively, four fisheye cameras are taken as a vehicle camera device as an example, image data collected by the fisheye cameras from front view, rear view and left and right side view are rasterized, coordinate information from sampling points on the top view to original image pixels is calculated by using internal parameters of the fisheye cameras and external parameters of the fisheye cameras relative to the origin of a coordinate system of the top view, and the top view obtained by projecting images of the fisheye cameras is calculated. Preferably, the top view coordinate system is located on the vehicle bottom plane, and the center of the vehicle is taken as an origin, and the horizontal forward direction and the horizontal rightward direction are positive directions of two coordinate axes respectively.

When external reference calibration is performed, the vehicle is preferably stationary on a flat road surface, and is pickedAnd directly acquiring the corresponding relation from the image of the fisheye camera to the coordinate system of the top view by using a calibration plate, and calculating and generating a mapping table of the original image and the pixels of the top view of each camera, wherein (xp, yp) represents the coordinate of the corrected image (namely the top view), and (xf, yf) represents the coordinate of the fisheye camera. From the calibrated 4 pairs of feature points (in the case of four fisheye cameras) 8 unknown parameters C can be calculated₀₀-C₂₁Thus, a homography matrix is obtained:

the top view transformation is done based on the homography matrix ROI.

And mapping the parking space information and the road characteristic information to a physical space of a top view coordinate system based on a mapping table of the original image and the top view pixels of each camera to obtain a road topological graph containing the parking space information and the road characteristic information.

For the parking tracking trajectory generation method S100 according to each of the above embodiments, preferably, the determining, at S106, whether the actual driving trajectory of the vehicle includes an irregular trajectory based on the road characteristic information and/or the parking space information includes:

judging whether the actual running track of the vehicle has a retrograde motion track or not based on the road characteristic information;

judging whether the actual running track of the vehicle has the running track of a non-motor lane, a sidewalk and/or a bus lane or not based on the road characteristic information; and/or judging whether the actual running track of the vehicle presses other parking spaces or not based on the parking space information.

With respect to the parking tracking track generation method S100 of the above embodiment, preferably, the determining whether the actual driving track of the vehicle compacts the lane line based on the road characteristic information includes:

With respect to the parking track generation method S100 of the above embodiment, preferably, if the actual driving track of the vehicle includes an irregular track, S108, performing a correction process on the actual driving track of the vehicle based on the road characteristic information and/or the parking space information to generate a compliant driving track of the vehicle includes:

based on the positions of the cross points, taking the cross point in the front of the time sequence as a line pressing starting point, taking the cross point in the back of the time sequence as a line pressing ending point, and obtaining a line pressing area between the line pressing starting point and the line pressing ending point;

More preferably, the determining whether the actual driving track of the vehicle has a retrograde motion track based on the road characteristic information includes:

and comparing the driving direction of each sampling point with the direction specified by the road mark arrow in the road characteristic information to judge whether the actual driving track of the vehicle has a retrograde motion track.

According to a preferred embodiment of the present disclosure, if a retrograde trajectory exists, a retrograde motion start trajectory point and a retrograde motion end trajectory point of a retrograde trajectory segment are extracted.

According to a preferred embodiment of the present disclosure, a parking tracking track generation method S100, S108, if an actual driving track of a vehicle includes an irregular track, performing a correction process on the actual driving track of the vehicle based on road characteristic information and/or parking space information to generate a compliant driving track of the vehicle, includes:

According to a preferred embodiment of the present disclosure, the vehicle compliant travel track generated after the translation process is smoothed.

According to a preferred embodiment of the present disclosure, if a compliant reference lane is not detected, it is determined that the correction has failed, and corresponding prompt information is generated to prompt a failure cause, preferably, and the corresponding actual travel track of the vehicle is deleted.

According to a preferred embodiment of the present disclosure, a parking tracking track generation method S100, wherein determining whether a driving track of a non-motor lane, a sidewalk and/or a bus lane exists in an actual driving track of a vehicle based on road characteristic information includes:

and judging whether each track sampling point violates the driving rule or not based on the lane type in the road characteristic information of the position of each track sampling point, wherein the lane type at least comprises a non-motor lane, a sidewalk and/or a bus lane.

According to the preferred embodiment of the present disclosure, if there is an illegal driving, an illegal start track point and an illegal end track point of an illegal track segment are extracted.

For the parking tracking trajectory generation method S100 according to each of the above embodiments, preferably, if the actual driving trajectory of the vehicle includes an irregular trajectory, S108, performing a correction process on the actual driving trajectory of the vehicle based on the road characteristic information and/or the parking space information to generate a compliant driving trajectory of the vehicle includes:

According to a preferred embodiment of the present disclosure, if no lane of a motor vehicle is detected, it is determined that the correction has failed, and corresponding prompt information is generated to prompt the cause of the failure, preferably, and the corresponding actual travel track of the vehicle is deleted.

As for the parking tracking trajectory generation method S100 according to each of the above embodiments, preferably, the determining whether the actual traveling trajectory of the vehicle is pressed against another parking space based on the parking space information includes:

According to a preferred embodiment of the present disclosure, an overlap start point and an overlap end point of the vehicle actual travel locus of the overlap portion are extracted.

and acquiring a compliance lane closest to the pressed vehicle location, and fitting by taking a road boundary line corresponding to the compliance lane and a lane line of the compliance lane as constraint conditions between the overlap starting point and the overlap ending point to generate a vehicle compliance driving track comprising the fitting track on the compliance lane.

According to a preferred embodiment of the present disclosure, if a compliant lane is not detected, it is determined that the correction has failed, and corresponding prompt information is generated to prompt a cause of the failure, preferably, and the corresponding actual travel track of the vehicle is deleted.

In the case of failed correction, generally, due to illegal driving, part of roads cannot be detected and identified, so that a complete map cannot be generated for trajectory correction.

For the parking tracking track generation method S100 according to each of the above embodiments, preferably, the step S110 of generating a virtual parking space based on the final parking position of the vehicle in the actual driving track of the vehicle, comparing the virtual parking space with parking space information obtained based on the collected image data to generate parking space correction information, and correcting the final parking position of the vehicle based on the parking space correction information to obtain a corrected final parking position of the vehicle includes:

Fig. 6 is a schematic diagram of a corrected final parking position of the vehicle obtained based on the final parking position of the vehicle in the actual travel trajectory of the vehicle.

According to a preferred embodiment of the present disclosure, a method S100, S112 for generating a parking track based on at least a vehicle compliant driving track and a corrected final parking position of a vehicle includes:

and extracting the vehicle compliant running track within the preset distance range of the corrected final parking position of the vehicle as a parking tracking track based on the corrected final parking position of the vehicle.

Preferably, S112, obtaining a parking tracking track based on at least the vehicle compliant running track and the corrected final parking position of the vehicle, further includes:

and generating a parking area (namely an area range frame capable of directly parking in the parking space) based on the corrected final parking position of the vehicle, and if only one track section appears in the parking area, judging the parking tracking track as a reasonable parking track.

More preferably, if more than two track segments appear in the parking area, the parking tracking track correction is performed:

sampling track points on each track segment;

and taking the foremost track sampling point in the time sequence as a parking point, deleting the track point of the actual running track of the vehicle after the parking point time sequence, and taking the track point as the corrected parking tracking track.

In the embodiment, the automatic parking is started from the parking point in the subsequent automatic driving process of the vehicle through the obtained corrected parking tracking track.

With respect to the parking tracking trajectory generation method S100 of each of the above embodiments, referring to fig. 10, it is preferable that the method further includes:

Preferably, the automobile is simplified into a two-wheel two-degree-of-freedom steering model, based on the automobile kinematic model, under the condition of a certain speed, the position and the heading of the automobile after a period of time can be presumed by inputting different front wheel steering angles, and a track cluster can be generated by using the different front wheel steering angles. And uniformly and discretely generating a group of (n) corners in a certain range according to the actual corners of the front wheels at the previous moment as a reference, and deducing the position of the automobile passing through m planning cycles according to the group of corners.

The closeness of each path to the planned path is evaluated by calculating an index w as follows:

the path corresponding to the minimum w is found from the group w1, w2, w 3.

On a given path curve, on the premise of meeting the operation limit of feedback control and meeting the output result of behavior decision, the path point is endowed with speed and acceleration information. The speed may preferably be generated for path-specific linear accelerations or interpolated using a cubic spline function of speed with respect to time by dividing the time domain into intervals.

A parking tracking trajectory generation device 1000 according to an embodiment of the present disclosure includes:

a synchronous positioning and mapping module 1002, wherein the synchronous positioning and mapping module 1002 carries out synchronous positioning and mapping (SLAM) based on image data acquired by a vehicle camera device and vehicle running data acquired by a vehicle wheel speed mileage device, and generates a vehicle actual running track and a point cloud map;

the information extraction module 1004 is used for acquiring parking space information and road characteristic information in the image data based on the acquired image data;

the first judgment module 1006, the first judgment module 1006 judges whether the actual driving track of the vehicle contains an irregular track based on the road characteristic information and/or the parking space information;

the first correction module 1008 is used for correcting the actual running track of the vehicle based on the road characteristic information and/or the parking space information to generate a vehicle compliant running track if the actual running track of the vehicle contains an irregular track;

the parking position obtaining module 1010 is used for generating a virtual parking space based on the final parking position of the vehicle in the actual driving track of the vehicle, comparing the virtual parking space with parking space information obtained based on the acquired image data to generate parking space correction information, and correcting the final parking position of the vehicle based on the parking space correction information to obtain the corrected final parking position of the vehicle;

the parking tracking track generation module 1012, the parking tracking track generation module 1012 obtains the parking tracking track based on at least the vehicle compliant running track and the corrected final parking position of the vehicle.

The parking track generation device 1000 according to the preferred embodiment of the present disclosure further includes:

the tracking track optimizing module 1014, the tracking track optimizing module 1014 performs path optimization on the generated parking tracking track based on the automobile kinematics model, and performs speed optimization on the parking tracking track after the path optimization, so as to obtain the optimized parking tracking track.

The parking tracking trajectory generation device 1000 of the present disclosure may be implemented in the form of a computer software architecture.

Fig. 11 illustrates a parking tracking trajectory generation apparatus employing a hardware implementation of a processing system.

The parking tracking trajectory generation apparatus 1000 may include corresponding modules that perform each or several steps of the flowcharts described above. Thus, each step or several steps in the above-described flow charts may be performed by a respective module, and the apparatus may comprise one or more of these modules. The modules may be one or more hardware modules specifically configured to perform the respective steps, or implemented by a processor configured to perform the respective steps, or stored within a computer-readable medium for implementation by a processor, or by some combination.

The hardware architecture may be implemented using a bus architecture. The bus architecture may include any number of interconnecting buses and bridges depending on the specific application of the hardware and the overall design constraints. The bus 1100 couples various circuits including the one or more processors 1200, the memory 1300, and/or the hardware modules together. The bus 1100 may also connect various other circuits 1400, such as peripherals, voltage regulators, power management circuits, external antennas, and the like.

The bus 1100 may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one connection line is shown, but no single bus or type of bus is shown.

Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or steps of the process, and the scope of the preferred embodiments of the present disclosure includes other implementations in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the implementations of the present disclosure. The processor performs the various methods and processes described above. For example, method embodiments in the present disclosure may be implemented as a software program tangibly embodied in a machine-readable medium, such as a memory. In some embodiments, some or all of the software program may be loaded and/or installed via memory and/or a communication interface. When the software program is loaded into memory and executed by a processor, one or more steps of the method described above may be performed. Alternatively, in other embodiments, the processor may be configured to perform one of the methods described above by any other suitable means (e.g., by means of firmware).

The logic and/or steps represented in the flowcharts or otherwise described herein may be embodied in any readable storage medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions.

For the purposes of this description, a "readable storage medium" can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of the readable storage medium include the following: an electrical connection (electronic device) having one or more wires, a portable computer diskette (magnetic device), a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable read-only memory (CDROM). In addition, the readable storage medium may even be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, via for instance optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner if necessary, and then stored in the memory.

It should be understood that portions of the present disclosure may be implemented in hardware, software, or a combination thereof. In the above embodiments, the various steps or methods may be implemented in software stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one or combination of the following techniques, which are known in the art, may be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application specific integrated circuit having an appropriate combinational logic gate circuit, a Programmable Gate Array (PGA), a Field Programmable Gate Array (FPGA), or the like.

It will be understood by those skilled in the art that all or part of the steps of the method implementing the above embodiments may be implemented by hardware that is instructed to be associated with a program, which may be stored in a readable storage medium, and which, when executed, includes one or a combination of the steps of the method embodiments.

In addition, each functional unit in the embodiments of the present disclosure may be integrated into one processing module, or each unit may exist alone physically, or two or more units are integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode. The integrated module, if implemented in the form of a software functional module and sold or used as a separate product, may also be stored in a readable storage medium. The storage medium may be a read-only memory, a magnetic or optical disk, or the like.

The present disclosure also provides an electronic device, including: a memory storing execution instructions; and a processor or other hardware module that executes the execution instructions stored by the memory, causing the processor or other hardware module to perform the above-described methods.

The present disclosure also provides a readable storage medium having stored therein execution instructions, which when executed by a processor, are used to implement the above-mentioned method.

In the description herein, reference to the description of the terms "one embodiment/implementation," "some embodiments/implementations," "an example," "a specific example," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment/implementation or example is included in at least one embodiment/implementation or example of the present application. In this specification, the schematic representations of the terms described above are not necessarily the same embodiment/mode or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments/modes or examples. Furthermore, the various embodiments/aspects or examples and features of the various embodiments/aspects or examples described in this specification can be combined and combined by one skilled in the art without conflicting therewith.

Furthermore, the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present application, "plurality" means at least two, e.g., two, three, etc., unless specifically limited otherwise.

It will be understood by those skilled in the art that the foregoing embodiments are merely for clarity of illustration of the disclosure and are not intended to limit the scope of the disclosure. Other variations or modifications may occur to those skilled in the art, based on the foregoing disclosure, and are still within the scope of the present disclosure.

Claims

1. A parking tracking trajectory generation method, comprising:

s102, synchronously positioning and establishing a map based on image data acquired by a vehicle camera device and vehicle running data acquired by a vehicle wheel speed mileage device to generate an actual vehicle running track and a point cloud map;

s108, if the actual running track of the vehicle contains an irregular track, correcting the actual running track of the vehicle based on the road characteristic information and/or the parking space information to generate a vehicle compliant running track;

s110, generating a virtual parking space based on the final parking position of the vehicle in the actual driving track of the vehicle, comparing the virtual parking space with the parking space information acquired based on the acquired image data to generate parking space correction information, correcting the final parking position of the vehicle based on the parking space correction information, and acquiring the corrected final parking position of the vehicle; and

and S112, obtaining a parking tracking track at least based on the vehicle compliant running track and the corrected final parking position of the vehicle.

2. The method for generating the parking tracking track according to claim 1, wherein the step S104 of acquiring the parking space information and the road characteristic information in the image data based on the acquired image data includes:

s1042, carrying out semantic segmentation processing on the acquired image data, and endowing each pixel in the image data with a semantic category label; and

s1044, acquiring an image area formed by pixel points with the same semantic category label and adjacent positions at least based on the semantic category label of each pixel in the image data to acquire road characteristic information in the image data; and at least carrying out parking space angle line characteristic extraction on the image data to acquire parking space information in the image data.

3. The vehicle parking tracking trajectory generation method according to claim 2, wherein step S104 further includes:

s1046, projecting image data acquired by the vehicle camera device to obtain a top view, wherein a coordinate system of the top view is positioned on a vehicle bottom plane; and

4. The method for generating the parking tracking track according to claim 3, wherein the step S106 of determining whether the actual driving track of the vehicle includes an irregular track based on the road characteristic information and/or the parking space information includes:

5. The vehicle parking tracking track generation method according to claim 4, wherein determining whether the actual travel track of the vehicle compacts a lane line based on the road characteristic information includes:

and respectively carrying out curve fitting on the actual driving track of the vehicle and the solid line lane line in the road characteristic information for multiple times, calculating whether the actual driving track curve of the vehicle and the solid line lane line curve have an intersection, and if the actual driving track curve of the vehicle and the solid line lane line curve have the intersection, judging as the compaction line lane line.

6. The method for generating a parking tracking track according to claim 5, wherein the step S108, if the actual driving track of the vehicle includes an irregular track, performing a correction process on the actual driving track of the vehicle based on the road characteristic information and/or the parking space information to generate a vehicle compliant driving track, comprises:

based on the positions of the cross points, taking the cross point in the front of the time sequence as a line pressing starting point, and taking the cross point in the back of the time sequence as a line pressing ending point, and obtaining a line pressing area between the line pressing starting point and the line pressing ending point; and

7. The method for generating a parking tracking track according to claim 4, wherein determining whether there is a retrograde trajectory in the actual travel track of the vehicle based on the road characteristic information includes:

comparing the driving direction of each sampling point with the direction specified by the road identification arrow in the road characteristic information to judge whether the actual driving track of the vehicle has a retrograde motion track;

preferably, in S108, if the actual driving track of the vehicle includes an irregular track, performing a correction process on the actual driving track of the vehicle based on the road characteristic information and/or the parking space information to generate a vehicle compliant driving track, including:

translating the retrograde trajectory segment to the center line of the reference lane;

preferably, the determining whether the actual driving track of the vehicle has a driving track of a non-motor lane, a sidewalk and/or a bus lane based on the road characteristic information includes:

judging whether each track sampling point violates the driving rule or not based on the lane types in the road characteristic information of the positions of the track sampling points, wherein the lane types at least comprise a non-motor lane, a sidewalk and/or a bus lane;

translating the violation track segment to the center line of the reference lane;

preferably, the determining whether the actual driving track of the vehicle presses another parking space based on the parking space information includes:

performing curve fitting on the actual running track of the vehicle for multiple times, comparing the curve of the actual running track of the vehicle after curve fitting with the area ranges of other parking spaces around the vehicle in the running process of the vehicle, and if overlapping exists, judging that the actual running track of the vehicle presses other parking spaces;

acquiring a compliance lane closest to the pressed vehicle location, and fitting by taking a road boundary line corresponding to the compliance lane and a lane line of the compliance lane as constraint conditions between an overlapping starting point and an overlapping terminal point to generate a vehicle compliance driving track comprising a fitting track on the compliance lane;

preferably, in S110, generating a virtual parking space based on a final parking position of a vehicle in an actual driving trajectory of the vehicle, comparing the virtual parking space with the parking space information obtained based on the acquired image data to generate parking space correction information, and correcting the final parking position of the vehicle based on the parking space correction information to obtain a corrected final parking position of the vehicle, including:

comparing the virtual parking space with an adjacent actual parking space, calculating an overlapping area, and correcting the final parking position of the vehicle to the adjacent actual parking space if the area/ratio of the overlapping area is greater than or equal to a preset threshold value;

preferably, the step S112 of obtaining a parking tracking track based on at least the vehicle compliant running track and the corrected final parking position of the vehicle includes:

extracting a vehicle compliant running track within a preset distance range of the corrected final parking position of the vehicle to be used as the parking tracking track based on the corrected final parking position of the vehicle;

preferably, S112, obtaining a parking tracking track based on at least the vehicle compliant driving track and the corrected final parking position of the vehicle, further includes:

generating a parking area based on the corrected final parking position of the vehicle, and if the parking area only has one track section, judging the parking tracking track as a reasonable parking track;

preferably, if more than two track segments appear in the parking area, the parking tracking track correction is performed:

sampling track points on each track segment;

taking the foremost track sampling point in the time sequence as a parking point, deleting the track point of the actual running track of the vehicle after the parking point time sequence as a corrected parking tracking track;

preferably, the method further comprises the following steps:

8. A parking tracking trajectory generation device, comprising:

the synchronous positioning and mapping module carries out synchronous positioning and mapping on the basis of image data acquired by a vehicle camera device and vehicle running data acquired by a vehicle wheel speed mileage device to generate a vehicle actual running track and a point cloud map;

the first correction module is used for correcting the actual running track of the vehicle based on the road characteristic information and/or the parking space information to generate a vehicle compliant running track if the actual running track of the vehicle contains an irregular track;

the parking position obtaining module generates a virtual parking space based on a final parking position of a vehicle in an actual driving track of the vehicle, compares the virtual parking space with parking space information obtained based on the acquired image data to generate parking space correction information, and corrects the final parking position of the vehicle based on the parking space correction information to obtain a corrected final parking position of the vehicle; and

a parking tracking track generation module, which obtains a parking tracking track at least based on the vehicle compliant running track and the corrected final parking position of the vehicle;

preferably, the method further comprises the following steps:

9. An electronic device, comprising:

a memory storing execution instructions; and

a processor executing execution instructions stored by the memory to cause the processor to perform the method of any of claims 1 to 7.

10. A readable storage medium having stored therein execution instructions, which when executed by a processor, are configured to implement the method of any one of claims 1 to 7.