CN113734157A

CN113734157A - Memory parking method, device, equipment, storage medium and program product

Info

Publication number: CN113734157A
Application number: CN202111117612.5A
Authority: CN
Inventors: 赵大成; 赵俊
Original assignee: Hubei Ecarx Technology Co Ltd
Current assignee: Ecarx Hubei Tech Co Ltd
Priority date: 2021-09-23
Filing date: 2021-09-23
Publication date: 2021-12-03
Anticipated expiration: 2041-09-23
Also published as: CN113734157B

Abstract

The embodiment of the application provides a memory parking method, a device, equipment, a storage medium and a program product, wherein the memory parking method comprises the following steps: judging whether the vehicle is located within a preset range of a preset starting point or not based on a vehicle-mounted ring view of the vehicle; if so, acquiring a reference track of the preset starting point; planning a driving track of the vehicle according to the reference track so as to control the vehicle to run to a parking point corresponding to the preset starting point; according to the parking points, the vehicle is controlled to park in the target parking position corresponding to the preset starting point, the vehicle is repositioned based on the vehicle-mounted environment view, hardware and software resources for repositioning are less, engineering complexity and cost are reduced, positioning is accurate and rapid, and parking memory efficiency is improved.

Description

Memory parking method, device, equipment, storage medium and program product

Technical Field

The embodiment of the application relates to the technical field of automatic parking, in particular to a memory parking method, a memory parking device, memory parking equipment, a memory parking storage medium and a program product.

Background

The automatic parking technology provides a convenient and quick parking mode for a driver. Among the automatic parking techniques, the emerging memory parking technique is widely used.

In the memory parking, the route of the driver in the parking process is automatically learned, when the vehicle drives into the field again, the vehicle can be positioned through environment sensing, the vehicle is controlled to drive to the memorized parking route, and the vehicle is parked into the parking space through the route.

In the existing memory parking system, when a vehicle is positioned, the positioning is often performed based on a SLAM (Simultaneous Localization and Mapping, instant positioning and memory parking) technology, and the vehicle is positioned by drawing a map of the environment where the vehicle is located, so that the required engineering is huge, and the cost and the complexity of the memory parking are increased.

Disclosure of Invention

The embodiment of the application provides a memory parking method, a device, equipment, a storage medium and a program product, realizes vehicle positioning based on pictures acquired by a vehicle-mounted looking-around camera, has less dependence on hardware resources and low algorithm complexity, and reduces the cost and complexity of parking control.

In a first aspect, an embodiment of the present application provides a method for memorizing parking, where the method includes: judging whether the vehicle is located within a preset range of a preset starting point or not based on a vehicle-mounted ring view of the vehicle; if so, acquiring a reference track of the preset starting point; planning a driving track of the vehicle according to the reference track so as to control the vehicle to run to a parking point corresponding to the preset starting point; and controlling the vehicle to park in the target parking space corresponding to the preset starting point according to the parking point.

Optionally, the determining, based on a vehicle-mounted ring view of the vehicle, whether the vehicle is located within a preset range of a preset starting point includes: loading a reference ring view corresponding to at least one preset starting point; and judging whether the vehicle is positioned in a preset range of one preset starting point or not according to the reference ring view corresponding to the vehicle-mounted ring view and at least one preset starting point.

Optionally, determining whether the vehicle is located within a preset range of one of the preset starting points according to the vehicle-mounted ring view and a reference ring view corresponding to at least one preset starting point includes: acquiring characteristic information of the vehicle-mounted ring view and reference ring views corresponding to the preset starting points; for each preset starting point, determining the matching degree of the vehicle-mounted ring view and the reference ring view corresponding to the preset starting point according to the vehicle-mounted ring view and the characteristic information of the reference ring view corresponding to each preset starting point; and judging whether the vehicle is located in a preset range of the preset starting point or not according to the matching degree.

Optionally, determining whether the vehicle is located within a preset range of one of the preset starting points according to the vehicle-mounted ring view and a reference ring view corresponding to at least one preset starting point includes: acquiring characteristic information of the vehicle-mounted ring view and a reference ring view corresponding to each preset starting point; calculating a mapping relation between the vehicle-mounted ring view and a reference ring view corresponding to the preset starting point according to the characteristic information, wherein the mapping relation is used for describing a conversion relation between each first characteristic point in the vehicle-mounted ring view and each second characteristic point corresponding to each first characteristic point in the reference ring view corresponding to the preset starting point; and judging whether the vehicle is positioned in a preset range of the preset starting point or not according to the mapping relation.

Optionally, determining whether the vehicle is located within a preset range of the preset starting point according to the mapping relationship includes: according to the mapping relation, carrying out coordinate conversion on the vehicle-mounted ring view to obtain a vehicle-mounted ring view of which the converted reference ring view corresponding to a preset starting point is in the same image coordinate system; and judging whether the vehicle is positioned in a preset range of the preset starting point or not according to the converted vehicle-mounted ring view and the coordinates of corresponding points in the reference ring view corresponding to the preset starting point.

Optionally, judging whether the vehicle is located within a preset range of the preset starting point according to the converted vehicle-mounted ring view and coordinates of corresponding points in the reference ring view corresponding to the preset starting point, including: calculating the position deviation between the current position of the vehicle and a preset starting point according to the converted vehicle-mounted ring view and the coordinates of corresponding points in the reference ring view corresponding to the preset starting point; when the position deviation meets a preset condition, respectively converting the vehicle-mounted annular view and a reference annular view corresponding to the preset starting point into a first aerial view and a second aerial view; and judging whether the vehicle is located within a preset range of the preset starting point or not according to the coordinates of the corresponding characteristic points of the parking space in the first aerial view and the second aerial view.

Optionally, the mapping relationship includes a rotation matrix and a translation matrix, and determining whether the vehicle is located within a preset range of the preset starting point according to the mapping relationship includes: judging whether the vehicle-mounted annular view and a reference annular view corresponding to the preset starting point meet set conditions or not according to the rotation matrix and the translation matrix; if so, respectively converting the vehicle-mounted annular view and the reference annular view corresponding to the preset starting point into a first aerial view and a second aerial view; and judging whether the vehicle is located within a preset range of the preset starting point or not according to the coordinates of the corresponding characteristic points of the parking space in the first aerial view and the second aerial view.

Optionally, judging whether the vehicle-mounted ring view and the reference ring view corresponding to the preset starting point meet a set condition according to the rotation matrix and the translation matrix, including: according to the rotation matrix, determining the angle deviation of the image coordinate system of the vehicle-mounted ring view relative to the image coordinate system of the reference ring view corresponding to the preset starting point; determining the distance deviation of the origin of the image coordinate system of the vehicle-mounted ring view relative to the origin of the image coordinate system of the reference ring view corresponding to the preset starting point according to the translation matrix; and when the angle deviation is smaller than a preset angle and the distance deviation is smaller than a preset distance, determining that the vehicle-mounted ring view and the reference ring view corresponding to the preset starting point meet a set condition.

Optionally, determining whether the vehicle is located within a preset range of the preset starting point according to the coordinates of the corresponding feature points of the parking space in the first bird's-eye view and the second bird's-eye view, including: respectively extracting the vehicle-to-bit lines in the first aerial view and the second aerial view; respectively acquiring parking space angle points of one or more parking spaces corresponding to the vehicle position lines in the first aerial view and the second aerial view according to the vehicle position lines; calculating coordinates of the parking space corner points corresponding to the first aerial view and the second aerial view under a world coordinate system; determining a first included angle of a parking space line corresponding to the first aerial view and the second aerial view according to coordinates of each parking space corner point in the first aerial view and the second aerial view under a world coordinate system; according to the mapping relation, carrying out coordinate conversion on each point in the first aerial view to obtain a converted first aerial view; determining an angle point corresponding relation based on the coordinates of each parking space angle point in the converted first aerial view under a world coordinate system and the coordinates of each parking space angle point in the second aerial view under the world coordinate system, wherein the angle point corresponding relation is used for describing the corresponding relation of the coordinates of the corresponding parking space angle points in the converted first aerial view and the second aerial view under the world coordinate system; and judging whether the vehicle is positioned in a preset range of the preset starting point or not according to the corresponding relation between the first included angle and the corner point.

Optionally, before determining whether the vehicle is located within a preset range of a preset starting point based on the vehicle-mounted ring view of the vehicle, the method further includes: acquiring positioning information of the vehicle; and when the vehicle is determined to be near a preset starting point according to the positioning information of the vehicle, acquiring a vehicle-mounted ring view of the vehicle in real time.

Optionally, before determining whether the vehicle is located within a preset range of a preset starting point based on the vehicle-mounted ring view of the vehicle, the method further includes: starting a parking learning function based on a user instruction, and determining the position of the current vehicle as a preset starting point; loading and storing a reference ring view of the vehicle corresponding to the preset starting point; during the parking of the user, the vehicle is located according to the rotating speed information recorded by the wheel speed meter of the vehicle, and a reference track of the vehicle at the preset starting point is generated.

Optionally, planning a driving trajectory of the vehicle according to the reference trajectory includes: determining a final parking position according to the reference track; and starting an environment sensing module of the vehicle, and controlling the vehicle to run along the reference track based on the final parking position so as to reach a parking point corresponding to the final parking position.

Optionally, planning a driving trajectory of the vehicle according to the reference trajectory includes: determining the angle deviation and the translation deviation of the vehicle according to the vehicle-mounted annular view of the vehicle and the reference annular view of the preset starting point; and planning a driving track of the vehicle according to the angle deviation, the translation deviation and the reference track so as to enable the vehicle to drive along the reference track.

Optionally, planning a driving trajectory of the vehicle according to the angle deviation, the translation deviation and the reference trajectory, including: according to the angle deviation and the translation deviation, position correction is carried out on the vehicle, so that the vehicle moves to the preset starting point; and controlling the vehicle to run to a parking point corresponding to the preset starting point along the reference track corresponding to the preset starting point based on the environment sensing module.

Optionally, controlling the vehicle to park in the target parking space corresponding to the preset starting point according to the parking point includes: judging whether a target parking space corresponding to the preset starting point is detected at the parking point or not; and if so, controlling the vehicle to park in the target parking space based on the position relation between the target parking space and the vehicle at the parking point.

Optionally, if the target parking space corresponding to the preset starting point is not detected, the method further includes: and generating parking failure prompt information.

Optionally, if the vehicle is located within a preset range of a preset starting point, before the reference track of the preset starting point is obtained, the method further includes: and generating parking prompt information.

Correspondingly, obtaining the reference track of the preset starting point includes: and after receiving parking confirmation information of the user, acquiring a reference track of the preset starting point.

In a second aspect, an embodiment of the present application further provides a parking memory device, including:

the vehicle positioning module is used for judging whether the vehicle is located in a preset range of a preset starting point or not based on a vehicle-mounted ring view of the vehicle; a reference track obtaining module, configured to obtain a reference track of the preset starting point if the preset starting point is found to be the starting point; the driving control module is used for planning the driving track of the vehicle according to the reference track so as to control the vehicle to drive to a parking point corresponding to the preset starting point; and the parking control module is used for controlling the vehicle to park in the target parking space corresponding to the preset starting point according to the parking point.

In a third aspect, an embodiment of the present application further provides a parking memory apparatus, including: a memory and at least one processor; the memory stores computer-executable instructions; the at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the memory parking method provided by any corresponding embodiment of the first aspect of the application.

In a fourth aspect, the present application further provides a computer-readable storage medium, where a computer executes instructions, and when a processor executes the computer to execute the instructions, the method for memorizing and parking a vehicle according to any embodiment corresponding to the first aspect of the present application is implemented.

In a fifth aspect, the present application further provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the computer program implements the memory parking method provided in any corresponding embodiment of the first aspect of the present application.

According to the memory parking method, the device, the equipment, the storage medium and the program product, aiming at a vehicle comprising a vehicle-mounted looking-around camera, when the vehicle-mounted looking-around view of the vehicle acquired by the vehicle-mounted looking-around camera is judged to be located near a preset starting point, the memory parking function is started, the driving track of the vehicle is planned by loading a reference track corresponding to the preset starting point, so that when the vehicle reaches a parking point corresponding to the preset starting point, the vehicle is controlled to automatically park in a target parking space corresponding to the parking point, the automatic parking of the vehicle is realized, the parking convenience is improved, meanwhile, the vehicle is relocated based on the vehicle-mounted looking-around view, hardware and software resources required by the relocation are reduced, the cost and the engineering complexity are reduced, meanwhile, the positioning speed is high, and the memory parking efficiency is improved.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and together with the description, serve to explain the principles of the application.

Fig. 1 is an application scenario diagram of a memory parking method according to an embodiment of the present application;

FIG. 2 is a flowchart of a method for memory parking provided in an embodiment of the present application;

FIG. 3 is a flowchart of step S201 in the embodiment of FIG. 2;

FIG. 4 is a flowchart of a method for memorizing parking of a vehicle according to another embodiment of the present application;

FIG. 5 is a flowchart of step S408 in the embodiment of FIG. 4 of the present application;

FIG. 6 is a schematic view of a first bird's eye view and a second bird's eye view of the embodiment of FIG. 5 of the present application;

FIG. 7 is a schematic diagram illustrating the result of extracting parking space feature points from the first and second bird's-eye views in the embodiment shown in FIG. 6;

FIG. 8 is a flowchart of step S503 in the embodiment of FIG. 5;

fig. 9 is a schematic structural diagram of a parking memory device according to an embodiment of the present application;

fig. 10 is a schematic structural diagram of a memory parking device according to an embodiment of the present application;

fig. 11 is a schematic structural diagram of a vehicle according to an embodiment of the present application.

With the above figures, there are shown specific embodiments of the present application, which will be described in more detail below. These drawings and written description are not intended to limit the scope of the inventive concepts in any manner, but rather to illustrate the inventive concepts to those skilled in the art by reference to specific embodiments.

Detailed Description

Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

The following describes the technical solutions of the present application and how to solve the above technical problems with specific embodiments. The following several specific embodiments may be combined with each other, and details of the same or similar concepts or processes may not be repeated in some embodiments. Embodiments of the present application will be described below with reference to the accompanying drawings.

The following explains an application scenario of the embodiment of the present application:

fig. 1 is an application scene diagram of a memory parking method according to an embodiment of the present application, as shown in fig. 1, memory parking is an unmanned parking technology for autonomously learning a parking route of a driver, and when the driver drives a vehicle 110 into a parking lot or near a fixed parking space 120, the driver may turn on a learning function of the vehicle 110, such as by a voice command, a key command, and the like, so as to construct and store an environment map during the driver parking based on SLAM technology, and record the parking route of the driver, and when the vehicle 110 drives into the parking lot or near the fixed parking space 120 again, environmental data of a current location of the vehicle 110 is collected based on sensors, such as a visual sensor, an ultrasonic sensor, a radar, and the like, arranged on the vehicle 110, and a relocation of the vehicle 110 is implemented in combination with the established environment map, so as to control the vehicle 110 to drive along the memorized parking route, and parking the vehicle 110 into the fixed parking space 120, so that the memory parking of the vehicle is realized.

The vehicle relocation technology based on the SLAM technology has the advantages that software engineering required for map construction is large, complexity is high, and because the SLAM technology cannot restore a real scale, besides a sensor required for map construction, an Inertial Measurement Unit (IMU) and a wheel speed meter need to be combined to provide scale information, dependent hardware resources are more, and coupling of multiple sensors can further increase the complexity of the engineering, so that the relocation technology for memory parking requires more software and hardware resources, is high in cost and high in engineering complexity, and is not easy to achieve.

In view of the above technical problems, an embodiment of the present application provides a method for memorizing parking, which implements repositioning of a vehicle based on a vehicle-mounted panoramic view acquired by a vehicle-mounted panoramic camera, that is, determines a preset starting point near the vehicle, so as to plan a vehicle driving trajectory based on a reference trajectory or a memory path corresponding to the preset starting point, so that the vehicle reaches a parking point corresponding to a target parking space and parks in the target parking space.

Fig. 2 is a flowchart of a memory parking method according to an embodiment of the present application, where the memory parking method is applied to a vehicle provided with an onboard looking-around camera, and may be executed by a memory parking device, as shown in fig. 2, the memory parking method according to the embodiment includes the following steps:

step S201, based on the vehicle-mounted ring view of the vehicle, judging whether the vehicle is located in a preset range of a preset starting point.

The preset starting points may be the current position of the vehicle when the parking learning function is started by the user, and the number of the preset starting points may be one or more. The vehicle-mounted all-round view can be collected based on a vehicle-mounted all-round view camera arranged on the vehicle, and the vehicle-mounted all-round view cameras are usually multiple and arranged around the outer shell of the vehicle, such as four directions of front, back, left and right. The predetermined range may be a range having a distance of at most 2 meters from the predetermined starting point and an angle difference of at most 10 °.

Specifically, the vehicle-mounted ring view of the vehicle may be acquired in real time or according to a certain period based on each vehicle-mounted ring view camera of the vehicle, and then whether the vehicle is near a certain preset starting point or within a preset range may be determined based on each frame of the vehicle-mounted ring view.

Further, it is possible to determine whether the vehicle is within a preset range of a certain preset starting point based on the extracted features by performing image feature extraction on the vehicle-mounted ring view.

In a parking scene, a vehicle is usually located in a parking lot, and whether the vehicle is located in a preset range of one of preset starting points can be judged based on the characteristic information corresponding to the identified parking space number.

If the vehicle is judged to be located in the preset range of one preset starting point, the repositioning is successful, otherwise, the repositioning is failed.

The reference ring view is a vehicle-mounted ring view acquired at corresponding preset starting points when the parking learning function is started, and each preset starting point can correspond to a group of reference ring views acquired by each vehicle-mounted ring view camera.

Specifically, when a user drives a vehicle to a preset starting point, a parking learning function is started at the preset starting point, and each vehicle-mounted all-round-view camera of the vehicle is controlled to acquire a vehicle-mounted all-round-view image corresponding to the preset starting point, so that a reference all-round-view image corresponding to the preset starting point is obtained.

Specifically, feature matching may be performed on the vehicle-mounted ring view and the reference ring views corresponding to the preset starting points, and then, based on a matching result, it is determined whether the vehicle is located near one of the preset starting points or within a preset range.

The judgment of the preset range of the preset starting point is the process of repositioning the vehicle, the matched preset starting point is determined in a vehicle-mounted ring view characteristic matching mode, and the vehicle is positioned based on the position relation between the current position of the vehicle and the matched preset starting point. The method realizes the relocation based on the vehicle-mounted ring view characteristic matching mode acquired at different time, reduces the dependence of the relocation on hardware resources and software resources, simplifies the implementation algorithm of the relocation, and improves the relocation efficiency.

Optionally, fig. 3 is a flowchart of step S201 in the embodiment shown in fig. 2 of the present application, and as shown in fig. 3, step S201 may include the following steps:

step S301, loading a reference ring view corresponding to at least one preset starting point.

Specifically, when there are a plurality of preset starting points, the reference ring view corresponding to each preset starting point may be loaded, for example, the reference ring view corresponding to each preset starting point may be loaded according to a set sequence, where the set sequence may be determined according to the usage frequency of the target parking spaces corresponding to each preset starting point, and the higher the usage frequency of the target parking spaces is, the earlier the sequence of the corresponding preset starting points is.

Step S302, acquiring characteristic information of the vehicle-mounted ring view and the reference ring view corresponding to each preset starting point.

Specifically, feature information of the vehicle-mounted ring view and the reference ring view corresponding to each preset starting point can be extracted based on a feature detection algorithm.

The Feature detection algorithm may be orb (organized FAST and ported bridge), SURF (speedup Robust Feature extraction algorithm), SIFT (Scale Invariant Feature Transform algorithm), and other algorithms. The Features of the ORB algorithm consist of modified FAST from accessed Segment Test (Features based on Accelerated segmentation testing) key points and BRIEF (Binary Robust independent Elementary Features) descriptors.

In some embodiments, the distortion of the acquired image is large because the vehicle-mounted panoramic camera is a fisheye camera, and after the vehicle-mounted panoramic view or the reference panoramic view is obtained, the distortion correction needs to be performed on the vehicle-mounted panoramic view or the reference panoramic view, so as to obtain the feature information of the corrected vehicle-mounted panoramic view corresponding to each preset starting point.

Specifically, distortion correction can be performed on the vehicle-mounted annular view or the reference annular view based on the internal reference and the external reference calibrated by the vehicle-mounted annular view camera.

Step S303, aiming at each preset starting point, determining the matching degree of the vehicle-mounted ring view and the reference ring view corresponding to the preset starting point according to the vehicle-mounted ring view and the characteristic information of the reference ring view corresponding to each preset starting point.

Specifically, the matching feature points of the vehicle-mounted ring view and the reference ring view of the preset starting point may be determined based on the feature information, and then the matching degree of the vehicle-mounted ring view and the reference ring view of the preset starting point may be determined based on the matching feature points. The greater the number of matched feature points, the higher the degree of matching.

Further, for each preset starting point, each feature point matched with the feature information in the vehicle-mounted ring view and the reference ring view corresponding to the preset starting point may be determined based on the vehicle-mounted ring view and the feature information in the reference ring view corresponding to the preset starting point, and the mapping relationship between the vehicle-mounted ring view and the reference ring view corresponding to the preset starting point may be determined based on the image coordinates of each feature point in the vehicle-mounted ring view and the reference ring view corresponding to the preset starting point.

Specifically, feature point pairs matched with feature information can be found from two images, namely the vehicle-mounted ring view and the reference ring view corresponding to the preset starting point, for example, each feature point matched with the feature information in the vehicle-mounted ring view and the reference ring view corresponding to the preset starting point is determined based on a KNN (K-Nearest Neighbor) algorithm, so that a plurality of feature point pairs matched with the feature information are obtained. And fitting a coordinate conversion relation between the vehicle-mounted ring view and the reference ring view corresponding to the preset starting point based on the image coordinates of the feature point pairs matched with the feature information, so as to obtain a mapping relation between the vehicle-mounted ring view and the reference ring view corresponding to the preset starting point.

Further, after determining the feature points matched with the respective feature information, since the mapping relationship is obtained by fitting, for each matched feature point, it is necessary to reversely determine the matching error of each matched feature point based on the mapping relationship. The matching error of each pair of matched feature points or each pair of matched feature points is determined based on the mapping relation for each matched feature point, and the matched feature points are screened based on the matching error, for example, the matched feature points with the matching error larger than the preset error are removed, so that the matching degree of the vehicle-mounted ring view and the reference ring view of the preset starting point is determined based on the screened matched feature points.

Specifically, the matching degree between the vehicle-mounted ring view and the reference ring view of the preset starting point may be determined based on the number of the screened matched feature points.

Specifically, the matching degree between the vehicle-mounted ring view and the reference ring view of the preset starting point can be determined based on the ratio of the number of the screened matched feature points to the total number of the pixel points in the vehicle-mounted ring view.

Specifically, some mismatching feature points may exist in the matched feature points determined by the above method, so that each matched feature point may be accurately matched based on a Random Sample Consensus algorithm (RANSAC), that is, the matched feature points are screened based on the Random Sample Consensus algorithm, so as to obtain screened matched feature points, and the matching degree between the vehicle-mounted ring view and the reference ring view of the preset starting point is determined based on the screened matched feature points, so as to improve the accuracy of matching degree calculation.

And step S304, judging whether the vehicle is located in a preset range of the preset starting point or not according to the matching degree.

Specifically, if the matching degree is greater than a preset threshold, it is determined that the vehicle is located within a preset range of a preset starting point.

Step S202, if yes, acquiring the reference track of the preset starting point.

Specifically, if it is determined that the vehicle is located within the preset range of the preset starting point through the repositioning process corresponding to step S201, the reference track corresponding to the preset starting point is obtained.

When the parking learning function is started, a reference ring view corresponding to a preset starting point and a reference track corresponding to the preset starting point are recorded, wherein the reference track is generated based on a driving route of a vehicle when a driver or a user parks the vehicle.

Specifically, the reference trajectory may be generated based on vehicle localization based on the rotation speeds of the respective wheels of the vehicle detected by the wheel speed meters during the driver's control of the parking of the vehicle after the parking learning function is turned on.

In some embodiments, the reference trajectory may be a trajectory from a preset starting point to a parking point of the target parking space when the user drives.

Further, if the current position of the vehicle is determined not to be within the preset range of any preset starting point, first prompt information is generated to prompt a user to control the vehicle to move so as to update the position of the vehicle.

Step S203, planning a driving trajectory of the vehicle according to the reference trajectory, so as to control the vehicle to travel to a parking point corresponding to the preset starting point.

The parking point corresponding to the preset starting point is a position near or around the target parking space corresponding to the preset starting point, and the vehicle can be controlled to park in the corresponding target parking space at the parking point.

Specifically, after the reference trajectory corresponding to the preset starting point is obtained, the vehicle may be controlled to follow the reference trajectory, so that the vehicle automatically travels to the parking point corresponding to the preset starting point.

Further, the driving track of the vehicle can be planned according to the rotating speeds of the wheels recorded by the wheel speed meter corresponding to each node in the reference track, and the vehicle is controlled to run according to the corresponding rotating speeds, so that the parking point is reached.

Specifically, the driving track of the vehicle may be planned according to the position of each node in the reference track, so that the vehicle travels along the reference track. Meanwhile, in the process of following the reference track, the environment sensing function of the vehicle is started to avoid the obstacle.

Further, when the vehicle automatically travels along the planned trajectory, and it is determined through environmental perception that an obstacle exists on the trajectory, the vehicle may wait for parking or bypass the obstacle, and then continue to follow the reference trajectory to travel to a parking point corresponding to the preset starting point.

And step S204, controlling the vehicle to park in a target parking space corresponding to the preset starting point according to the parking point.

Specifically, when the vehicle automatically travels to a parking point corresponding to a preset starting point along a reference path corresponding to the preset starting point, the vehicle is controlled to automatically park at the parking point corresponding to a target parking space, so that parking is completed.

Further, after automatically driving to a parking spot based on the reference trajectory, the vehicle is controlled to automatically park at the parking spot to park in the corresponding fixed parking spot.

Because the wheel speed meter is used for planning the vehicle driving track, an accumulated error occurs, after the vehicle automatically drives to a corresponding parking point, the fixed parking space can be detected at the parking point, so that the position of the parking point is updated based on the detection result, and automatic parking is performed based on the updated position of the parking point to park in the fixed parking space.

The memory parking method provided by the embodiment is used for a vehicle comprising a vehicle-mounted looking-around camera, when the vehicle is judged to be located near a preset starting point based on a vehicle-mounted looking-around image of the vehicle acquired by the vehicle-mounted looking-around camera, a memory parking function is started, and a driving track of the vehicle is planned by loading a reference track corresponding to the preset starting point, so that the vehicle is controlled to automatically park in a target parking space corresponding to the parking point when the vehicle reaches the parking point corresponding to the preset starting point, the automatic parking of the vehicle is realized, the parking convenience is improved, meanwhile, the vehicle is relocated based on the vehicle-mounted looking-around image, hardware and software resources required by relocation are reduced, the cost and the engineering complexity are reduced, meanwhile, the positioning speed is high, and the memory parking efficiency is improved.

Fig. 4 is a flowchart of a memory parking method according to another embodiment of the present application, where in this embodiment, based on the embodiment shown in fig. 2, step S201 and step S204 are further defined, and a step related to reference trajectory generation and a step of vehicle-mounted view collection based on vehicle positioning information are added before step S201, as shown in fig. 4, the memory parking method according to this embodiment may include the following steps:

step S401, based on the user instruction, starting a parking learning function, and determining that the position of the current vehicle is a preset starting point.

Specifically, the user or the driver may issue an instruction to start the parking learning function through a key corresponding to the parking learning function, such as a virtual key or an entity key, provided on the vehicle. The user or the driver can also start the parking learning function through a voice instruction, such as 'learning to park and start' or 'starting to learn to park' and the like.

Specifically, when an instruction for starting the parking learning function is received, the position of the vehicle at the time is used as a preset starting point, and the vehicle-mounted panoramic camera is started to acquire a vehicle-mounted panoramic view corresponding to the preset starting point, namely the reference panoramic view.

In some embodiments, a user or a driver may correspond to a plurality of fixed parking spaces, for example, a company and a family may respectively correspond to one fixed parking space, and the user may start a parking learning function in a vicinity of each fixed parking space, for example, within a range of 50 meters, so as to obtain a preset starting point corresponding to each fixed parking space. One fixed parking space may correspond to one preset starting point or a plurality of preset starting points. When a fixed parking space corresponds to a plurality of predetermined starting points, the predetermined starting points should be spaced apart by a certain distance, such as 10 meters.

And S402, loading and storing the reference ring view of the vehicle corresponding to the preset starting point.

In some embodiments, one or more sets of reference ring views may be acquired at each preset starting point, where one set of reference ring views includes an image of the surroundings of the vehicle of one corresponding field of view acquired by each vehicle-mounted ring view camera. When a plurality of groups of ring views exist, image fusion is needed, so that a fused group of reference ring views is obtained.

And S403, during the parking of the user, positioning the vehicle according to the rotating speed information recorded by the wheel speed meter of the vehicle, and generating a reference track of the vehicle at the preset starting point.

Specifically, after the parking learning function is started, each wheel speed meter may be initialized, and the rotation speed information corresponding to each node during the parking of the user is recorded based on the wheel speed meter of the vehicle, so that the vehicle is positioned based on the rotation speed information of each node, and the position of the vehicle corresponding to each node is obtained, so that the reference trajectory of the vehicle at the preset starting point is obtained by sequentially connecting each node.

Specifically, the rotation speed information recorded by the wheel speed meter may include the rotation speed of the left front wheel and the rotation speed of the right front wheel of the vehicle, and based on the rotation speeds of the left front wheel and the right front wheel corresponding to each node, the rotation angle and the travel distance of the vehicle from the previous node to the current node may be determined, so as to obtain the reference track.

Specifically, the linear velocity v, the angular velocity w and the circular arc motion radius r of the wheel speed meter are calculated by the following formula:

in the formula u₁The rotating speed of a wheel speed meter corresponding to the right front wheel; u. of₂The rotating speed of a wheel speed meter corresponding to the left front wheel; l is the distance between the left front wheel and the right front wheel.

Specifically, during the parking of the user, a coordinate system is established with a preset starting point as a coordinate origin, and based on the linear velocity v, the angular velocity w and the circular arc motion radius r of the wheel speed meter corresponding to each node, the coordinate value of the vehicle at each node can be obtained, so as to obtain the reference track of the preset starting point.

After the parking of the user is finished, a first corresponding relation of the reference track, the preset starting point and the reference ring view of the preset starting point is established and stored.

And step S404, acquiring the positioning information of the vehicle.

Here, the Positioning information of the vehicle may be acquired based on a GPS (Global Positioning System) installed on the vehicle, or may be determined based on the Positioning information of a user terminal bound to the vehicle.

Specifically, the location information of the vehicle may be determined based on WiFi location technology and the WiFi that the vehicle can search for.

Specifically, the location information of the vehicle may be determined according to a base station used by a user terminal bound to the vehicle based on a base station location technology.

Step S405, when the vehicle is determined to be near a preset starting point according to the positioning information of the vehicle, acquiring a vehicle-mounted ring view of the vehicle in real time, and loading a reference ring view corresponding to the preset starting point.

Specifically, when the vehicle is determined to be located near one of the preset starting points based on the positioning information of the vehicle, if the distance from the preset starting point is smaller than a preset distance, such as 10 meters, the reference ring view and the reference track corresponding to the preset starting point are loaded, and the vehicle-mounted ring view of the vehicle is acquired in real time.

Further, the reference ring view and the reference track corresponding to the start point identifier may be obtained based on the start point identifier of the preset start point and a first correspondence relationship established in advance, where the first correspondence relationship is a correspondence relationship between the start point identifier of each preset start point, the reference ring view corresponding to each preset start point, and the reference track corresponding to each preset start point.

Step S406, acquiring characteristic information of the vehicle-mounted ring view and the reference ring view corresponding to the preset starting point.

Specifically, the feature information of the reference ring view corresponding to the vehicle-mounted ring view and the preset starting point may be obtained based on a feature detection algorithm.

Step S407, calculating the mapping relation between the vehicle-mounted ring view and the reference ring view corresponding to the preset starting point according to the characteristic information.

The mapping relation is used for describing the conversion relation between each first characteristic point in the vehicle-mounted annular view and each second characteristic point corresponding to each first characteristic point in the reference annular view corresponding to the preset starting point. The position and posture relation between the vehicle-mounted ring view and the reference ring view corresponding to the preset starting point can be directly determined through the mapping relation.

Specifically, the mapping relationship H between the vehicle-mounted ring view and the reference ring view may be determined based on the pixel coordinates of the matched feature points in the vehicle-mounted ring view and the corresponding reference ring view, so as to describe the mapping relationship between each pixel point in the vehicle-mounted ring view and the corresponding reference ring view based on the mapping relationship H.

And establishing a linear equation by referring to a plurality of groups of matched pixel coordinates, such as at least 4 groups of pixel coordinates corresponding to the feature points, in the annular view and the vehicle-mounted annular view, and solving the linear equation to obtain the mapping relation H.

And step S408, judging whether the vehicle is positioned in a preset range of the preset starting point or not according to the mapping relation.

Specifically, based on the mapping relationship, that is, the pose relationship between the vehicle-mounted ring view and the reference ring view, the position relationship between the position of the vehicle and the preset starting point when the vehicle-mounted ring view is acquired can be determined, and then based on the position relationship, whether the current position of the vehicle, that is, the position when the vehicle-mounted ring view is shot, is within the preset range of the preset starting point when the reference ring view is acquired can be judged.

Further, if the vehicle exceeds the preset range of the preset starting point, the repositioning fails, and repositioning failure prompt information can be generated to prompt a user to continue to drive the vehicle manually so as to update the position of the vehicle, so that vehicle-mounted ring views corresponding to all the positions are acquired in real time, and whether the vehicle after the position update is located in the preset range of the preset starting point is judged based on the subsequently acquired vehicle-mounted ring views.

Optionally, determining whether the vehicle is located within a preset range of the preset starting point according to the mapping relationship includes: judging whether the vehicle-mounted annular view and a reference annular view corresponding to the preset starting point meet set conditions or not according to the rotation matrix and the translation matrix; if so, respectively converting the vehicle-mounted annular view and the reference annular view corresponding to the preset starting point into a first aerial view and a second aerial view; and judging whether the vehicle is located within a preset range of the preset starting point or not according to the coordinates of the corresponding characteristic points of the parking space in the first aerial view and the second aerial view.

Optionally, judging whether the vehicle-mounted ring view and the reference ring view corresponding to the preset starting point meet a set condition according to the rotation matrix and the translation matrix, including:

according to the rotation matrix, determining the angle deviation of the image coordinate system of the vehicle-mounted ring view relative to the image coordinate system of the reference ring view corresponding to the preset starting point; determining the distance deviation of the origin of the image coordinate system of the vehicle-mounted ring view relative to the origin of the image coordinate system of the reference ring view corresponding to the preset starting point according to the translation matrix; and when the angle deviation is smaller than a preset angle and the distance deviation is smaller than a preset distance, determining that the vehicle-mounted ring view and the reference ring view corresponding to the preset starting point meet a set condition.

For example, if the distance deviation is smaller than a preset distance (e.g., 2 meters) and the angle deviation is smaller than a preset angle (e.g., 10 °), it is determined that the vehicle-mounted ring view and the reference ring view corresponding to the preset starting point satisfy the set condition.

Specifically, if the reference ring view corresponding to the vehicle-mounted ring view and the preset starting point meets the set condition, it indicates that the shooting position corresponding to the vehicle-mounted ring view is closer to the preset starting point, for example, within the preset range of the preset starting point.

Further, if the reference ring view corresponding to the vehicle-mounted ring view and the preset starting point does not meet the set condition, determining that the position of the vehicle for collecting the vehicle-mounted ring view exceeds the preset range of the preset starting point.

Optionally, determining whether the vehicle is located within a preset range of the preset starting point according to the mapping relationship includes: according to the mapping relation, carrying out coordinate conversion on the vehicle-mounted annular view to obtain a converted vehicle-mounted annular view; and judging whether the vehicle is positioned in a preset range of the preset starting point or not according to the converted vehicle-mounted ring view and the coordinates of corresponding points in the reference ring view corresponding to the preset starting point.

Specifically, the coordinate deviation value corresponding to each point may be calculated according to the coordinates of each point in the converted vehicle-mounted ring view and the reference ring view corresponding to the preset starting point, and whether the vehicle is located within the preset range of the preset starting point may be determined based on the coordinate deviation value corresponding to each point, such as an average value, a maximum value, and the like of the coordinate deviation values corresponding to each point.

Optionally, fig. 5 is a flowchart of step S408 in the embodiment shown in fig. 4 of the present application, and as shown in fig. 5, step S408 may include the following steps:

and S501, performing coordinate conversion on the vehicle-mounted ring view according to the mapping relation to obtain the converted vehicle-mounted ring view of which the reference ring view corresponding to the preset starting point is in the same image coordinate system.

Specifically, based on the mapping relationship, coordinate conversion may be performed on each point of the vehicle-mounted ring view, so that the point is converted into a reference ring view corresponding to the preset starting point and is located in the same image coordinate system.

Step S502, calculating the position deviation between the current position of the vehicle and the preset starting point according to the converted vehicle-mounted ring view and the coordinates of the corresponding points in the reference ring view corresponding to the preset starting point.

Specifically, a plurality of target points, such as 10, 30, 50 or all, are selected from the converted on-board annular map, and coordinate deviations corresponding to the target points are calculated based on image coordinates of the target points and image coordinates of points in a reference annular view corresponding to preset starting points corresponding to the target points; and further determining a position deviation of the current position of the vehicle from a preset starting point based on each coordinate deviation.

Step S503, when the position deviation satisfies a preset condition, respectively converting the vehicle-mounted ring view and the reference ring view corresponding to the preset starting point into a first bird 'S-eye view and a second bird' S-eye view.

The preset condition may be that the position deviation is smaller than a preset deviation. The preset deviation may be determined based on a preset range.

Specifically, when the position deviation is small, that is, when the preset condition is met, in order to improve the accuracy of the repositioning, both the vehicle-mounted annular view and the reference annular view corresponding to the preset starting point are converted into the bird's-eye view, so that the first bird's-eye view and the second bird's-eye view are obtained.

Further, if the position deviation does not meet the preset condition, determining that the position of the vehicle for collecting the vehicle-mounted ring view exceeds a preset range of a preset starting point.

Step S504, judging whether the vehicle is located in a preset range of a preset starting point according to the coordinates of the corresponding characteristic points of the parking space in the first aerial view and the second aerial view.

The characteristic points of the parking space can be corresponding points of a parking space such as a vehicle position line and a vehicle position number. The parking space line is used for limiting the range of parking spaces, and the parking space number is used for identifying each parking space.

Specifically, the vehicle position line and/or the vehicle position number of the same parking space in the first aerial view and the second aerial view are respectively extracted, and then whether the vehicle is located in the preset range of the preset starting point or not is judged based on the pixel coordinates of each point corresponding to the extracted vehicle position line and/or vehicle position number in the first aerial view and the second aerial view.

Further, a real unit distance represented by each pixel point on each aerial view can be acquired based on a monocular distance measuring principle, actual coordinates of each characteristic point are determined based on the real unit distance and pixel coordinates of each characteristic point of the parking space in the first aerial view and the second aerial view, so that a pose relation between the current position of the vehicle and a preset starting point is determined, and whether the vehicle is located in a preset range of the preset starting point is judged based on the pose relation.

For example, fig. 6 is a schematic diagram of a first bird's eye view and a second bird's eye view in the embodiment shown in fig. 5 of the present application, fig. 7 is a schematic diagram of an extraction result of parking space feature points of the first bird's eye view and the second bird's eye view in the embodiment shown in fig. 6, the first bird's eye view corresponding to the vehicle-mounted ring view acquired at the current position is an image on the right side of fig. 6, the second bird's eye view corresponding to a reference ring view acquired at the right side of fig. 6 in the start parking learning function or in the learning stage of the vehicle is an image on the left side of fig. 6, the second bird's eye view corresponding to the preset starting point includes parking spaces 040 and 042, the first bird's eye view includes parking spaces 041 and 043, the parking space numbers and linear features of the first bird's eye view and the second bird's eye view are extracted, so that the parking space numbers and parking space lines of the parking spaces can be obtained, the image 710 on the right side of fig. 7 is an extraction result corresponding to the first bird's eye view in fig. 6, the image 720 on the left side of fig. 7 is the extraction result corresponding to the second bird's eye view in fig. 6.

Optionally, fig. 8 is a flowchart of step S504 in the embodiment shown in fig. 5 of the present application, and as shown in fig. 8, step S504 may include the following steps:

step S801, respectively extracting the vehicle location lines in the first and second bird' S-eye views.

Specifically, feature detection may be performed on the first and second bird's-eye views based on Hough Transform (Hough Transform) or other linear feature extraction algorithms, so as to extract linear features in the first and second bird's-eye views, and thus obtain the parking space lines of the respective parking spaces in the first and second bird's-eye views.

Step S802, according to the vehicle position lines, parking space corner points of one or more parking spaces corresponding to the vehicle position lines in the first aerial view and the second aerial view are respectively obtained.

The parking space angular point is a point where two parking space lines of the parking space intersect.

Specifically, the parking space corner points of the parking spaces in the first aerial view and the second aerial view can be determined according to the coordinates of the characteristic points of each parking space on each parking space line.

Step S803, calculating coordinates of each parking space corner point corresponding to the first and second bird' S-eye views in a world coordinate system.

Specifically, the coordinates of the corner point of the parking space in the world coordinate system may be determined by coordinate transformation based on the pixel coordinates of the corner point of the same parking space in the first bird's-eye view and the second bird's-eye view.

Step S804, determining a first included angle of the parking space line corresponding to the first aerial view and the second aerial view according to coordinates of each parking space corner point in the first aerial view and the second aerial view under a world coordinate system.

Specifically, the included angle between the parking space lines or the horizontal lines corresponding to the at least two parking space angle points in the first aerial view and the second aerial view can be determined based on the physical coordinates of the at least two parking space angle points located on the same horizontal line.

Specifically, for each aerial view, the first aerial view or the second aerial view may establish a parking space corner linear equation corresponding to the aerial view based on physical coordinates of parking space corners located on the same horizontal line on the aerial view, and then determine a first included angle of the parking space lines corresponding to the first aerial view and the second aerial view based on a difference of the inclination angles of the parking space corner linear equations corresponding to the first aerial view and the second aerial view.

Exemplarily, the coordinates of two corner points of the parking space corresponding to the first aerial view and the second aerial view are respectively as follows: (x)₁，y₁)、(x₂，y₂)、(x₃，y₃) And (x)₄，y₄) Wherein, point (x)₁，y₁) And point (x)₃，y₃) Corresponds to, point (x)₂，y₂) And point (x)₄，y₄) Correspondingly, the first included angle yaw may be calculated as:

yaw＝Ang1-Ang2

Ang1＝tan^-1(y₁-y₂)/(x₁-x₂)

Ang2＝tan^-1(y₃-y₄)/(x₃-x₄)

the Ang1 is an inclination angle of a parking space corner point linear equation corresponding to the first aerial view, and the Ang2 is an inclination angle of a parking space corner point linear equation corresponding to the second aerial view.

And step S805, performing coordinate transformation on each point in the first aerial view according to the mapping relation to obtain a transformed first aerial view.

Specifically, based on the mapping relationship, the first bird's-eye view, that is, the view corresponding to the repositioning phase, is converted into the image coordinate system of the view corresponding to the learning phase, that is, the image coordinate system of the second bird's-eye view, so as to obtain the converted first bird's-eye view.

Step 806, determining a corner corresponding relation based on coordinates of each parking space corner point corresponding to the converted first aerial view under a world coordinate system and coordinates of each parking space corner point corresponding to the second aerial view under the world coordinate system.

And the corner point corresponding relation is used for describing the corresponding relation of coordinates of the corresponding parking space corner points in the first aerial view and the second aerial view under the world coordinate system after conversion.

After the first aerial view and the second aerial view are converted into the same image coordinate system, the corner point corresponding relation of the first aerial view and the second aerial view is determined based on the coordinates of the corner points of the parking spaces of the first aerial view and the second aerial view in the world coordinate system.

Step S807, determining whether the vehicle is located within a preset range of the preset starting point according to the corresponding relationship between the first included angle and the corner point.

And determining the pose relationship between the current position of the vehicle and a preset starting point according to the calculated corresponding relationship between the first included angle and the angular point, and judging whether the vehicle is located in a preset range of the preset starting point or not based on the pose relationship, or judging whether the pose relationship meets the condition corresponding to the preset range, such as judging whether the distance in the pose relationship is smaller than the preset distance or not, and judging whether the angle in the pose relationship is smaller than the preset angle or not.

Step S409, if yes, acquiring a reference track of the preset starting point; and planning a driving track of the vehicle according to the reference track so as to control the vehicle to drive to a parking point corresponding to the preset starting point.

The final parking position may be a position near the target parking determined in real time, and the vehicle is automatically parked at the final parking position, for example, a parking garage is backed up.

Specifically, the final parking position may be determined according to a position corresponding to the end point of the reference trajectory.

When the relocation is successful, namely the vehicle is located in a preset range of one preset starting point, starting an environment sensing module of the vehicle, loading a reference track of the preset starting point, and determining a final parking position based on a position corresponding to an end point of the reference track; and controlling the vehicle to travel along the reference track and avoid obstacles so as to reach the final parking position.

Specifically, the angular deviation and the translational deviation of the current position of the vehicle from the preset starting point can be determined based on the pose relationship calculated in the above steps, and then the driving track of the vehicle is planned based on the angular deviation, the translational cheating difference and the positions of the nodes on the reference track, so that the vehicle can follow the reference track as soon as possible, and then the vehicle can drive along the reference track, and the terminal point corresponding to the reference track can be reached.

Specifically, the position of the vehicle can be corrected based on the angle deviation and the translation deviation, so that the vehicle moves to a preset starting point, and then after the vehicle reaches the preset starting point, the vehicle is controlled to run based on the rotating speed information of the wheel speed meters corresponding to the nodes in the reference track corresponding to the preset starting point, and meanwhile, the environment sensing module is started to avoid the obstacle until the vehicle is controlled to run to the parking point corresponding to the preset starting point along the reference track.

The parking prompt message can be in a text and/or voice form. The parking prompt information may include related information of the target parking space, prompt information for memorizing parking start, and the like.

Specifically, the parking prompt information is used for prompting the user that the vehicle is successfully relocated, that is, the automatic driving or the parking mode is to be memorized so as to park in the corresponding target parking space.

Specifically, after receiving the parking prompt information, the user can confirm or reject the parking prompt information, so that after receiving the parking confirmation information of the user, the reference track of the preset starting point is obtained, and the parking is memorized based on the reference track; and stopping memorizing parking after receiving the parking rejection information of the user, and manually driving by the user to perform operations such as parking and the like.

And step S410, judging whether a target parking space corresponding to the preset starting point is detected at the parking point.

Specifically, because the reference track is followed based on the rotational speed information recorded by the wheel speed meter, errors are accumulated, so that the current position of the vehicle at the parking point has a large difference with the position of the vehicle at the parking point after the vehicle is driven based on the reference track, and if the vehicle is directly parked automatically, collision occurs, which causes user loss.

In order to improve the safety and accuracy of automatic parking, after the vehicle reaches a parking point or finishes traveling along a reference trajectory, it is necessary to determine whether a target parking space can be detected or searched.

Specifically, after the vehicle automatically travels to a parking point corresponding to the target parking space based on the reference trajectory, whether the target parking space corresponding to the preset starting point can be detected or not may be determined based on the vehicle-mounted panoramic camera or other sensors.

Step S411, if yes, controlling the vehicle to park in the target parking space based on the position relation between the target parking space and the vehicle at the parking point.

If the parking position is the target parking position, the position relationship between the target parking position and the current position of the vehicle (which may be the parking point or has a certain deviation from the parking point) may be determined based on the detection result, and the vehicle is controlled to automatically park in the target parking position based on the position relationship, so that the memory parking is completed.

Optionally, if the target parking space corresponding to the preset starting point is not detected, a parking failure prompt message is generated, so that a user can conveniently drive the vehicle manually.

In the embodiment, simple and quick memory parking is realized through three stages, wherein the first stage is a learning stage, and in the learning stage, reference ring views of all preset starting points when a learning function is started are collected, and reference tracks recorded by a user based on a wheel speed meter when the user drives corresponding to all the preset starting points are recorded; in the second stage, namely the repositioning stage, when the current position of the vehicle is determined to be near one of the preset starting points based on the vehicle-mounted GPS or the mobile terminal, the vehicle-mounted panoramic camera is started, the vehicle-mounted panoramic view is collected in real time, and then whether the current position of the vehicle is within the preset range of the preset starting point or not is judged aiming at each frame of the vehicle-mounted panoramic view, the specific judgment process is that the position and attitude relationship or the position relationship between the current position of the vehicle and the preset starting point is determined by combining the characteristic matching mode of the vehicle-mounted panoramic view, the corresponding relationship between the parking stall corner points of the parking stalls in the bird's-eye view corresponding to the vehicle-mounted panoramic view and the like, so as to determine whether the vehicle is within the preset range of the preset starting point or not, if so, the repositioning is successful, and the vehicle can be repositioned only depending on the vehicle-mounted panoramic view and the bird's-eye view converted by the repositioning mode, so that the hardware and software resources depended on the repositioning are reduced, the engineering complexity is reduced, and the aerial view is combined for repositioning, so that the repositioning accuracy is improved; after the relocation is successful, the third stage, namely the automatic parking stage is entered, the target parking space is searched, and automatic parking control is performed based on the position of the target parking space so as to control the vehicle to automatically park in the target parking space, so that memory parking is realized, and the parking convenience is improved.

Fig. 9 is a schematic structural diagram of a memory parking device according to an embodiment of the present application, and as shown in fig. 9, the memory parking device includes: a vehicle positioning module 910, a reference trajectory acquisition module 920, a traffic control module 930, and a parking control module 940.

The vehicle positioning module 910 is configured to determine whether a vehicle is located within a preset range of a preset starting point based on a vehicle-mounted ring view of the vehicle; a reference track obtaining module 920, configured to obtain a reference track of the preset starting point if the preset starting point is found; a driving control module 930, configured to plan a driving trajectory of the vehicle according to the reference trajectory, so as to control the vehicle to travel to a parking point corresponding to the preset starting point; and a parking control module 940, configured to control the vehicle to park in the target parking space corresponding to the preset starting point according to the parking point.

Optionally, the vehicle positioning module 910 includes: the reference ring view loading unit is used for loading at least one reference ring view corresponding to a preset starting point; and the vehicle positioning unit is used for judging whether the vehicle is positioned in a preset range of one preset starting point according to the vehicle-mounted ring view and a reference ring view corresponding to at least one preset starting point.

Optionally, the vehicle positioning unit is specifically configured to: acquiring characteristic information of the vehicle-mounted ring view and reference ring views corresponding to the preset starting points; for each preset starting point, according to the characteristic information, determining the matching degree of the vehicle-mounted ring view and the reference ring view corresponding to the preset starting point; and judging whether the vehicle is located in a preset range of the preset starting point or not according to the matching degree.

Optionally, the vehicle positioning unit comprises: the characteristic extraction subunit is used for acquiring characteristic information of the vehicle-mounted ring view and a reference ring view corresponding to each preset starting point; the mapping calculation subunit is configured to calculate, according to the feature information, a mapping relationship between the vehicle-mounted ring view and a reference ring view corresponding to the preset starting point, where the mapping relationship is used to describe a conversion relationship between each first feature point in the vehicle-mounted ring view and each second feature point corresponding to each first feature point in the reference ring view corresponding to the preset starting point; and the vehicle positioning subunit is used for judging whether the vehicle is positioned in a preset range of the preset starting point or not according to the mapping relation.

Optionally, the vehicle positioning subunit is specifically configured to: according to the mapping relation, carrying out coordinate conversion on the vehicle-mounted annular view to obtain a converted vehicle-mounted annular view; and judging whether the vehicle is positioned in a preset range of the preset starting point or not according to the converted vehicle-mounted ring view and the coordinates of corresponding points in the reference ring view corresponding to the preset starting point.

Optionally, the vehicle positioning subunit includes: the first image conversion part is used for carrying out coordinate conversion on the vehicle-mounted ring view according to the mapping relation so as to obtain the converted vehicle-mounted ring view of which the reference ring view corresponding to the preset starting point is in the same image coordinate system; a deviation calculation part for calculating the position deviation between the current position of the vehicle and a preset starting point according to the transformed vehicle-mounted ring view and the coordinates of corresponding points in the reference ring view corresponding to the preset starting point; a second image conversion unit, configured to convert the vehicle-mounted ring view and a reference ring view corresponding to the preset starting point into a first aerial view and a second aerial view, respectively, when the position deviation satisfies a preset condition; and the vehicle positioning part is used for judging whether the vehicle is positioned in a preset range of the preset starting point or not according to the coordinates of the corresponding characteristic points of the parking spaces in the first aerial view and the second aerial view.

Optionally, the mapping relationship includes a rotation matrix and a translation matrix, and the vehicle positioning subunit includes: the image matching part is used for judging whether the vehicle-mounted ring view and the reference ring view corresponding to the preset starting point meet set conditions or not according to the rotation matrix and the translation matrix; the image conversion part is used for converting the vehicle-mounted annular view and the reference annular view corresponding to the preset starting point into a first aerial view and a second aerial view respectively if the set conditions are met; and the positioning determination part is used for determining whether the vehicle is positioned in a preset range of the preset starting point according to the coordinates of the characteristic points of the corresponding parking spaces in the first aerial view and the second aerial view.

Optionally, the image matching unit is specifically configured to: according to the rotation matrix, determining the angle deviation of the image coordinate system of the vehicle-mounted ring view relative to the image coordinate system of the reference ring view corresponding to the preset starting point; determining the distance deviation of the origin of the image coordinate system of the vehicle-mounted ring view relative to the origin of the image coordinate system of the reference ring view corresponding to the preset starting point according to the translation matrix; and when the angle deviation is smaller than a preset angle and the distance deviation is smaller than a preset distance, determining that the vehicle-mounted ring view and the reference ring view corresponding to the preset starting point meet a set condition.

Optionally, the positioning determination part or the vehicle positioning part is specifically configured to: respectively extracting the vehicle-to-bit lines in the first aerial view and the second aerial view; respectively acquiring parking space angle points of one or more parking spaces corresponding to the vehicle position lines in the first aerial view and the second aerial view according to the vehicle position lines; calculating coordinates of the parking space corner points corresponding to the first aerial view and the second aerial view under a world coordinate system; determining a first included angle of a parking space line corresponding to the first aerial view and the second aerial view according to coordinates of each parking space corner point in the first aerial view and the second aerial view under a world coordinate system; according to the mapping relation, carrying out coordinate conversion on each point in the first aerial view to obtain a converted first aerial view; determining an angle point corresponding relation based on the coordinates of each parking space angle point in the converted first aerial view under a world coordinate system and the coordinates of each parking space angle point in the second aerial view under the world coordinate system, wherein the angle point corresponding relation is used for describing the corresponding relation of the coordinates of the corresponding parking space angle points in the converted first aerial view and the second aerial view under the world coordinate system; and judging whether the vehicle is positioned in a preset range of the preset starting point or not according to the corresponding relation between the first included angle and the corner point.

Optionally, the apparatus further comprises: the positioning information acquisition module is used for acquiring positioning information of the vehicle before judging whether the vehicle is positioned in a preset range of a preset starting point based on a vehicle-mounted ring view of the vehicle; and the image acquisition triggering module is used for acquiring the vehicle-mounted ring view of the vehicle in real time when the vehicle is determined to be near a preset starting point according to the positioning information of the vehicle.

Optionally, the apparatus further comprises: the parking learning module is used for starting a parking learning function based on a user instruction before judging whether the vehicle is located in a preset range of a preset starting point based on a vehicle-mounted ring view of the vehicle, and determining that the position of the current vehicle is the preset starting point; loading and storing a reference ring view of the vehicle corresponding to the preset starting point; during the parking of the user, the vehicle is located according to the rotating speed information recorded by the wheel speed meter of the vehicle, and a reference track of the vehicle at the preset starting point is generated.

Optionally, the driving control module 930 is specifically configured to: determining a final parking position according to the reference track; and starting an environment sensing module of the vehicle, and controlling the vehicle to run along the reference track based on the final parking position so as to reach a parking point corresponding to the final parking position.

Optionally, the driving control module 930 includes: the deviation determining unit is used for determining the angle deviation and the translation deviation of the vehicle according to the vehicle-mounted ring view of the vehicle and the reference ring view of the preset starting point; and the track tracking unit is used for planning the driving track of the vehicle according to the angle deviation, the translation deviation and the reference track so as to enable the vehicle to drive along the reference track.

Optionally, the trajectory tracking unit is specifically configured to: according to the angle deviation and the translation deviation, position correction is carried out on the vehicle, so that the vehicle moves to the preset starting point; and controlling the vehicle to run to a parking point corresponding to the preset starting point along the reference track corresponding to the preset starting point based on the environment sensing module.

Optionally, the parking control module 940 includes: the parking space detection unit is used for judging whether a target parking space corresponding to the preset starting point is detected at the parking point or not; and the parking control unit is used for controlling the vehicle to park in the target parking space based on the position relation between the target parking space and the vehicle at the parking point if the target parking space is detected.

Optionally, the parking control module 940 further includes: and the failure prompt unit is used for generating parking failure prompt information if the target parking space corresponding to the preset starting point is not detected.

Optionally, the apparatus further comprises: and the parking prompt module is used for generating parking prompt information before acquiring the reference track of the preset starting point if the vehicle is located in the preset range of the preset starting point.

Correspondingly, the reference track obtaining module 920 is specifically configured to: and after receiving parking confirmation information of the user, acquiring a reference track of the preset starting point.

The memory parking device provided by the embodiment of the application can execute the memory parking method provided by any embodiment of the application, and has corresponding functional modules and beneficial effects of the execution method.

Fig. 10 is a schematic structural diagram of a memory parking device according to an embodiment of the present application, and as shown in fig. 10, the memory parking device includes: memory 1010, processor 1020, and computer programs.

The computer program is stored in the memory 1010 and configured to be executed by the processor 1020 to implement the parking memory method according to any one of the embodiments corresponding to fig. 2 to 5 and 8 of the present application.

Wherein the memory 1010 and the processor 1020 are connected by a bus 1030.

The related description may be understood by referring to the related description and effects corresponding to the steps in fig. 2 to fig. 5 and fig. 8, and will not be described in detail herein.

Fig. 11 is a schematic structural diagram of a vehicle according to an embodiment of the present application, and as shown in fig. 11, the vehicle includes: a main body 1110, an in-vehicle surround view camera 1120, and a memory parking device 1130.

The memory parking device 1130 may be the memory parking device provided in the embodiment shown in fig. 10.

In some embodiments, the in-vehicle looking-around cameras 1120 may be respectively disposed at four sides of the front, rear, left, and right of the main body 1110 of the vehicle.

One embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored, where the computer program is executed by a processor to implement the memory parking method provided in any one of the embodiments corresponding to fig. 2 to 5 and fig. 8 of the present application.

The computer readable storage medium may be, among others, ROM, Random Access Memory (RAM), CD-ROM, magnetic tape, floppy disk, optical data storage device, and the like.

The present application also provides a program product comprising an executable computer program, the executable computer program being stored in a readable storage medium. The computer program can be read from a readable storage medium by at least one processor of the memory parking device or the vehicle, and the computer program is executed by the at least one processor to enable the memory parking device to implement the memory parking method provided by the above-mentioned various embodiments.

In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described device embodiments are merely illustrative, and for example, the division of the modules is only one logical division, and other divisions may be realized in practice, for example, a plurality of modules may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or modules, and may be in an electrical, mechanical or other form.

The modules described as separate parts may or may not be physically separate, and parts displayed as modules may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment.

In addition, functional modules in the embodiments of the present application may be integrated into one processing unit, or each module may exist alone physically, or two or more modules are integrated into one unit. The unit formed by the modules can be realized in a hardware form, and can also be realized in a form of hardware and a software functional unit.

The integrated module implemented in the form of a software functional module may be stored in a computer-readable storage medium. The software functional module is stored in a storage medium and includes several instructions for enabling a computer device (which may be a personal computer, a server, or a network device) or a processor (processor) to execute some steps of the methods according to the embodiments of the present application.

It should be understood that the Processor may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of a method disclosed in connection with the present application may be embodied directly in a hardware processor, or in a combination of the hardware and software modules in the processor.

The memory may comprise a high-speed RAM memory, and may further comprise a non-volatile storage NVM, such as at least one disk memory, and may also be a usb disk, a removable hard disk, a read-only memory, a magnetic or optical disk, etc.

The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, an Extended ISA (enhanced Industry Standard Architecture) bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, the buses in the figures of the present application are not limited to only one bus or one type of bus.

The storage medium may be implemented by any type or combination of volatile or non-volatile memory devices, such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disks. A storage media may be any available media that can be accessed by a general purpose or special purpose computer.

An exemplary storage medium is coupled to the processor such the processor can read information from, and write information to, the storage medium. Of course, the storage medium may also be integral to the processor. The processor and the storage medium may reside in an Application Specific Integrated Circuits (ASIC). Of course, the processor and the storage medium may reside as discrete components in an electronic device or host device.

Those of ordinary skill in the art will understand that: all or a portion of the steps of implementing the above-described method embodiments may be performed by hardware associated with program instructions. The program may be stored in a computer-readable storage medium. When executed, the program performs steps comprising the method embodiments described above; and the aforementioned storage medium includes: various media that can store program codes, such as ROM, RAM, magnetic or optical disks.

Finally, it should be noted that: the above embodiments are only used for illustrating the technical solutions of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A memory parking method, comprising:

judging whether the vehicle is located within a preset range of a preset starting point or not based on a vehicle-mounted ring view of the vehicle;

if so, acquiring a reference track of the preset starting point;

planning a driving track of the vehicle according to the reference track so as to control the vehicle to run to a parking point corresponding to the preset starting point;

and controlling the vehicle to park in the target parking space corresponding to the preset starting point according to the parking point.

2. The method of claim 1, wherein determining whether the vehicle is within a preset range of a preset starting point based on an on-board view of the vehicle comprises:

loading a reference ring view corresponding to at least one preset starting point;

and judging whether the vehicle is positioned in a preset range of one preset starting point or not according to the reference ring view corresponding to the vehicle-mounted ring view and at least one preset starting point.

3. The method according to claim 2, wherein judging whether the vehicle is located within a preset range of one of preset starting points according to the reference ring view of the vehicle-mounted ring view corresponding to at least one preset starting point comprises:

acquiring characteristic information of the vehicle-mounted ring view and reference ring views corresponding to the preset starting points;

for each preset starting point, determining the matching degree of the vehicle-mounted ring view and the reference ring view corresponding to the preset starting point according to the vehicle-mounted ring view and the characteristic information of the reference ring view corresponding to each preset starting point;

and judging whether the vehicle is located in a preset range of the preset starting point or not according to the matching degree.

4. The method according to claim 2, wherein judging whether the vehicle is located within a preset range of one of preset starting points according to the reference ring view of the vehicle-mounted ring view corresponding to at least one preset starting point comprises:

acquiring characteristic information of the vehicle-mounted ring view and a reference ring view corresponding to each preset starting point;

calculating a mapping relation between the vehicle-mounted ring view and a reference ring view corresponding to the preset starting point according to the characteristic information, wherein the mapping relation is used for describing a conversion relation between each first characteristic point in the vehicle-mounted ring view and each second characteristic point corresponding to each first characteristic point in the reference ring view corresponding to the preset starting point;

and judging whether the vehicle is positioned in a preset range of the preset starting point or not according to the mapping relation.

5. The method of claim 4, wherein determining whether the vehicle is within a preset range of the preset starting point according to the mapping relationship comprises:

according to the mapping relation, carrying out coordinate conversion on the vehicle-mounted ring view to obtain a vehicle-mounted ring view of which the converted reference ring view corresponding to a preset starting point is in the same image coordinate system;

and judging whether the vehicle is positioned in a preset range of the preset starting point or not according to the converted vehicle-mounted ring view and the coordinates of corresponding points in the reference ring view corresponding to the preset starting point.

6. The method according to claim 5, wherein judging whether the vehicle is located within a preset range of the preset starting point according to the converted vehicle-mounted ring view and the coordinates of corresponding points in the reference ring view corresponding to the preset starting point comprises:

calculating the position deviation between the current position of the vehicle and a preset starting point according to the converted vehicle-mounted ring view and the coordinates of corresponding points in the reference ring view corresponding to the preset starting point;

when the position deviation meets a preset condition, respectively converting the vehicle-mounted annular view and a reference annular view corresponding to the preset starting point into a first aerial view and a second aerial view;

and judging whether the vehicle is located within a preset range of the preset starting point or not according to the coordinates of the corresponding characteristic points of the parking space in the first aerial view and the second aerial view.

7. The method of claim 6, wherein determining whether the vehicle is within a preset range of the preset starting point from coordinates of the corresponding characteristic points of the parking space in the first and second bird's eye views comprises:

respectively extracting the vehicle-to-bit lines in the first aerial view and the second aerial view;

respectively acquiring parking space angle points of one or more parking spaces corresponding to the vehicle position lines in the first aerial view and the second aerial view according to the vehicle position lines;

calculating coordinates of the parking space corner points corresponding to the first aerial view and the second aerial view under a world coordinate system;

determining a first included angle of a parking space line corresponding to the first aerial view and the second aerial view according to coordinates of each parking space corner point in the first aerial view and the second aerial view under a world coordinate system;

according to the mapping relation, carrying out coordinate conversion on each point in the first aerial view to obtain a converted first aerial view;

determining an angle point corresponding relation based on the coordinates of each parking space angle point in the converted first aerial view under a world coordinate system and the coordinates of each parking space angle point in the second aerial view under the world coordinate system, wherein the angle point corresponding relation is used for describing the corresponding relation of the coordinates of the corresponding parking space angle points in the converted first aerial view and the second aerial view under the world coordinate system;

and judging whether the vehicle is positioned in a preset range of the preset starting point or not according to the corresponding relation between the first included angle and the corner point.

8. The method according to any one of claims 1-7, wherein before determining whether the vehicle is within a preset range of a preset starting point based on an on-board ring view of the vehicle, the method further comprises:

acquiring positioning information of the vehicle;

and when the vehicle is determined to be near a preset starting point according to the positioning information of the vehicle, acquiring a vehicle-mounted ring view of the vehicle in real time.

9. The method according to any one of claims 2-7, wherein before determining whether the vehicle is within a preset range of a preset starting point based on an on-board ring view of the vehicle, the method further comprises:

starting a parking learning function based on a user instruction, and determining the position of the current vehicle as a preset starting point;

loading and storing a reference ring view of the vehicle corresponding to the preset starting point;

during the parking of the user, the vehicle is located according to the rotating speed information recorded by the wheel speed meter of the vehicle, and a reference track of the vehicle at the preset starting point is generated.

10. The method according to any one of claims 1-7, wherein planning the trajectory of the vehicle based on the reference trajectory comprises:

determining the angle deviation and the translation deviation of the vehicle according to the vehicle-mounted annular view of the vehicle and the reference annular view of the preset starting point;

and planning a driving track of the vehicle according to the angle deviation, the translation deviation and the reference track so as to enable the vehicle to drive along the reference track.

11. The method of claim 10, wherein planning the trajectory of the vehicle based on the angular deviation, the translational deviation, and the reference trajectory comprises:

according to the angle deviation and the translation deviation, position correction is carried out on the vehicle, so that the vehicle moves to the preset starting point;

and controlling the vehicle to run to a parking point corresponding to the preset starting point along the reference track corresponding to the preset starting point based on the environment sensing module.

12. The method according to any one of claims 1 to 7, wherein controlling the vehicle to park in the target parking space corresponding to the preset starting point according to the parking point comprises:

judging whether a target parking space corresponding to the preset starting point is detected at the parking point or not;

and if so, controlling the vehicle to park in the target parking space based on the position relation between the target parking space and the vehicle at the parking point.

13. The method according to any one of claims 1-7, wherein if the vehicle is within a predetermined range of a predetermined starting point, before obtaining the reference trajectory of the predetermined starting point, the method further comprises:

generating parking prompt information;

correspondingly, obtaining the reference track of the preset starting point includes:

and after receiving parking confirmation information of the user, acquiring a reference track of the preset starting point.

14. A memory parking device, comprising:

the vehicle positioning module is used for judging whether the vehicle is located in a preset range of a preset starting point or not based on a vehicle-mounted ring view of the vehicle;

a reference track obtaining module, configured to obtain a reference track of the preset starting point if the preset starting point is found to be the starting point;

the driving control module is used for planning the driving track of the vehicle according to the reference track so as to control the vehicle to drive to a parking point corresponding to the preset starting point;

and the parking control module is used for controlling the vehicle to park in the target parking space corresponding to the preset starting point according to the parking point.

15. A memory parking apparatus, comprising:

a memory and at least one processor;

the memory stores computer-executable instructions;

the at least one processor executing the computer-executable instructions stored by the memory causes the at least one processor to perform the memory parking method of any of claims 1-13.

16. A computer-readable storage medium having computer-executable instructions stored thereon, which when executed by a processor, implement a memory parking method according to any one of claims 1-13.

17. A computer program product comprising computer instructions which, when executed by a processor, carry out the memory parking method according to any one of claims 1 to 13.