CN115861417A - Parking space reconstruction method and device, electronic equipment and storage medium - Google Patents
Parking space reconstruction method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN115861417A CN115861417A CN202211092360.XA CN202211092360A CN115861417A CN 115861417 A CN115861417 A CN 115861417A CN 202211092360 A CN202211092360 A CN 202211092360A CN 115861417 A CN115861417 A CN 115861417A
- Authority
- CN
- China
- Prior art keywords
- parking space
- current
- frame set
- target frame
- determining
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Traffic Control Systems (AREA)
Abstract
The embodiment of the disclosure discloses a parking space reconstruction method, a parking space reconstruction device, electronic equipment and a storage medium, wherein the method comprises the following steps: determining current parking space observation information under a vehicle coordinate system corresponding to a current target frame set, wherein the current target frame set comprises a current frame and a preset number of historical frames; determining a current parking space tracking result based on the current parking space observation information; determining first state quantities respectively corresponding to all parking spaces of a current target frame set based on a current parking space tracking result; optimizing each first state quantity based on the current parking space observation information, the current parking space tracking result and a preset optimization rule to obtain optimized second state quantities corresponding to the parking spaces of the current target frame set respectively; and determining target position information under the world coordinate system corresponding to each parking space based on the second state quantities corresponding to each parking space of the current target frame set. The embodiment of the disclosure realizes the parking space reconstruction based on the camera image, does not need to adopt a laser radar, and effectively reduces the parking space reconstruction cost.
Description
Technical Field
The present disclosure relates to computer vision technologies, and in particular, to a parking space reconstruction method and apparatus, an electronic device, and a storage medium.
Background
Parking space reconstruction has become an important technology in parking scenes such as HPA (self-learning parking), AVP (passenger parking) and the like, and is an essential part in high-precision maps. At present, a laser radar scanning mode is usually adopted for parking space reconstruction to obtain point cloud, and then parameter information of a parking space is extracted based on the point cloud, so that the parking space reconstruction is realized. But the reconstruction cost of the existing parking space reconstruction mode is higher.
Disclosure of Invention
This is disclosed in order to solve above-mentioned parking stall and rebuild the higher class of technical problem of cost. The embodiment of the disclosure provides a parking space reconstruction method and device, electronic equipment and a storage medium.
According to an aspect of the embodiment of the present disclosure, a parking space reconstruction method is provided, including: determining current parking space observation information under a vehicle coordinate system corresponding to a current target frame set, wherein the current target frame set comprises a current frame and a preset number of historical frames; determining a current parking space tracking result based on the current parking space observation information; determining first state quantities respectively corresponding to all parking spaces of the current target frame set based on the current parking space tracking result, wherein the first state quantities comprise parking space information of the corresponding parking spaces in a world coordinate system; optimizing each first state quantity based on the current parking space observation information, the current parking space tracking result and a preset optimization rule to obtain an optimized second state quantity corresponding to each parking space of the current target frame set; and determining target position information under a world coordinate system corresponding to each parking space based on the second state quantities corresponding to the parking spaces of the current target frame set.
According to another aspect of the disclosed embodiment, a parking space reconstruction device is provided, which includes: the system comprises a first determination module, a second determination module and a third determination module, wherein the first determination module is used for determining current parking space observation information under a vehicle coordinate system corresponding to a current target frame set, and the current target frame set comprises a current frame and a preset number of historical frames; the first processing module is used for determining a current parking space tracking result based on the current parking space observation information; a second determining module, configured to determine, based on the current parking space tracking result, first state quantities corresponding to the parking spaces of the current target frame set, where the first state quantities include parking space information of the corresponding parking spaces in a world coordinate system; the second processing module is used for optimizing each first state quantity based on the current parking space observation information, the current parking space tracking result and a preset optimization rule to obtain optimized second state quantities corresponding to the parking spaces of the current target frame set respectively; and the third processing module is used for determining target position information under a world coordinate system corresponding to each parking space based on the second state quantity corresponding to each parking space of the current target frame set.
According to another aspect of the embodiments of the present disclosure, a computer-readable storage medium is provided, where the storage medium stores a computer program, and the computer program is configured to execute the parking space reconstruction method according to any one of the embodiments of the present disclosure.
According to still another aspect of the embodiments of the present disclosure, there is provided an electronic apparatus including: a processor; a memory for storing the processor-executable instructions; the processor is configured to read the executable instruction from the memory, and execute the instruction to implement the parking space reconstruction method according to any one of the embodiments of the present disclosure.
Based on the parking space reconstruction method, the parking space reconstruction device, the electronic equipment and the storage medium provided by the embodiment of the disclosure, the state quantity of the parking space under the world coordinate system is optimized through observation information, tracking results and a certain optimization rule based on the parking space, so that the parking space reconstruction is realized, the camera image-based parking space reconstruction is realized, a laser radar is not needed, the parking space reconstruction cost is effectively reduced, and the problems of high parking space reconstruction cost and the like in the prior art are solved.
The technical solution of the present disclosure is further described in detail by the accompanying drawings and examples.
Drawings
The above and other objects, features and advantages of the present disclosure will become more apparent by describing in more detail embodiments of the present disclosure with reference to the attached drawings. The accompanying drawings are included to provide a further understanding of the embodiments of the disclosure and are incorporated in and constitute a part of this specification, illustrate embodiments of the disclosure and together with the description serve to explain the principles of the disclosure and not to limit the disclosure. In the drawings, like reference numbers generally represent like parts or steps.
Fig. 1 is an exemplary application scenario of the parking space reconstruction method provided in the present disclosure;
fig. 2 is a schematic flow chart of a parking space reconstruction method according to an exemplary embodiment of the present disclosure;
fig. 3 is a schematic flow chart of a parking space reconstruction method according to another exemplary embodiment of the present disclosure;
FIG. 4 is a flowchart of step 204 provided by an exemplary embodiment of the present disclosure;
FIG. 5 is a schematic diagram illustrating a determination principle of coordinates of a third endpoint and coordinates of a fourth endpoint of a warehousing line provided by an exemplary embodiment of the present disclosure;
FIG. 6 is a flowchart of step 204 provided by another exemplary embodiment of the present disclosure;
fig. 7 is a schematic flow chart of a parking space reconstruction method according to still another exemplary embodiment of the present disclosure;
fig. 8 is a schematic structural diagram of a parking space reconstruction device according to an exemplary embodiment of the present disclosure;
fig. 9 is a schematic structural diagram of a second processing module 504 provided in an exemplary embodiment of the present disclosure;
fig. 10 is a schematic structural diagram of the first processing unit 5041 provided in an exemplary embodiment of the present disclosure;
fig. 11 is a schematic structural diagram of a third processing unit 5043 provided in an exemplary embodiment of the present disclosure;
fig. 12 is a schematic structural diagram of a parking space reconstruction device according to another exemplary embodiment of the present disclosure;
fig. 13 is a schematic structural diagram of a third processing module 505 according to an exemplary embodiment of the present disclosure;
fig. 14 is a schematic structural diagram of a parking space reconstruction device according to still another exemplary embodiment of the present disclosure;
fig. 15 is a schematic structural diagram of the first determining module 501 provided in an exemplary embodiment of the present disclosure;
fig. 16 is a schematic structural diagram of a first processing module 502 provided in an exemplary embodiment of the present disclosure;
fig. 17 is a schematic structural diagram of the second determining module 503 according to an exemplary embodiment of the disclosure;
fig. 18 is a schematic structural diagram of an application embodiment of the electronic device of the present disclosure.
Detailed Description
Hereinafter, example embodiments according to the present disclosure will be described in detail with reference to the accompanying drawings. It is to be understood that the described embodiments are merely a subset of the embodiments of the present disclosure and not all embodiments of the present disclosure, with the understanding that the present disclosure is not limited to the example embodiments described herein.
It should be noted that: the relative arrangement of the components and steps, the numerical expressions, and numerical values set forth in these embodiments do not limit the scope of the present disclosure unless specifically stated otherwise.
It will be understood by those of skill in the art that the terms "first," "second," and the like in the embodiments of the present disclosure are used merely to distinguish one element from another, and are not intended to imply any particular technical meaning, nor is the necessary logical order between them.
It is also understood that in embodiments of the present disclosure, "a plurality" may refer to two or more than two, and "at least one" may refer to one, two or more than two.
It is also to be understood that any reference to any component, data, or structure in the embodiments of the disclosure, may be generally understood as one or more, unless explicitly defined otherwise or stated otherwise.
In addition, the term "and/or" in the present disclosure is only one kind of association relationship describing an associated object, and means that three kinds of relationships may exist, for example, a and/or B may mean: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the character "/" in the present disclosure generally indicates that the former and latter associated objects are in an "or" relationship.
It should also be understood that the description of the various embodiments of the present disclosure emphasizes the differences between the various embodiments, and the same or similar parts may be referred to each other, so that the descriptions thereof are omitted for brevity.
Meanwhile, it should be understood that the sizes of the respective portions shown in the drawings are not drawn in an actual proportional relationship for the convenience of description.
The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the disclosure, its application, or uses.
Techniques, methods, and apparatus known to one of ordinary skill in the relevant art may not be discussed in detail but are intended to be part of the specification where appropriate.
It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be discussed further in subsequent figures.
The disclosed embodiments may be applied to electronic devices such as terminal devices, computer systems, servers, etc., which are operational with numerous other general purpose or special purpose computing system environments or configurations. Examples of well known terminal devices, computing systems, environments, and/or configurations that may be suitable for use with electronic devices, such as terminal devices, computer systems, servers, and the like, include, but are not limited to: personal computer systems, server computer systems, thin clients, thick clients, hand-held or laptop devices, microprocessor-based systems, set top boxes, programmable consumer electronics, network pcs, minicomputer systems, mainframe computer systems, distributed cloud computing environments that include any of the above systems, and the like.
Electronic devices such as terminal devices, computer systems, servers, etc. may be described in the general context of computer system-executable instructions, such as program modules, being executed by a computer system. Generally, program modules may include routines, programs, objects, components, logic, data structures, etc. that perform particular tasks or implement particular abstract data types. The computer system/server may be practiced in distributed cloud computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed cloud computing environment, program modules may be located in both local and remote computer system storage media including memory storage devices.
Summary of the disclosure
In the process of implementing the present disclosure, the inventor finds that parking space reconstruction has become an important technology in parking scenes such as HPA (self-learning parking), AVP (passenger parking), and the like, and is an essential part in high-precision maps. At present, a laser radar scanning mode is usually adopted for parking space reconstruction to obtain point cloud, and then parameter information of a parking space is extracted based on the point cloud, so that the parking space reconstruction is realized. But the cost of the existing parking space reconstruction mode is higher.
Brief description of the drawings
Fig. 1 is an exemplary application scenario of the parking space reconstruction method provided by the present disclosure.
In a parking scene, a camera arranged on a vehicle can be used for shooting an image of the surrounding environment of the vehicle, the image of the surrounding environment comprises parking space information, so that current parking space observation information under a vehicle coordinate system corresponding to a current target frame set can be determined based on the image of the surrounding environment of the vehicle, the current target frame set comprises a current frame and a preset number of historical frames, and the current parking space observation information can comprise parking space related information obtained by respectively observing each frame in the current target frame set, such as one or more of information of parking space type, parking space angular point position, parking space long side line direction, parking space occupation condition, parking space angular point type (whether the current parking space is a cut-off point) and the like. And then determining a current parking space tracking result based on the current parking space observation information, wherein the current parking space can comprise the tracking information of all parking spaces existing in the current target frame set and the tracking information of the parking spaces existing in the previous target frame set but not existing in the current target frame set according to the result, and the previous target frame set comprises the previous frame of the current frame and a preset number of historical frames before the previous frame. The tracking information of the parking spaces may include information such as a parking space identifier (e.g., a parking space ID) and a parking space observation frequency, and the parking spaces existing in the current target frame set specifically include the parking spaces existing in the preset number of history frames but not existing in the current frame, and the parking spaces newly appearing in the current frame. Determining first state quantities respectively corresponding to all parking spaces of a current target frame set based on a current parking space tracking result, wherein the first state quantities comprise parking space information of the corresponding parking spaces in a world coordinate system; for existing parking spaces in a preset number of historical frames, the state quantity optimized by a previous target frame set can be used as the first state quantity of the parking space in a current target frame set, and for a parking space newly observed in a current frame, the initial state quantity of the parking space can be determined based on the observation information of the parking space and used as the first state quantity of the parking space; the first state quantities of the same slot in different frames are the same. Based on current parking space observation information, a current parking space tracking result and a preset optimization rule, optimizing each first state quantity to obtain an optimized second state quantity corresponding to each parking space of a current target frame set respectively, wherein the preset optimization rule can be set according to actual requirements, specifically, a target function can be set based on adjacent parking space restriction, parking space warehousing line endpoint observation reprojection restriction and the like, the optimization result enables each parking space to enable a target function value to be minimum under the condition that certain restriction is met, the optimized second state quantity is used for determining target position information under a world coordinate system corresponding to each parking space respectively, the accuracy of parking space information is guaranteed, the parking spaces can meet the restriction of the adjacent parking spaces, parking space reconstruction is achieved based on observation and tracking of the parking spaces, and therefore parking space reconstruction based on camera images is achieved, laser radars are not needed, and parking space reconstruction cost is effectively reduced.
Exemplary method
Fig. 2 is a schematic flow chart of a parking space reconstruction method according to an exemplary embodiment of the present disclosure. The embodiment can be applied to an electronic device, specifically, for example, a vehicle-mounted computing platform, as shown in fig. 2, and includes the following steps:
The current target frame set is a target frame set corresponding to the current moment, the current target frame set comprises a current frame and a preset number of historical frames, and the preset number can be set according to actual requirements. For example, 7 frames may be set, the current target frame set includes 8 frames, and the preset number may also be determined according to the vehicle driving distance, for example, the preset number is obtained by subtracting the current frame from the number of frames corresponding to the distance of 5 meters. And is not particularly limited. The current parking space observation information may include parking space related information obtained by respectively observing each frame in the current target frame set, such as one or more of information of parking space type, parking space angular point position, long edge line direction of parking space, parking space occupation condition, parking space angular point type (whether it is a cut-off point), and the like. The current parking space observation information can be obtained based on environmental image data shot by a camera on the vehicle. For example, the environment image data is converted into a vehicle coordinate system, a bird's-eye view image in the vehicle coordinate system is obtained, and the current parking space observation information is determined based on the bird's-eye view image. Or the first parking space observation information can be obtained based on the environment image data, and then the first parking space observation information is converted into the vehicle coordinate system to obtain the current parking space observation information. The specific obtaining mode of the current parking space observation information is not limited.
And step 202, determining a current parking space tracking result based on the current parking space observation information.
The current parking space tracking result is a parking space tracking result obtained by tracking a parking space, and may include tracking information of parking spaces existing in a current target frame set and tracking information of parking spaces existing in a previous target frame set but not existing in the current target frame set, where the previous target frame set includes a previous frame of the current frame and a preset number of historical frames before the previous frame. The tracking information of the parking spaces may include information such as a parking space identifier (e.g., a parking space ID), a number of parking space observations, and the like, and the parking spaces that exist in the current target frame set specifically include parking spaces that exist in the history frame of the preset number but do not exist in the current frame, and parking spaces that newly appear in the current frame. The parking space observation frequency refers to the frequency observed by the parking space in the current target frame set, that is, the number of frames of the parking space tracked by the current target frame set in the current target frame set, for example, the current target frame set includes 8 frames, and the parking space 1 is tracked in the previous 4 frames, the observation frequency of the parking space 1 is 4, new frame data is continuously generated along with the movement of the vehicle, each new frame is generated and is used as the current frame, the original current frame becomes a frame in a preset number of history frames, the earliest frame in the original preset number of history frames is eliminated to form a new current target frame set, if the parking space 1 is not observed in the current frame in the new current target frame set, the observation frequency of the parking space 1 is reduced by 1 due to the elimination of the earliest history frames to be 3, for the current frame newly observed parking space (for example, the parking space 2), the observation frequency of the current frame is set to be 1, the parking space 2 is tracked again in the next frame, and the observation frequency of the parking space 2 is changed to be 2. And the like, and maintaining the current parking space tracking result of the current target frame set in real time. The tracking of parking stall can adopt and predetermine tracking algorithm to realize, predetermine tracking algorithm and can adopt arbitrary implementable algorithm according to the actual demand, for example the intersection of two adjacent frame parking stalls is compared, determines the corresponding relation of parking stall in two frames to the tracking result of parking stall in the current frame is confirmed to the tracking result based on previous frame, so as to analogize the tracking that realizes the parking stall. The method can be specifically set according to actual requirements.
In practical applications, the current target frame set may be updated by a sliding window algorithm or other implementable manners as time goes on or as the vehicle moves, which is not limited specifically.
The first state quantity is an initial state quantity of each parking space in an optimization process of the current target frame set, and the first state quantity of each parking space can be set according to actual requirements.
For example, for a parking space existing in a parking space tracking result corresponding to a previous target frame set, a second state quantity of the parking space determined based on the previous target frame set is used as a first state quantity of the parking space of a current target frame set, and for a newly added parking space in the current parking space tracking result, the first state quantity of the parking space is determined based on parking space observation information corresponding to the parking space in current parking space observation information. The first state quantity specifically comprises information such as an abscissa x and an ordinate y of a center point of the parking space warehousing line, a heading angle yaw of the warehousing line, a length width of the warehousing line and the like, and can be specifically set according to actual requirements.
And 204, optimizing each first state quantity based on the current parking space observation information, the current parking space tracking result and a preset optimization rule to obtain optimized second state quantities corresponding to the parking spaces of the current target frame set respectively.
The preset optimization rule can be set according to actual requirements, for example, a target function can be set based on adjacent parking space constraints, parking space warehousing line endpoint observation reprojection constraints and the like, and the optimization result enables the target function value to be minimum under the condition that each parking space meets certain constraints, and the target function value is not limited specifically. The second state quantity is similar to the first state quantity and comprises optimized parking space information of a corresponding parking space under a world coordinate system. Details are not repeated.
In practical applications, the optimization of the first state quantity may be implemented by using an optimizer, which may be any implementable optimizer, and the disclosure is not limited thereto.
And step 205, determining target position information under the world coordinate system corresponding to each parking space based on the second state quantities corresponding to each parking space of the current target frame set.
The target position information of the parking space in the world coordinate system may include contour point coordinates of the parking space, for example, coordinates of at least four corner points of the parking space, and may be specifically set according to actual requirements. For a parking space, the target position information of the parking space is determined by the second state quantity after at least one optimization process, and the target position information can be specifically set according to actual requirements.
Illustratively, the obtained second state quantity comprises information such as an abscissa x and an ordinate y of a center point of an optimized parking space entry line, a heading angle yaw of the parking space entry line, and a length width of the parking space entry line, the parking space direction can be determined based on the heading angle of the parking space entry line in combination with the parking space type, the length of an adjacent side of the parking space entry line can be determined in combination with the length width of the parking space entry line, coordinates of two end points of the parking space entry line can be determined in combination with the abscissa and the ordinate of the center point of the parking space entry line, the heading angle of the parking space entry line and the length of the parking space entry line, coordinates of two other end points of the parking space can be determined based on the coordinates of the two end points of the parking space entry line, the parking space type, and the lengths of the adjacent sides of the parking space entry line, so that target position information of the parking space is obtained.
For example, when a parking space exists in a previous target frame set but does not exist in a current target frame set, that is, the observation of the parking space in the current target frame set is finished, the target position information of the parking space in the world coordinate system may be determined based on the second state quantity of the parking space obtained by the previous target frame set, and when a parking space can also be observed in the current target frame set, a next optimization process is performed based on the second state quantity of the parking space, which is used as the first state quantity of the parking space in the current target frame set of the next optimization process, and the target position information of the parking space may be determined based on the optimization until the parking space is slid out of the current target frame set.
Alternatively, the parking spaces may be vertical parking spaces, horizontal parking spaces, and diagonal parking spaces. The length of the line of warehousing of perpendicular parking stall is the length of perpendicular parking stall minor face, and the length of the line of warehousing of horizontal parking stall is the length on the long limit of horizontal parking stall, and the slant parking stall is similar with perpendicular parking stall, and the length of line of warehousing is minor face length, and to the slant parking stall, parking stall target position information can combine angle or the long limit direction between the adjacent both sides to confirm. For example, when a parking space slides out of the current target frame set, the final second state quantity of the parking space is obtained as [ x1, y1, yaw1, width1], the length of the side adjacent to the warehousing line is a preset length or a length based on an observation result, for the preset length, different types of parking spaces can be set to different lengths, for example, for a transverse parking space, the warehousing line is a long side, the wide side is 2.5 meters, for a vertical parking space, the warehousing line is a short side, the long side is 5.3 meters, for an oblique parking space, the parking space can be set according to a common condition, or determined by the observation result, and specifically set according to an actual requirement.
For example, the current target frame set is updated based on the sliding window, and then after the parking space slides out of the current window, the target position information of the parking space can be determined.
According to the parking space reconstruction method provided by the embodiment, the state quantity of the parking space under the world coordinate system is optimized through the observation information, the tracking result and a certain optimization rule based on the parking space, so that the parking space reconstruction is realized, the parking space reconstruction based on the camera image is realized, a laser radar is not required, the parking space reconstruction cost is effectively reduced, and the problems of high parking space reconstruction cost and the like in the prior art are solved.
Fig. 3 is a schematic flow chart of a parking space reconstruction method according to another exemplary embodiment of the present disclosure.
In an optional example, the optimizing each first state quantity based on the current parking space observation information, the current parking space tracking result, and the preset optimization rule in step 204 to obtain an optimized second state quantity corresponding to each parking space of the current target frame set, may specifically include the following steps:
The vehicle pose corresponding to each frame is the pose of the vehicle in the world coordinate system at the time when the vehicle acquires the data of each frame, for example, the vehicle pose corresponding to the current frame is the pose of the vehicle in the vehicle coordinate system at the time when the vehicle acquires the data of the current frame. Because the vehicle continuously moves and a time interval exists between two frames, the vehicle poses corresponding to different frames may be different. The first state quantity of the parking space comprises information such as the abscissa x and the ordinate y of the center point of the parking space warehousing line, the course angle yaw of the parking space warehousing line, the length width of the parking space warehousing line and the like, and the first end point coordinate and the second end point coordinate of the parking space warehousing line under the vehicle coordinate system can be reversely pushed out by combining the vehicle poses corresponding to the frames respectively. For example, the coordinates of the end point of the parking space in the world coordinate system may be determined based on the first state quantity, and then the coordinates of the end point of the parking space in the world coordinate system may be converted into the vehicle coordinate system of the corresponding frame based on the conversion relationship between the world coordinate system and the vehicle coordinate system, so as to obtain the coordinates of the first end point and the second end point of the parking space in the vehicle coordinate system of the corresponding frame.
And the first observation endpoint coordinate and the second observation endpoint coordinate are coordinates of two endpoints of the warehouse line obtained based on observation. The current parking space observation information includes parking space related information obtained by respectively observing each frame in the current target frame set, such as one or more of parking space type, parking space angular point position, parking space long edge line direction, parking space occupation condition, parking space angular point type (whether the current parking space observation information is a cut-off point) and the like. The first observation endpoint and the second observation endpoint of the warehousing line represent two angular points of the parking space, so that the coordinates of the first observation endpoint and the coordinates of the second observation endpoint can be obtained from the angular point position of the parking space of the current parking space observation information.
The preset objective function may be set according to actual optimization requirements, for example, the preset objective function may include a storage line endpoint re-projection residual function and an adjacent parking space constraint residual function. The system comprises a storage line end point re-projection residual error function, a first observation end point coordinate, a second observation end point coordinate, an adjacent parking space constraint residual error function and a state quantity re-projection residual error function, wherein the storage line end point re-projection residual error function is used for determining residual errors of the first end point coordinate and the second end point coordinate of each parking space of a current target frame set and the corresponding first observation end point coordinate and the second observation end point coordinate, and the adjacent parking space constraint residual error function is used for determining adjacent parking space constraint residual errors of the state quantity. And determining an objective function value by integrating the endpoint reprojection residual error and the adjacent parking space constraint residual error, wherein the optimization aims to minimize the objective function value when the state quantity meets certain constraint conditions. The detailed optimization principle is not described in detail.
This openly first quantity of state through each parking stall back-thrusts warehouse entry line extreme point coordinate, contrasts with the warehouse entry line extreme point coordinate of observing, optimizes first quantity of state based on predetermineeing the objective function, realizes the parking stall and rebuilds to realize that the parking stall based on camera image rebuilds, need not to adopt laser radar, effectively reduce the parking stall and rebuild the cost, solve prior art parking stall and rebuild the higher scheduling problem of cost.
Fig. 4 is a flowchart illustrating step 204 provided by an exemplary embodiment of the present disclosure.
In an optional example, the first state quantity comprises a center point coordinate of a storage line of a corresponding parking space in a world coordinate system, a course angle of the storage line and the length of the storage line; 2041, determining a first endpoint coordinate and a second endpoint coordinate of a warehousing line corresponding to each parking space of the current target frame set under a vehicle coordinate system based on each first state quantity and a vehicle pose corresponding to each frame in the current target frame set respectively, includes:
Wherein, the parking stall can be perpendicular parking stall, horizontal parking stall and slant parking stall. The length of the warehousing line of the vertical parking spaces is the length of the short side of the vertical parking spaces, and the length of the warehousing line of the transverse parking spaces is the length of the long side of the transverse parking spaces. The first state quantity is parking space information under a world coordinate system, and therefore the coordinates of two end points of the parking space warehousing line can be determined through reverse estimation based on the length of the warehousing line, the course angle of the warehousing line and the coordinate of the center point of the warehousing line in the first state quantity.
For example, fig. 5 is a schematic diagram illustrating the determination principle of the coordinates of the third end point and the coordinates of the fourth end point of the warehousing line provided by an exemplary embodiment of the present disclosure. The first state quantity is represented as [ x1, y1, yaw1, width1], the coordinates of the center point of the warehousing line are (x 1, y 1), the heading angle of the warehousing line is yaw1, the length of the warehousing line is width1, the coordinates (x 3, y 3) of the third end point of the warehousing line correspond to the third end point P3, the coordinates (x 4, y 4) of the fourth end point correspond to the fourth end point P4, and the coordinates of the third end point and the fourth end point can be determined as follows:
Specifically, the third endpoint coordinates and the fourth endpoint coordinates corresponding to the parking places are converted into a vehicle coordinate system based on the vehicle poses corresponding to the frames, and the first endpoint coordinates and the second endpoint coordinates of the parking places in the vehicle coordinate system corresponding to the frames are obtained. The specific conversion principle is not described in detail.
In an optional example, the optimizing, based on the first endpoint coordinate, the second endpoint coordinate, the first observation endpoint coordinate, the second observation endpoint coordinate, and the preset objective function respectively corresponding to the parking spaces of the current target frame set in step 2043, each first state quantity to obtain each second state quantity includes:
The preset objective function may include a storage line endpoint reprojection residual function and an adjacent parking space constraint residual function. And the adjacent parking space constraint residual function is used for determining the adjacent parking space constraint residual of the state quantity. And determining an objective function value by integrating the endpoint reprojection residual error and the adjacent parking space constraint residual error, wherein the optimization aims to minimize the objective function value when the state quantity meets certain constraint conditions. The detailed optimization principle is not described in detail.
And based on the objective function value, gradient reduction is carried out by adopting a least square algorithm to obtain an optimized result. The detailed description of the principle is omitted.
The preset condition may be set according to an actual requirement, for example, the preset condition is that the objective function value is minimum, or the objective function value is minimum when a certain constraint is met, which is not specifically limited.
In practical applications, the optimization of the state quantities may be implemented by an optimizer, and solving the optimization problem by the optimizer may include: and constructing a cost function, namely an optimized objective function, constructing an optimization problem to be solved through the cost function, configuring solver parameters and solving the problem, namely setting how to solve, whether the solving process is output or not and the like. Specifically, the solution of the optimization problem can be realized by using a certain optimization tool, which is not described herein again.
Fig. 6 is a flowchart illustrating step 204 provided by another exemplary embodiment of the present disclosure.
In an optional example, the determining, based on the current parking space observation information, a first observation end point coordinate and a second observation end point coordinate of a corresponding entry line of each parking space respectively observed in the current target frame set in step 2042 includes:
The observation center point coordinates of the warehousing line can be obtained based on the first observation endpoint coordinates and the second observation endpoint coordinates, and details are not repeated.
The endpoint re-projection residual error function value is obtained by comparing the observation endpoint coordinates (including the first observation endpoint coordinates and the second observation endpoint coordinates) of the warehousing line of each parking space with the endpoint coordinates (including the first endpoint coordinates and the second endpoint coordinates) based on the first state quantity reverse-deduction.
Illustratively, the preset objective function is:
wherein e represents the frame number included in the current target frame set, f represents the parking space number in the ith frame, and e ij Representing the end point re-projection residual error of the jth parking space in the ith frame, m and n representing two adjacent parking spaces in the adjacent parking space set, e mn And representing the adjacent parking space constraint residual errors of the adjacent parking spaces m and n.
Wherein e is ij Is represented as follows:
e ij ＝z ij -h(T i ,P j )
wherein z is ij Is shown in the vehicle pose T i Is located to observe the parking space P j The coordinates of the end points of the warehousing line, i.e., the coordinates of the first observation end point and the coordinates of the second observation end point, h (T) are obtained i ,P j ) And represents the coordinates of the end points of the warehousing line obtained by the back-stepping of the first state quantity, i.e., the coordinates of the first end point and the coordinates of the second end point.
e mn Is represented as follows:
e mn ＝z mn -f(P m ,P n )
wherein z is mn Indicates adjacent parking space P m And P n Distance between the observation center points of the entry lines of (1), f (P) m ,P n ) Indicating the parking space P obtained based on the first state quantity reverse-estimation m And P n The distance between the center points of the entry lines.
Wherein, width m And width n Respectively indicate parking spaces P m And P n The length of the warehousing line in the first state quantity.
And determining an objective function value based on the endpoint reprojection residual function value and the adjacent parking space constraint residual function value.
In practical application, when the first state quantity of each parking space is determined, the heading angles of the entry lines of the adjacent parking spaces may be set to be the same based on the constraint of the adjacent parking spaces, for example, the heading angle of the adjacent parking space P is set to be the same as the heading angle of the entry line of the adjacent parking space P m And P n Course angle yaw of the line in the first state quantity m ＝yaW n To ensure that the directions of the warehousing lines of the adjacent parking spaces in the result of the reconstructed parking spaces are consistent, and moreThe actual parking space condition is met.
Fig. 7 is a schematic flow chart of a parking space reconstruction method according to still another exemplary embodiment of the present disclosure.
In an optional example, after determining the current parking space tracking result based on the current parking space observation information in step 202, the method further includes:
The preset grouping rule can be set according to actual requirements, for example, the distance between the warehousing angular points of the two parking frames is smaller than a preset threshold value, the two parking frames have parallel edges, and the like, and based on the preset grouping rule, a clustering algorithm is adopted for grouping to obtain adjacent parking space groups in the current target frame set.
Based on the current parking space observation information, the current parking space tracking result, and the preset optimization rule, step 204 optimizes each first state quantity to obtain an optimized second state quantity corresponding to each parking space of the current target frame set, including:
step 2041a, based on the current parking space observation information, the current parking space tracking result, the adjacent parking space grouping, the preset adjacent parking space constraint rule and the preset optimization rule, optimizing each first state quantity to obtain second state quantities respectively corresponding to each parking space of the current target frame set.
The preset adjacent parking space constraint rule can comprise distance constraint and angle constraint, wherein the distance constraint means that the distance between the center points of the warehousing lines of the two parking spaces observed in the optimization process is half of the sum of the lengths of the warehousing lines of the two parking spaces in the state quantity. The angle constraint means the parallelism of the adjacent parking spaces, that is, the warehousing lines of the adjacent parking spaces are parallel, so that the warehousing lines of the adjacent parking spaces in the state quantity are optimized by adopting the same course angle.
This is disclosed through setting up adjacent parking stall restraint rule for the parking stall of rebuilding accords with the adjacent parking stall condition of reality more, effectively improves the accuracy of parking stall rebuilding result.
In an optional example, the determining, in step 205, the target position information in the world coordinate system corresponding to each space based on the second state quantities corresponding to each space in the current target frame set includes:
Wherein, the next target frame set is obtained by removing the oldest historical frame in the current target frame set and adding a new frame (namely, the next frame of the current frame). Specifically, the method may be implemented based on a sliding window or other similar manners, and details are not described herein.
And step 2052, taking the next target frame set as the current target frame set, responding to that the observation frequency of the target parking space in the parking space tracking result corresponding to the current target frame set is 0, and determining the contour point coordinates of the target parking space in the world coordinate system based on the second state quantity corresponding to the target parking space as the target position information of the target parking space.
Specifically, when the next frame is generated, the time is shifted, the next frame becomes the current frame, the original current frame becomes the history frame, and so on, which is not described in detail. Taking a sliding window as an example, the current target frame set represents the current window, the current parking space tracking result records the parking space tracking information in the current window and the parking space tracking information just sliding out of the window, the parking space tracking information includes a parking space identifier and a parking space observation frequency, the observation frequency becomes 0 to represent that the parking space slides out of the window, that is, represents that the optimization of the parking space is finished, and the target position information of the parking space can be determined based on the optimization result.
In an optional example, when one carport 1 slides out of the window, the state quantity of the carport 1 is not changed any more, and if the carport 2 in the window is a carport adjacent to the carport 1 sliding out of the window, the carport 2 needs to keep the adjacent carport constraint with the carport 1 sliding out of the window. For example, the entry line heading angle of the state quantity of parking space 2 does not change any more in the optimization process, and remains the same as the finally optimized heading angle of parking space 1, and so on.
And step 2053, regarding the parking space with the observation frequency greater than 0 in the parking space tracking result corresponding to the current target frame set, taking the second state quantity corresponding to the parking space as the first state quantity, and continuing to optimize until the observation frequency of the parking space is 0, so as to obtain the target position information of the parking space.
And for the parking space with the observation frequency of the current target frame set greater than 9, indicating that the optimization is not completed, and continuing to optimize, taking the second state quantity after the optimization of the previous optimization process as the initial state quantity of the current optimization process, namely the first state quantity, continuing to optimize according to the optimization process until the observation frequency of the parking space is 0, and determining the target position information of the parking space based on the second state quantity obtained by final optimization.
In an optional example, the current parking space tracking result comprises the number of tracked observation times of each parking space; after determining the current parking space tracking result based on the current parking space observation information in step 202, the method further includes:
and step 206, determining the contour point coordinates of the target parking space in the world coordinate system as the target position information of the target parking space based on the second state quantity of the target parking space obtained by the previous target frame set for the target parking space with the observation frequency of 0 in the current parking space tracking result.
And determining the contour point coordinates of the parking space under the world coordinate system based on the second state quantity obtained by the previous target frame set as the final optimized state quantity, and taking the contour point coordinates as the target position information of the parking space. The contour point coordinates may include four corner coordinates of the parking space. The optimized second state quantity comprises two end point coordinates of the entry line, namely two corner point coordinates of the parking space, and based on the two corner point coordinates and other related information of the parking space (such as the type of the parking space, the long side direction of the parking space, the general length and the width of the parking space and the like obtained by observation), the other two corner point coordinates can be determined, so that the contour point coordinates of the parking space are obtained and serve as target position information.
In an optional example, the determining, in step 201, current parking space observation information in a vehicle coordinate system corresponding to the current target frame set includes:
in step 2011, a bird's-eye view image in the vehicle coordinate system corresponding to the current frame in the current target frame set is determined.
The bird's-eye view image in the vehicle coordinate system corresponding to the current frame may be obtained by IPM (Inverse Perspective transformation) transformation based on the current frame image captured by the camera, and the detailed principle is not repeated.
And step 2012, determining current frame parking space observation information under the vehicle coordinate system corresponding to the current frame based on the bird's-eye view image.
The current frame parking space observation information can be based on a preset perception algorithm or a preset perception model to perceive the aerial view image and obtain parking space information contained in the aerial view image, so that the current frame parking space observation information is obtained, and specific principles are not repeated.
And 2013, determining the current parking space observation information based on the current frame parking space observation information and historical frame parking space observation information of a preset number, which is obtained in advance.
Specifically, because each new frame is generated, the parking space observation can be performed, and the corresponding parking space observation information can be stored after the observation, under the current target frame set, the current frame parking space observation information can be obtained only by observing the current frame, and then the current parking space observation information corresponding to the current target frame set can be obtained by obtaining the historical frame parking space observation information from the storage area.
Similarly, when determining the current parking space tracking result based on the current parking space observation information, the current parking space tracking result can be determined together with the historical parking space tracking result, and the specific principle is not repeated.
In an optional example, the determining the current parking space tracking result based on the current parking space observation information in step 202 includes:
The Intersection and Union ratio (IOU) is the ratio of the Intersection and Union of the parking spaces of two frames, and because the time between two frames of the vehicle is short, the change of the same parking space in two adjacent frames is not too large, and the corresponding relation of the parking spaces in the two frames can be determined based on the Intersection and Union ratio, thereby realizing the tracking of the parking spaces.
Specifically, the corresponding relation between the parking spaces in the current frame and the parking spaces in the previous frame can be determined based on the intersection ratio of the parking spaces, the parking spaces in the previous frame are tracked, corresponding parking space IDs are provided, the parking space IDs of the parking spaces in the current frame can be determined based on the corresponding relation between the current frame and the parking spaces in the previous frame, the parking spaces newly observed in the current frame can be set according to the parking space ID setting rules, the parking space IDs can be set for the parking space IDs according to the parking space ID setting rules, tracking is carried out in the subsequent frame, and the specific parking space tracking principle is not repeated.
In an optional example, the determining, based on the current parking space tracking result, first state quantities respectively corresponding to the parking spaces of the current target frame set in step 203 includes: regarding the existing parking space in the parking space tracking result corresponding to the previous target frame set, taking a second state quantity determined by the parking space based on the previous target frame set as a first state quantity of the parking space of the current target frame set; and for the newly added parking space in the current parking space tracking result, determining a first state quantity of the parking space based on the parking space observation information corresponding to the parking space in the current parking space observation information.
The parking space observation information is parking space information under a vehicle coordinate system, a center point of a warehousing line, a direction of the warehousing line and a length of the warehousing line under the vehicle coordinate system can be determined, and a first state quantity under a world coordinate system is obtained through conversion from the vehicle coordinate system to the world coordinate system. Details are not repeated.
The parking space reconstruction method based on the IPM image (aerial view image) is used for reconstructing the parking space, only three-dimensional reconstruction is conducted on each vertex of the parking space outline in the IPM image, the point cloud on the ground is not required to be reconstructed through a laser radar, and then the parameter information of the parking space is extracted on the basis of the point cloud, so that the parking space reconstruction cost is effectively reduced, and the parking space outline point coordinates which accord with the physical significance can be obtained by minimizing the reprojection error of each vertex of the parking space and constructing the adjacent constraint of the parking space. Wherein, IPM picture can use the fisheye camera of volume production to obtain, further reduces the parking stall and rebuilds the cost.
Any parking space reconstruction method provided by the embodiment of the present disclosure may be executed by any suitable device with data processing capability, including but not limited to: terminal equipment, a server and the like. Alternatively, any parking space reconstruction method provided by the embodiment of the present disclosure may be executed by the processor, for example, the processor executes any parking space reconstruction method mentioned in the embodiment of the present disclosure by calling a corresponding instruction stored in the memory. And will not be described in detail below.
Exemplary devices
Fig. 8 is a schematic structural diagram of a parking space reconstruction device according to an exemplary embodiment of the present disclosure. The apparatus of this embodiment may be used to implement the corresponding method embodiment of the present disclosure, and the apparatus shown in fig. 8 includes: a first determination module 501, a first processing module 502, a second determination module 503, a second processing module 504, and a third processing module 505.
A first determining module 501, configured to determine current parking space observation information in a vehicle coordinate system corresponding to a current target frame set, where the current target frame set includes a current frame and a preset number of historical frames; the first processing module 502 is configured to determine a current parking space tracking result based on the current parking space observation information determined by the first determining module 501; a second determining module 503, configured to determine, based on the current parking space tracking result determined by the first processing module 502, first state quantities respectively corresponding to the parking spaces of the current target frame set, where the first state quantities include parking space information of the corresponding parking spaces in a world coordinate system; a second processing module 504, configured to optimize each first state quantity based on the current parking space observation information, the current parking space tracking result, and a preset optimization rule, and obtain optimized second state quantities corresponding to each parking space of the current target frame set, respectively; a third processing module 505, configured to determine, based on the second state quantities respectively corresponding to the parking spaces of the current target frame set obtained by the second processing module 504, target position information in a world coordinate system respectively corresponding to the parking spaces.
In an alternative example, fig. 9 is a schematic structural diagram of the second processing module 504 according to an exemplary embodiment of the disclosure. In this example, the second processing module 504 includes: a first processing unit 5041, a second processing unit 5042, and a third processing unit 5043.
The first processing unit 5041 is configured to determine, based on each first state quantity and a vehicle pose respectively corresponding to each frame in the current target frame set, a first endpoint coordinate and a second endpoint coordinate of a storage line respectively corresponding to each parking space of the current target frame set in a vehicle coordinate system; the second processing unit 5042 is configured to determine, based on the current parking space observation information, a first observation endpoint coordinate and a second observation endpoint coordinate of a warehousing line corresponding to each parking space that is observed in the current target frame set; the third processing unit 5043 is configured to optimize each first state quantity based on a first endpoint coordinate, a second endpoint coordinate, a first observation endpoint coordinate, a second observation endpoint coordinate, and a preset objective function respectively corresponding to each parking space of the current target frame set, and obtain each second state quantity.
In an alternative example, fig. 10 is a schematic structural diagram of the first processing unit 5041 provided in an exemplary embodiment of the present disclosure. In this example, the first state quantity includes a center point coordinate of a storage line of a corresponding parking space in a world coordinate system, a heading angle of the storage line, and a length of the storage line; the first processing unit 5041 includes: a first processing sub-unit 50411 and a second processing sub-unit 50412.
The first processing subunit 50411 is configured to determine, based on the length of the entry line in each first state quantity, the heading angle of the entry line, and the center point coordinate of the entry line, a third end point coordinate and a fourth end point coordinate of the entry line, which correspond to each parking space of the current target frame set respectively, in a world coordinate system; the second processing subunit 50412 is configured to determine, based on the third endpoint coordinate and the fourth endpoint coordinate respectively corresponding to each parking space, and the vehicle pose respectively corresponding to each frame in the current target frame set, the first endpoint coordinate and the second endpoint coordinate respectively corresponding to each parking space in the vehicle coordinate system of the frame corresponding to each parking space.
In an alternative example, fig. 11 is a schematic structural diagram of a third processing unit 5043 provided in an exemplary embodiment of the present disclosure. In this example, the third processing unit 5043 includes: a third processing sub-unit 50431, a fourth processing sub-unit 50432, and a fifth processing sub-unit 50433.
A third processing subunit 50431, configured to determine an objective function value based on a first endpoint coordinate, a second endpoint coordinate, a first observation endpoint coordinate, a second observation endpoint coordinate, and a preset objective function, where the first endpoint coordinate, the second endpoint coordinate, the first observation endpoint coordinate, and the second observation endpoint coordinate respectively correspond to each parking space of the current target frame set; a fourth processing subunit 50432, configured to update, based on the objective function value and by using a least square algorithm, each first state quantity to obtain a third state quantity corresponding to each first state quantity; the fifth processing subunit 50433, in response to that each third state quantity does not satisfy the preset condition, takes each third state quantity as each first state quantity, performs optimization again, and so on until each third state quantity satisfies the preset condition, and takes each third state quantity as each second state quantity.
In an optional example, the second processing unit 5042 is further configured to determine, based on the current parking space observation information, an observation center point coordinate, a first observation end point coordinate, and a second observation end point coordinate of a corresponding entry line of each parking space observed in the current target frame set; the third processing subunit 50431 is specifically configured to: determining an endpoint re-projection residual error function value in a preset objective function based on a first endpoint coordinate, a second endpoint coordinate, a first observation endpoint coordinate and a second observation endpoint coordinate which correspond to each parking space of a current target frame set respectively; determining a constraint residual error function value of adjacent parking spaces in a preset objective function based on the coordinates of an observation central point corresponding to each parking space of the current target frame set and the length of a warehousing line in a first state quantity corresponding to each parking space; and determining a target function value based on the endpoint re-projection residual function value and the adjacent parking space constraint residual function value.
Fig. 12 is a schematic structural diagram of a parking space reconstruction device according to another exemplary embodiment of the present disclosure.
In one optional example, the apparatus of the present disclosure further comprises: a fourth processing module 506, configured to determine, based on the current parking space observation information, the current parking space tracking result, and a preset grouping rule, a group of adjacent parking spaces in the current target frame set; accordingly, the second processing module 504 includes: and the fourth processing unit 5041a is configured to optimize each first state quantity based on the current parking stall observation information, the current parking stall tracking result, the adjacent parking stall group, the preset adjacent parking stall constraint rule, and the preset optimization rule, and obtain second state quantities respectively corresponding to the parking stalls of the current target frame set.
Fig. 13 is a schematic structural diagram of the third processing module 505 according to an exemplary embodiment of the present disclosure.
In an alternative example, the third processing module 505 includes: a first determination unit 5051, a second determination unit 5052, and a third determination unit 5053.
A first determination unit 5051, configured to determine a next target frame set based on the current target frame set and the next frame; the second determining unit 5052 is configured to determine, by using the next target frame set as the current target frame set, a contour point coordinate of the target parking space in the world coordinate system based on a second state quantity corresponding to the target parking space as target position information of the target parking space in response to that the number of observations of the target parking space in the parking space tracking result corresponding to the current target frame set is 0; the third determining unit 5053 is configured to, for a parking space with the observation frequency greater than 0 in the parking space tracking result corresponding to the current target frame set, continue to perform optimization by using the second state quantity corresponding to the parking space as the first state quantity until the observation frequency of the parking space is 0, and obtain target position information of the parking space.
Fig. 14 is a schematic structural diagram of a parking space reconstruction device according to still another exemplary embodiment of the present disclosure.
In an optional example, the current parking space tracking result comprises the number of tracked observation times of each parking space; the device of this disclosure still includes: the fifth processing module 507 is configured to determine, for a target parking space with an observation frequency of 0 in the current parking space tracking result, a contour point coordinate of the target parking space in the world coordinate system based on the second state quantity of the target parking space obtained in the previous target frame set, as target position information of the target parking space.
In an alternative example, fig. 15 is a schematic structural diagram of the first determining module 501 provided in an exemplary embodiment of the present disclosure. In this example, the first determining module 501 includes: the fourth determining unit 5011, the fifth determining unit 5012, and the sixth determining unit 5013.
A fourth determining unit 5011, configured to determine a bird's-eye view image in the vehicle coordinate system corresponding to the current frame in the current target frame set; the fifth determining unit 5012 is configured to determine current frame parking space observation information in a vehicle coordinate system corresponding to the current frame based on the bird's eye view image; the sixth determining unit 5013 is configured to determine current parking space observation information based on the current frame parking space observation information and the preset number of historical frame parking space observation information obtained in advance.
In an alternative example, fig. 16 is a schematic structural diagram of the first processing module 502 according to an exemplary embodiment of the disclosure. In this example, the first processing module 502 includes: a seventh determining unit 5021 and an eighth determining unit 5022.
A seventh determining unit 5021, configured to determine, based on the current parking space observation information, an intersection ratio between each parking space of the current frame and each parking space of a previous frame of the current frame; the eighth determining unit 5022 is configured to determine a current parking space tracking result based on a cross-over ratio between each parking space of the current frame and each parking space of a previous frame of the current frame.
In an alternative example, fig. 17 is a schematic structural diagram of the second determining module 503 provided in an exemplary embodiment of the present disclosure. In this example, the second determining module 503 includes: a fifth processing unit 5031 and a sixth processing unit 5032.
A fifth processing unit 5031, configured to, for a parking space already existing in a parking space tracking result corresponding to a previous target frame set, use a second state quantity, determined based on the previous target frame set, of the parking space as a first state quantity of the parking space of a current target frame set; the sixth processing unit 5032 is configured to, for a new parking space in the current parking space tracking result, determine the first state quantity of the parking space based on the parking space observation information corresponding to the parking space in the current parking space observation information.
The various embodiments or alternative examples of the apparatus of the present disclosure may be implemented individually or in any combination without conflict.
Exemplary electronic device
An embodiment of the present disclosure further provides an electronic device, including: a memory for storing a computer program; and the processor is used for executing the computer program stored in the memory, and when the computer program is executed, the parking space reconstruction method of any embodiment of the disclosure is realized.
Fig. 18 is a schematic structural diagram of an application embodiment of the electronic device of the present disclosure. In this embodiment, the electronic device 10 includes one or more processors 11 and a memory 12.
The processor 11 may be a Central Processing Unit (CPU) or other form of processing unit having data processing capabilities and/or instruction execution capabilities, and may control other components in the electronic device 10 to perform desired functions.
Memory 12 may include one or more computer program products that may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory. The volatile memory may include, for example, random Access Memory (RAM), cache memory (cache), and/or the like. The non-volatile memory may include, for example, read Only Memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium and executed by processor 11 to implement the methods of the various embodiments of the disclosure described above and/or other desired functionality. Various contents such as an input signal, a signal component, a noise component, etc. may also be stored in the computer-readable storage medium.
In one example, the electronic device 10 may further include: an input device 13 and an output device 14, which are interconnected by a bus system and/or other form of connection mechanism (not shown).
The input means 13 may be, for example, a microphone or a microphone array as described above for capturing an input signal of a sound source.
The input device 13 may also include, for example, a keyboard, a mouse, and the like.
The output device 14 may output various information including the determined distance information, direction information, and the like to the outside. The output devices 14 may include, for example, a display, speakers, a printer, and a communication network and its connected remote output devices, among others.
Of course, for simplicity, only some of the components of the electronic device 10 relevant to the present disclosure are shown in fig. 18, omitting components such as buses, input/output interfaces, and the like. In addition, the electronic device 10 may include any other suitable components depending on the particular application.
Exemplary computer program product and computer-readable storage Medium
In addition to the above-described methods and apparatus, embodiments of the present disclosure may also be a computer program product comprising computer program instructions that, when executed by a processor, cause the processor to perform steps in a method according to various embodiments of the present disclosure as described in the "exemplary methods" section of this specification above.
The computer program product may write program code for carrying out operations for embodiments of the present disclosure in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server.
Furthermore, embodiments of the present disclosure may also be a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, cause the processor to perform steps in a method according to various embodiments of the present disclosure as described in the "exemplary methods" section above of this specification.
The computer readable storage medium may take any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may include, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
The foregoing describes the general principles of the present disclosure in conjunction with specific embodiments, however, it is noted that the advantages, effects, etc. mentioned in the present disclosure are merely examples and are not limiting, and they should not be considered essential to the various embodiments of the present disclosure. Furthermore, the foregoing disclosure of specific details is for the purpose of illustration and description and is not intended to be limiting, since the disclosure is not intended to be limited to the specific details so described.
In the present specification, the embodiments are described in a progressive manner, and each embodiment focuses on differences from other embodiments, and the same or similar parts in each embodiment are referred to each other. For the system embodiment, since it basically corresponds to the method embodiment, the description is relatively simple, and for the relevant points, reference may be made to the partial description of the method embodiment.
The block diagrams of devices, apparatuses, systems referred to in this disclosure are only given as illustrative examples and are not intended to require or imply that the connections, arrangements, configurations, etc. must be made in the manner shown in the block diagrams. These devices, apparatuses, devices, systems may be connected, arranged, configured in any manner, as will be appreciated by those skilled in the art. Words such as "including," "comprising," "having," and the like are open-ended words that mean "including, but not limited to," and are used interchangeably herein. The words "or" and "as used herein mean, and are used interchangeably with, the word" and/or, "unless the context clearly dictates otherwise. The word "such as" is used herein to mean, and is used interchangeably with, the phrase "such as but not limited to".
The methods and apparatus of the present disclosure may be implemented in a number of ways. For example, the methods and apparatus of the present disclosure may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above-described order for the steps of the method is for illustration only, and the steps of the method of the present disclosure are not limited to the order specifically described above unless specifically stated otherwise. Further, in some embodiments, the present disclosure may also be embodied as programs recorded in a recording medium, the programs including machine-readable instructions for implementing the methods according to the present disclosure. Thus, the present disclosure also covers a recording medium storing a program for executing the method according to the present disclosure.
It is also noted that in the devices, apparatuses, and methods of the present disclosure, each component or step can be decomposed and/or recombined. These decompositions and/or recombinations are to be considered equivalents of the present disclosure.
The previous description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other aspects without departing from the scope of the disclosure. Thus, the present disclosure is not intended to be limited to the aspects shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
The foregoing description has been presented for purposes of illustration and description. Furthermore, this description is not intended to limit embodiments of the disclosure to the form disclosed herein. While a number of example aspects and embodiments have been discussed above, those of skill in the art will recognize certain variations, modifications, alterations, additions and sub-combinations thereof.
Claims (14)
1. A parking space reconstruction method comprises the following steps:
determining current parking space observation information under a vehicle coordinate system corresponding to a current target frame set, wherein the current target frame set comprises a current frame and a preset number of historical frames;
determining a current parking space tracking result based on the current parking space observation information;
determining first state quantities respectively corresponding to all parking spaces of the current target frame set based on the current parking space tracking result, wherein the first state quantities comprise parking space information of the corresponding parking spaces in a world coordinate system;
optimizing each first state quantity based on the current parking space observation information, the current parking space tracking result and a preset optimization rule to obtain optimized second state quantities corresponding to the parking spaces of the current target frame set respectively;
and determining target position information under a world coordinate system corresponding to each parking space respectively based on the second state quantity corresponding to each parking space of the current target frame set.
2. The method of claim 1, wherein the optimizing each of the first state quantities based on the current parking space observation information, the current parking space tracking result, and a preset optimization rule to obtain optimized second state quantities corresponding to the respective parking spaces of the current target frame set comprises:
determining first endpoint coordinates and second endpoint coordinates of a storage line corresponding to each parking space of the current target frame set under the vehicle coordinate system based on each first state quantity and the vehicle pose corresponding to each frame in the current target frame set;
determining a first observation endpoint coordinate and a second observation endpoint coordinate of the warehousing line respectively corresponding to each parking space intensively observed by the current target frame based on the current parking space observation information;
and optimizing each first state quantity based on the first endpoint coordinate, the second endpoint coordinate, the first observation endpoint coordinate, the second observation endpoint coordinate and a preset objective function corresponding to each parking space of the current target frame set, so as to obtain each second state quantity.
3. The method according to claim 2, wherein the first state quantity comprises coordinates of a center point of a corresponding parking space entry line in the world coordinate system, a heading angle of the entry line and a length of the entry line;
the determining, based on the first state quantities and the vehicle poses respectively corresponding to the frames in the current target frame set, first endpoint coordinates and second endpoint coordinates of a parking line respectively corresponding to each parking space in the current target frame set under the vehicle coordinate system includes:
determining a third endpoint coordinate and a fourth endpoint coordinate of the warehousing line, which correspond to the parking spaces of the current target frame set respectively under the world coordinate system, based on the length of the warehousing line, the course angle of the warehousing line and the center point coordinate of the warehousing line in each first state quantity;
and determining the first endpoint coordinate and the second endpoint coordinate respectively corresponding to each parking space under the vehicle coordinate system of the corresponding frame of the parking space based on the third endpoint coordinate and the fourth endpoint coordinate respectively corresponding to each parking space and the vehicle pose respectively corresponding to each frame in the current target frame set.
4. The method of claim 2, wherein the optimizing each first state quantity based on the first endpoint coordinate, the second endpoint coordinate, the first observation endpoint coordinate, the second observation endpoint coordinate, and a preset objective function respectively corresponding to each parking space of the current target frame set to obtain each second state quantity comprises:
determining an objective function value based on the first endpoint coordinate, the second endpoint coordinate, the first observation endpoint coordinate, the second observation endpoint coordinate and a preset objective function respectively corresponding to each parking space of the current target frame set;
updating each first state quantity by adopting a least square algorithm based on the objective function value to obtain a third state quantity corresponding to each first state quantity;
and in response to that each third state quantity does not meet a preset condition, taking each third state quantity as each first state quantity, optimizing again, and so on until each third state quantity meets the preset condition, and taking each third state quantity as each second state quantity.
5. The method of claim 4, wherein the determining, based on the current parking space observation information, a first observation end point coordinate and a second observation end point coordinate of the warehousing line respectively corresponding to the parking spaces observed in the current target frame set comprises:
based on the current parking space observation information, determining an observation center point coordinate, the first observation endpoint coordinate and the second observation endpoint coordinate of the storage line corresponding to each parking space observed in the current target frame set;
determining an objective function value based on the first endpoint coordinate, the second endpoint coordinate, the first observation endpoint coordinate, the second observation endpoint coordinate, and a preset objective function corresponding to each parking space of the current target frame set, including:
determining an endpoint re-projection residual error function value in the preset objective function based on the first endpoint coordinate, the second endpoint coordinate, the first observation endpoint coordinate and the second observation endpoint coordinate respectively corresponding to each parking space of the current target frame set;
determining an adjacent parking space constraint residual error function value in the preset objective function based on the observation central point coordinate corresponding to each parking space of the current target frame set and the length of a warehousing line in the first state quantity corresponding to each parking space;
and determining the objective function value based on the endpoint re-projection residual function value and the adjacent parking space constraint residual function value.
6. The method of claim 1, wherein after said determining a current slot tracking result based on said current slot observation information, further comprising:
determining adjacent parking space groups concentrated by the current target frame based on the current parking space observation information, the current parking space tracking result and a preset grouping rule;
the optimizing each first state quantity based on the current parking space observation information, the current parking space tracking result and a preset optimization rule to obtain an optimized second state quantity corresponding to each parking space of the current target frame set respectively includes:
and optimizing each first state quantity based on the current parking space observation information, the current parking space tracking result, the adjacent parking space grouping, a preset adjacent parking space constraint rule and a preset optimization rule to obtain second state quantities respectively corresponding to the parking spaces of the current target frame set.
7. The method of claim 1, wherein the determining, based on the second state quantities corresponding to the parking spaces of the current target frame set, target position information in a world coordinate system corresponding to the parking spaces includes:
determining a next target frame set based on the current target frame set and a next frame;
taking the next target frame set as the current target frame set, responding to that the observation times of the target parking space in the parking space tracking result corresponding to the current target frame set are 0, and determining the contour point coordinates of the target parking space in the world coordinate system based on the second state quantity corresponding to the target parking space as the target position information of the target parking space;
and for the parking spaces with observation times larger than 0 in the parking space tracking result corresponding to the current target frame set, taking the second state quantity corresponding to the parking spaces as the first state quantity, and continuing to optimize until the observation times of the parking spaces are 0, so as to obtain the target position information of the parking spaces.
8. The method of claim 1, wherein the current slot tracking results include a number of observations of each tracked slot;
after determining a current parking space tracking result based on the current parking space observation information, the method further comprises the following steps:
and determining contour point coordinates of the target parking space in the world coordinate system based on the second state quantity of the target parking space obtained by the previous target frame set as the target position information of the target parking space for the target parking space with the observation frequency of 0 in the current parking space tracking result.
9. The method of claim 1, wherein the determining current space observation information in a vehicle coordinate system corresponding to the current target frame set comprises:
determining a bird's-eye view image under the vehicle coordinate system corresponding to a current frame in the current target frame set;
determining current frame parking space observation information under a vehicle coordinate system corresponding to the current frame based on the aerial view image;
and determining the current parking space observation information based on the current frame parking space observation information and the historical frame parking space observation information of the preset number, which is obtained in advance.
10. The method according to any one of claims 1-9, wherein said determining a current space tracking result based on said current space observation information comprises:
determining intersection ratios of the parking spaces of the current frame and the parking spaces of the previous frame of the current frame respectively based on the current parking space observation information;
and determining the current parking space tracking result based on the intersection ratio of each parking space of the current frame and each parking space of the previous frame of the current frame.
11. The method according to any one of claims 1 to 9, wherein the determining, based on the current slot tracking result, first state quantities respectively corresponding to slots of the current target frame set includes:
regarding the existing parking space in the parking space tracking result corresponding to the previous target frame set, taking a second state quantity determined by the parking space based on the previous target frame set as the first state quantity of the parking space of the current target frame set;
and determining the first state quantity of the parking space based on the parking space observation information corresponding to the parking space in the current parking space observation information for the newly added parking space in the current parking space tracking result.
12. A parking space reconstruction device, comprising:
the system comprises a first determination module, a second determination module and a third determination module, wherein the first determination module is used for determining current parking space observation information under a vehicle coordinate system corresponding to a current target frame set, and the current target frame set comprises a current frame and a preset number of historical frames;
the first processing module is used for determining a current parking space tracking result based on the current parking space observation information;
a second determining module, configured to determine, based on the current parking space tracking result, first state quantities corresponding to the parking spaces of the current target frame set, where the first state quantities include parking space information of the corresponding parking spaces in a world coordinate system;
the second processing module is used for optimizing each first state quantity based on the current parking space observation information, the current parking space tracking result and a preset optimization rule to obtain optimized second state quantities corresponding to the parking spaces of the current target frame set respectively;
and the third processing module is used for determining target position information under a world coordinate system corresponding to each parking space based on the second state quantity corresponding to each parking space of the current target frame set.
13. A computer-readable storage medium, in which a computer program is stored, the computer program being configured to execute the parking space reconstruction method according to any one of claims 1 to 11.
14. An electronic device, the electronic device comprising:
a processor;
a memory for storing the processor-executable instructions;
the processor is configured to read the executable instructions from the memory and execute the instructions to implement the parking space reconstruction method according to any one of claims 1 to 11.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211092360.XA CN115861417A (en) | 2022-09-07 | 2022-09-07 | Parking space reconstruction method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211092360.XA CN115861417A (en) | 2022-09-07 | 2022-09-07 | Parking space reconstruction method and device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115861417A true CN115861417A (en) | 2023-03-28 |
Family
ID=85660788
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211092360.XA Pending CN115861417A (en) | 2022-09-07 | 2022-09-07 | Parking space reconstruction method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115861417A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117012053A (en) * | 2023-09-28 | 2023-11-07 | 东风悦享科技有限公司 | Post-optimization method, system and storage medium for parking space detection point |
-
2022
- 2022-09-07 CN CN202211092360.XA patent/CN115861417A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117012053A (en) * | 2023-09-28 | 2023-11-07 | 东风悦享科技有限公司 | Post-optimization method, system and storage medium for parking space detection point |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109214980B (en) | Three-dimensional attitude estimation method, three-dimensional attitude estimation device, three-dimensional attitude estimation equipment and computer storage medium | |
| JP6902122B2 (en) | Double viewing angle Image calibration and image processing methods, equipment, storage media and electronics | |
| CN111985036A (en) | House type frame line drawing method and device, storage medium and electronic equipment | |
| JP7422105B2 (en) | Obtaining method, device, electronic device, computer-readable storage medium, and computer program for obtaining three-dimensional position of an obstacle for use in roadside computing device | |
| CN111640180B (en) | Three-dimensional reconstruction method and device and terminal equipment | |
| CN109509236B (en) | Vehicle bounding box generation method and device in unmanned scene and storage medium | |
| CN110060230B (en) | Three-dimensional scene analysis method, device, medium and equipment | |
| CN112116655B (en) | Target object position determining method and device | |
| CN113610172B (en) | Neural network model training method and device and sensing data fusion method and device | |
| CN115861417A (en) | Parking space reconstruction method and device, electronic equipment and storage medium | |
| CN115512044A (en) | Visual perception method and device, readable storage medium and electronic equipment | |
| CN113592706B (en) | Method and device for adjusting homography matrix parameters | |
| CN115164919B (en) | Method and device for constructing spatial travelable area map based on binocular camera | |
| CN115775382A (en) | Parking space reconstruction method and device, electronic equipment and storage medium | |
| CN110827337B (en) | Method and device for determining posture of vehicle-mounted camera and electronic equipment | |
| CN114648639B (en) | Target vehicle detection method, system and device | |
| CN115620250A (en) | Road surface element reconstruction method, device, electronic device and storage medium | |
| CN115423852A (en) | Point cloud data registration method and system, electronic device and storage medium | |
| CN114926316A (en) | Distance measuring method, distance measuring device, electronic device, and storage medium | |
| CN114677280A (en) | Method, apparatus, device and program product for generating panoramic image | |
| CN114937250A (en) | Method and device for calculating relative pose of vehicle body, vehicle, equipment and storage medium | |
| US11281935B2 (en) | 3D object detection from calibrated 2D images | |
| CN113129437B (en) | Method and device for determining space coordinates of markers | |
| US9165208B1 (en) | Robust ground-plane homography estimation using adaptive feature selection | |
| CN113129437A (en) | Method and device for determining space coordinates of marker |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |