CN111968071A - Method, device, equipment and storage medium for generating spatial position of vehicle - Google Patents
Method, device, equipment and storage medium for generating spatial position of vehicle Download PDFInfo
- Publication number
- CN111968071A CN111968071A CN202010605263.0A CN202010605263A CN111968071A CN 111968071 A CN111968071 A CN 111968071A CN 202010605263 A CN202010605263 A CN 202010605263A CN 111968071 A CN111968071 A CN 111968071A
- Authority
- CN
- China
- Prior art keywords
- vehicle
- bounding box
- orientation angle
- value
- projection line
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 54
- 238000005457 optimization Methods 0.000 claims abstract description 29
- 238000005070 sampling Methods 0.000 claims description 39
- 230000015654 memory Effects 0.000 claims description 19
- 230000006870 function Effects 0.000 claims description 17
- 238000001514 detection method Methods 0.000 claims description 11
- 239000000126 substance Substances 0.000 claims 1
- 230000004438 eyesight Effects 0.000 abstract description 3
- 238000004590 computer program Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 238000004891 communication Methods 0.000 description 4
- 238000013528 artificial neural network Methods 0.000 description 3
- 230000008901 benefit Effects 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 230000003993 interaction Effects 0.000 description 2
- 239000004973 liquid crystal related substance Substances 0.000 description 2
- 230000001154 acute effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000011217 control strategy Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000001953 sensory effect Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
- 230000016776 visual perception Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
- G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30248—Vehicle exterior or interior
- G06T2207/30252—Vehicle exterior; Vicinity of vehicle
Abstract
The application provides a method, a device, equipment and a storage medium for generating a spatial position of a vehicle, which relate to the technical field of computer vision and intelligent traffic, and the specific implementation scheme is as follows: generating a bounding box projection line of a 2D bounding box of the vehicle from the image of the vehicle; acquiring size information of the vehicle according to the image; establishing a 2D coordinate system of the vehicle relative to the ground plane according to the projection line of the enclosure frame; optimizing the position of a center point and an orientation angle of a lower plane of the vehicle in a 2D coordinate system to generate a coordinate optimized value and an orientation angle optimized value; and generating the spatial position of the vehicle according to the coordinate optimization value, the orientation angle optimization value and the size information of the vehicle. According to the method and the device, the accuracy of the spatial position estimation of the vehicle can be improved, and further, the reliability of the application based on the spatial position of the vehicle is improved.
Description
Technical Field
The application relates to the technical field of image processing, in particular to the technical field of computer vision and intelligent traffic, and provides a method, a device, equipment and a storage medium for generating a spatial position of a vehicle.
Background
The 3D positioning of the vehicle refers to estimating the spatial position of the vehicle, including estimating the size, position and direction of the vehicle. The spatial position of the vehicle is very important for intelligent traffic, and important road condition information can be provided for the unmanned system so as to assist the unmanned vehicle in path planning and improve the safety of the unmanned system; the traffic flow condition in the intersection can be counted, a basis is provided for the signal control strategy planning of the intelligent signal lamp system, and the traffic passing efficiency is improved.
Currently, when generating the spatial position of the vehicle from the vehicle image, the accuracy of the spatial position of the vehicle is to be improved due to the influence of the perspective effect.
Disclosure of Invention
The present application is directed to solving, at least to some extent, one of the technical problems in the related art.
Therefore, the application provides a method, a device, equipment and a storage medium for generating the spatial position of the vehicle.
An embodiment of a first aspect of the present application provides a spatial position generating method for a vehicle, including:
generating a bounding box projection line of a 2D bounding box of a vehicle from an image of the vehicle;
acquiring size information of the vehicle according to the image;
establishing a 2D coordinate system of the vehicle relative to the ground plane according to the surrounding frame projection line;
optimizing the center point position and the orientation angle of the lower plane of the vehicle in the 2D coordinate system to generate a coordinate optimized value and an orientation angle optimized value; and
and generating the space position of the vehicle according to the coordinate optimization value, the orientation angle optimization value and the size information of the vehicle.
An embodiment of a second aspect of the present application provides a spatial position generating apparatus for a vehicle, including:
a first generating module for generating a bounding box projection line of a 2D bounding box of a vehicle from an image of the vehicle;
the acquisition module is used for acquiring the size information of the vehicle according to the image;
the establishing module is used for establishing a 2D coordinate system of the vehicle relative to the ground plane according to the surrounding frame projection line;
an optimization module for optimizing a center point position and an orientation angle of a lower plane of the vehicle in the 2D coordinate system to generate a coordinate optimized value and an orientation angle optimized value; and
and the second generation module is used for generating the spatial position of the vehicle according to the coordinate optimization value, the orientation angle optimization value and the size information of the vehicle.
The embodiment of the third aspect of the present application provides an electronic device, which includes at least one processor, and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform a method of generating a spatial location of a vehicle as described in an embodiment of the first aspect.
A fourth aspect of the present application provides a non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute the method for generating a spatial position of a vehicle according to the first aspect.
One embodiment in the above application has the following advantages or benefits: the method has the advantages that the surrounding frame projection line of the 2D surrounding frame of the vehicle is generated according to the image of the vehicle, the 2D coordinate system of the vehicle relative to the ground plane is established according to the surrounding frame projection line, the coordinate optimization value and the orientation angle optimization value are generated by optimizing the position of the central point and the orientation angle of the lower plane of the vehicle in the 2D coordinate system, the more accurate central point and the orientation angle of the bottom of the vehicle can be obtained, the accuracy of the space position of the vehicle is improved, and further, the reliability of the application based on the space position of the vehicle can be improved.
It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present application, nor do they limit the scope of the present application. Other features of the present application will become apparent from the following description.
Drawings
The drawings are included to provide a better understanding of the present solution and are not intended to limit the present application. Wherein:
fig. 1 is a schematic flowchart of a method for generating a spatial position of a vehicle according to an embodiment of the present disclosure;
FIG. 2 is a schematic flow chart illustrating another method for generating a spatial position of a vehicle according to an embodiment of the present disclosure;
FIG. 3 is a schematic flow chart illustrating another method for generating a spatial location of a vehicle according to an embodiment of the present disclosure;
FIG. 4 is a schematic flow chart illustrating another method for generating a spatial location of a vehicle according to an embodiment of the present disclosure;
fig. 5 is a schematic structural diagram of a spatial position generating device of a vehicle according to an embodiment of the present application;
fig. 6 is a schematic structural diagram of another spatial position generating apparatus for a vehicle according to an embodiment of the present application;
FIG. 7 illustrates a block diagram of an exemplary electronic device suitable for use in implementing embodiments of the present application.
Detailed Description
The following description of the exemplary embodiments of the present application, taken in conjunction with the accompanying drawings, includes various details of the embodiments of the application for the understanding of the same, which are to be considered exemplary only. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present application. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
Fig. 1 is a schematic flowchart of a method for generating a spatial position of a vehicle according to an embodiment of the present application, where as shown in fig. 1, the method includes:
The method for generating the spatial position of the vehicle can be applied to estimation of the position and the posture of the vehicle.
In this embodiment, the image of the vehicle is collected by the image collecting device, the 2D bounding box of the vehicle in the image is obtained according to the image of the vehicle, and then, the projection is performed according to the 2D bounding box, so as to generate a bounding box projection line of the 2D bounding box. The image acquisition device is a camera, for example, the 2D bounding box of the vehicle is used for representing the area of the vehicle in the image, and the bounding box projection line is obtained by projecting the edge of the 2D bounding box of the vehicle. The shape of the 2D bounding box in this embodiment may be a rectangle, or may be other polygons, which is not limited herein.
In one embodiment of the present application, generating bounding box projection lines of a 2D bounding box of a vehicle from an image of the vehicle comprises: the method includes detecting a vehicle in an image, generating a 2D bounding box position of the vehicle, and projecting the 2D bounding box position of the vehicle to a ground plane to generate a bounding box projection line. Alternatively, the vehicle in the image may be detected by means of object detection to generate a 2D bounding box position of the vehicle in the image.
It should be noted that the implementation of generating the bounding box projection line of the 2D bounding box is only an example and is not limited herein.
And 102, acquiring size information of the vehicle according to the image.
In this embodiment, in order to generate the spatial position of the vehicle, the size information of the vehicle may also be acquired from the image. The implementation manner of obtaining the vehicle size information may be implemented in various manners, and as an example, after the image of the vehicle is obtained, the image of the vehicle may be processed through a deep neural network to obtain the size information of the vehicle.
The size information includes, for example, the length, width, and height of the vehicle.
And 103, establishing a 2D coordinate system of the vehicle relative to the ground plane according to the projection line of the enclosure frame.
In this embodiment, after the projection line of the bounding box is acquired, the projection line of the bounding box is taken as a coordinate axis u, and a coordinate axis v is established by taking the projection line of the bounding box as a direction perpendicular to the projection line of the bounding box, so as to establish a u-v coordinate system, and the u-v coordinate system is taken as a 2D coordinate system of the vehicle on the ground plane.
Alternatively, when there are a plurality of vehicles, one 2D coordinate system is established for each vehicle.
And 104, optimizing the central point position and the orientation angle of the lower plane of the vehicle in a 2D coordinate system to generate a coordinate optimized value and an orientation angle optimized value.
In this embodiment, the image of the vehicle may be recognized to obtain the heading angle of the vehicle, and obtain the center point position of the lower plane of the vehicle, for example, the plane where the bottom of the vehicle is located. Alternatively, the orientation angle and the center point position are obtained as estimated values. The accuracy of the position and attitude estimation of the vehicle is further improved by optimizing the center point position and the orientation angle of the lower plane of the vehicle in the 2D coordinate system to generate a more accurate coordinate optimized value and orientation angle optimized value, as an example, by establishing a loss function to optimize the center point position and the orientation angle of the lower plane of the vehicle to generate a coordinate optimized value and an orientation angle optimized value.
The optimized coordinate value is the position of the center point of the lower plane of the optimized vehicle, the optimized coordinate value is, for example, the coordinate in the 2D coordinate system, and the optimized orientation angle value is the optimized orientation angle of the vehicle.
And 105, generating the space position of the vehicle according to the coordinate optimization value, the orientation angle optimization value and the size information of the vehicle.
In this embodiment, after obtaining the coordinate optimized value, the orientation angle optimized value, and the size information of the vehicle, the spatial position of the vehicle may be generated in combination with a 2D coordinate system of the vehicle relative to the ground plane. As an example, (coordinate optimized value, orientation angle optimized value) is (u)b,vb,ryb) The length, width and height of the vehicle are respectively le,we,heIn combination with the 2D coordinate system (u-v), the spatial position of the vehicle can be solved.
According to the spatial position generating method of the vehicle, the surrounding frame projection line of the 2D surrounding frame of the vehicle is generated according to the image of the vehicle; acquiring size information of the vehicle according to the image; establishing a 2D coordinate system of the vehicle relative to the ground plane according to the projection line of the enclosure frame; optimizing the position of a center point and an orientation angle of a lower plane of the vehicle in a 2D coordinate system to generate a coordinate optimized value and an orientation angle optimized value; and generating the spatial position of the vehicle according to the coordinate optimization value, the orientation angle optimization value and the size information of the vehicle. According to the method and the device, the position of the central point and the orientation angle of the lower plane of the vehicle are optimized under the 2D coordinate system to generate the coordinate optimized value and the orientation angle optimized value, the more accurate central point and the orientation angle of the bottom of the vehicle can be obtained, and the accuracy of the space position of the vehicle is improved. Furthermore, the spatial position of the vehicle is applied to intelligent traffic applications such as vehicle-road coordination and unmanned driving, more accurate road condition information can be provided for the applications, and the reliability of the applications based on the spatial position of the vehicle is improved.
Based on the above embodiments, the present application may detect the vehicle in the image by means of object detection to generate the 2D surrounding frame position of the vehicle in the image. The following further describes the generation of the bounding box projection lines of the 2D bounding box of the vehicle from the image of the vehicle and the establishment of the 2D coordinate system of the vehicle relative to the ground plane from the bounding box projection lines in the foregoing embodiments.
Fig. 2 is a schematic flowchart of another method for generating a spatial position of a vehicle according to an embodiment of the present application, where as shown in fig. 2, the method includes:
In this embodiment, the vehicle in the image may be detected by means of object detection in computer vision, so as to generate a 2D bounding box position of the vehicle in the image.
As an example, a sample image including a vehicle is collected in advance, an annotation frame of the vehicle is annotated in the sample image, an object detection model is trained through the sample image, the object detection model is input as an image, and the image is output as a detection frame of the vehicle in the image. And generating the 2D surrounding frame position of the vehicle in the object detection model with the trained acquired vehicle image input value.
The generation of the bounding box projection line in this embodiment is exemplified below.
As one example, ground equations and camera parameters are obtained, and the 2D bounding box position of the vehicle is projected to the ground plane according to the ground equations and camera parameters to generate a bounding box projection line. Optionally, the 2D bounding box is a rectangular box, and then the left and right lower boundaries of the 2D bounding box are respectively projected to the ground plane to generate a bounding box projection line. The ground equation refers to a ground equation of a ground plane in a scene under a camera coordinate system, and the camera parameters include internal parameters of the camera.
In this example, the camera calibration technique may be used to obtain the internal reference of the camera, and the ground equation may be obtained according to ground modeling in combination with the external reference calibration technique. The projection from each point in the image to the ground plane can be realized by acquiring the camera parameters and the ground equation, then the vehicle in the image is detected by adopting an object detection method, the position of the 2D surrounding frame of the vehicle is generated, and the position of the 2D surrounding frame of the vehicle is projected to the ground plane according to the ground equation and the camera parameters, so that the projection line of the surrounding frame is generated.
It should be noted that, for a scene, one ground equation may be used, or modeling may be performed at different positions by using different ground equations, which is not limited herein.
And step 203, establishing a 2D coordinate system of the vehicle relative to the ground plane according to the projection line of the enclosure frame.
The establishment of the 2D coordinate system in this embodiment will be exemplified below.
As an example, the 2D bounding box is a rectangular box, and the number of bounding box projection lines is three, i.e. the bounding box projection line comprises a first bounding box projection linelThe projection line of the second enclosure framerAnd a third bounding box projection linebWherein the first bounding box projection linelThe projection line of the second enclosure framerRespectively obtained by projecting the left and right boundaries of a rectangular frame of the 2D bounding box and a projection line of a third bounding boxbThe method is obtained by projecting the lower boundary of the rectangular frame of the 2D bounding box.
In this example, the hatched line along the third bounding boxbAnd establishing a coordinate axis u in the direction, and establishing a coordinate axis v on the ground plane equation in a vertical direction u. Optionally, a coordinate axis v is established on the side perpendicular to u and at an acute angle to the direction of the line of sight on the ground plane equation, in 2D for the origin of the coordinate systemAnd the projection point of the lower left vertex of the bounding box on the ground plane is a coordinate system origin, wherein the coordinate system origin can be obtained by calculation according to the lower left vertex, the camera internal reference and the ground equation.
In the embodiment of the application, the projection lines of the surrounding frame projected to the ground plane from the lower side and the left side and the right side of the 2D surrounding frame are obtained, the 2D coordinate system of the vehicle relative to the ground plane is established according to the projection lines of the surrounding frame, the relation between the 2D surrounding frame and the ground equation is established, and support is provided for optimization of the loss function. Moreover, the sampling based on the 2D coordinate system is more suitable for the physical meaning of the scene.
Based on the foregoing embodiment, further, in this embodiment, the bottom surface center point sampling value and the orientation angle sampling value may be optimized according to the loss function by obtaining the center point sampling value and the orientation angle sampling value, and the generation of the coordinate optimized value and the orientation angle optimized value in the foregoing embodiment is described below.
Fig. 3 is a schematic flowchart of another method for generating a spatial position of a vehicle according to an embodiment of the present application, where as shown in fig. 3, the method includes:
in step 301, an estimated value of the heading angle of the vehicle is obtained according to the image.
In this embodiment, when generating the orientation angle optimized value of the vehicle, the orientation angle predicted value of the vehicle is obtained according to the image, for example, the orientation angle predicted value ry of the vehicle may be obtained through a deep neural network according to the image of the vehiclee。
And step 302, acquiring a bottom center point of the vehicle in a 2D coordinate system, and sampling in a search space according to the bottom center point and the orientation angle estimated value to generate a bottom center point sampling value and an orientation angle sampling value.
The bottom center point refers to the center point of a plane where the bottom of the vehicle is located, and the bottom center point is in a 2D coordinate system.
In this embodiment, the search space refers to the range of samples, and the search space may be represented by (u)min,umax)*(vmin,vmax)*(rymim,ry,ax) Definition of wherein umin,umax，vmin,vmax，rymin,rymaxThe 2D coordinate u minimum, u maximum, v minimum, v maximum, orientation angle minimum, and orientation angle maximum are respectively referred to, and may be specifically determined as required.
In this embodiment, the position of the center point of the bottom surface of the vehicle is sampled in a 2D coordinate system (u-v), and the vehicle orientation angle is sampled, and optionally, N values (N is a positive integer) and the sampled value (u) may be sampled in a search spacet,vt,ryt) As the bottom center point sample and the orientation angle sample. Sampling is carried out under the constraint of a ground equation, so that the method is more suitable for the physical significance of an application scene.
And 303, optimizing the sampling value of the center point of the bottom surface and the sampling value of the orientation angle according to the loss function to generate a coordinate optimized value and an orientation angle optimized value.
In this embodiment, a loss function is established in advance, for each sampled value of the N sampled values, a loss value is calculated by the loss function, and a sampling value whose loss value satisfies a preset condition is selected as a coordinate optimized value and an orientation angle optimized value. Wherein the loss function takes the distance between the vertex of the 3D bounding box and the projection line of the bounding box as a constraint condition.
As an example, the bounding box projection line number is three, and the loss function is as follows:
Cost(ut,vt,ryt)＝dist(pl,linel)+dist(pr,liner)+dist(pb,lineb)+(rye-ryt)2
the Cost is a loss value, 4 vertexes of the bottom surface of the 3D vehicle surrounding frame can be determined according to the sampling value and the length, width and height of the vehicle for each sampling value, and then the 4 vertexes of the bottom surface of the 3D vehicle surrounding frame are projected back to the image of the vehicle by combining the camera internal parameters to obtain four projection points. Determining projection points which are closest to the left, right and lower boundaries of the 2D bounding box in the image of the vehicle, and respectively recording the projection points as p on the ground plane equationl，pr,pb。
As a possible implementation, forCalculating loss values of the bottom surface center point sampling value and the orientation angle sampling value, determining a minimum loss value from the loss values corresponding to the sampling values, and taking the sampling value corresponding to the minimum loss value as an optimized value, namely determining a coordinate optimized value and an orientation angle optimized value through the following formulas: (u)b,vb,ryb)＝argmin(Cost(ut,vt,ryt) Wherein (u)b,vb,ryb) A coordinate optimization value and an orientation angle optimization value.
In one embodiment of the present application, the above (u) is obtainedb,vb,ryb) Then, (u) can also beb,vb,ryb) As a sampling center, the next round of sampling iteration is carried out after the search space is reduced to obtain new (u)b,vb,ryb). The iteration process is repeated until the preset iteration times are met, and finally obtained (u) is recordedb,vb,ryb) As a coordinate optimized value and an orientation angle optimized value.
In the embodiment of the application, by obtaining the orientation angle estimated value of the vehicle according to the image, sampling is carried out in the search space to generate the bottom surface center point sampling value and the orientation angle sampling value, optimizing the bottom surface center point sample value and the orientation angle sample value according to a loss function to generate a coordinate optimized value and an orientation angle optimized value, because the relative estimation accuracy of the 2D bounding box is higher in the application of the visual perception technology, the accuracy of the central point and the orientation angle of the bottom of the vehicle can be improved by introducing the 2D bounding box for constraint, the problem of poor optimization effect caused by the large and small distances caused by the perspective effect is solved, compared with the method that eight vertexes of the 3D bounding box are projected back to a 2D image, the minimum 2D bounding limited by eight projection points is calculated, the accuracy of the spatial position of the vehicle is improved by the scheme of intersection ratio between the 2D surrounding frames output by the detector.
Fig. 4 is a schematic flowchart of another method for generating a spatial position of a vehicle according to an embodiment of the present disclosure.
As an example, refer to fig. 4, wherein a 2D image containing a vehicle is captured by a camera, an object detection is performed on the 2D image to obtain a 2D bounding box of the vehicle in the image, and estimated values of length, width, height and orientation angle of the vehicle are obtained by a deep neural network; and determining camera internal parameters and a ground equation through camera calibration and ground equation estimation. Further, a bounding box projection line is generated according to the 2D bounding box, the camera internal parameters and the ground equation, and a uv coordinate system is established on the ground plane. Further, the coordinates and the orientation angles of the center point of the bottom surface of the vehicle are sampled in a sampling range based on a uv coordinate system, the sampling values are optimized through a loss function by combining the projection line of the surrounding frame, the length, the width, the height and the orientation angle estimated value to generate the optimized coordinates and the optimized orientation angles of the center point of the bottom surface, and the optimization process is iterated to obtain the final optimized coordinates and optimized orientation angles. And generating the space position of the vehicle according to the coordinate optimization value, the orientation angle optimization value, the length, the width, the height and the uv coordinate system. This can improve the accuracy of the spatial position of the vehicle, and further improve the reliability of the application based on the spatial position of the vehicle.
In order to implement the above embodiments, the present application also proposes a spatial position generating device of a vehicle.
Fig. 5 is a schematic structural diagram of a spatial position generating device of a vehicle according to an embodiment of the present application, and as shown in fig. 5, the device includes: the system comprises a first generation module 50, an acquisition module 51, a building module 52, an optimization module 53 and a second generation module 54.
The first generation module 50 is configured to generate a bounding box projection line of a 2D bounding box of a vehicle according to an image of the vehicle.
And an obtaining module 51, configured to obtain size information of the vehicle according to the image.
An establishing module 52, configured to establish a 2D coordinate system of the vehicle relative to the ground plane according to the projection line of the enclosure frame.
And an optimizing module 53, configured to optimize the center point position and the orientation angle of the lower plane of the vehicle in the 2D coordinate system to generate a coordinate optimized value and an orientation angle optimized value.
And a second generating module 54, configured to generate a spatial position of the vehicle according to the coordinate optimized value and the orientation angle optimized value, and the size information of the vehicle.
In an embodiment of the present application, on the basis of fig. 5, as shown in fig. 6, the first generating module 50 includes: a detection unit 501, configured to detect a vehicle in an image and generate a 2D bounding box position of the vehicle; and a projection unit 502 for projecting the 2D bounding box position of the vehicle to a ground plane to generate a bounding box projection line.
In an embodiment of the present application, the projection unit 502 is specifically configured to: acquiring a ground equation and camera parameters; projecting the 2D bounding box position of the vehicle to a ground plane according to the ground equation and camera parameters to generate a bounding box projection line.
In one embodiment of the present application, the bounding box projection line comprises a first bounding box projection linelThe projection line of the second enclosure framerAnd a third bounding box projection linebThe establishing module 52 is specifically configured to: a hatched line along the third bounding boxbAnd establishing a coordinate axis u in the direction, and establishing a coordinate axis v on the ground plane equation in a vertical direction u.
In an embodiment of the present application, the optimization module 53 is specifically configured to: obtaining an orientation angle estimated value of the vehicle according to the image; acquiring a bottom surface center point of the vehicle in the 2D coordinate system, and sampling in a search space according to the bottom surface center point and the orientation angle predicted value to generate a bottom surface center point sampling value and an orientation angle sampling value; and optimizing the bottom surface center point sampling value and the orientation angle sampling value according to a loss function to generate the coordinate optimized value and the orientation angle optimized value.
The explanation of the spatial position generating method of the vehicle in the foregoing embodiment is also applicable to the spatial position generating device of the vehicle in the present embodiment, and will not be described again here.
The spatial position generating device of the vehicle of the embodiment of the application generates the surrounding frame projection line of the 2D surrounding frame of the vehicle according to the image of the vehicle; acquiring size information of the vehicle according to the image; establishing a 2D coordinate system of the vehicle relative to the ground plane according to the projection line of the enclosure frame; optimizing the position of a center point and an orientation angle of a lower plane of the vehicle in a 2D coordinate system to generate a coordinate optimized value and an orientation angle optimized value; and generating the spatial position of the vehicle according to the coordinate optimization value, the orientation angle optimization value and the size information of the vehicle. According to the method and the device, the more accurate central point and orientation angle of the bottom of the vehicle can be obtained, and the accuracy of the spatial position of the vehicle is improved. Furthermore, the spatial position of the vehicle is applied to intelligent traffic applications such as vehicle-road coordination and unmanned driving, more accurate road condition information can be provided for the applications, and the reliability of the applications based on the spatial position of the vehicle is improved.
In order to implement the above embodiments, the present application also proposes a computer program product, wherein when the instructions of the computer program product are executed by a processor, the spatial position generating method of the vehicle according to any of the foregoing embodiments is implemented.
According to an embodiment of the present application, an electronic device and a readable storage medium are also provided.
As shown in fig. 7, the present invention is a block diagram of an electronic device of a spatial position generating method of a vehicle according to an embodiment of the present application. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the present application that are described and/or claimed herein.
As shown in fig. 7, the electronic apparatus includes: one or more processors 701, a memory 702, and interfaces for connecting the various components, including a high-speed interface and a low-speed interface. The various components are interconnected using different buses and may be mounted on a common motherboard or in other manners as desired. The processor may process instructions for execution within the electronic device, including instructions stored in or on the memory to display graphical information of a GUI on an external input/output apparatus (such as a display device coupled to the interface). In other embodiments, multiple processors and/or multiple buses may be used, along with multiple memories and multiple memories, as desired. Also, multiple electronic devices may be connected, with each device providing portions of the necessary operations (e.g., as a server array, a group of blade servers, or a multi-processor system). In fig. 7, one processor 701 is taken as an example.
The memory 702 is a non-transitory computer readable storage medium as provided herein. Wherein the memory stores instructions executable by at least one processor to cause the at least one processor to perform the spatial location generation method of a vehicle provided herein. The non-transitory computer-readable storage medium of the present application stores computer instructions for causing a computer to execute the spatial position generation method of a vehicle provided by the present application.
The memory 702, which is a non-transitory computer-readable storage medium, may be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as program instructions/modules corresponding to the spatial position generating method of the vehicle in the embodiment of the present application (for example, the first generating module 50, the obtaining module 51, the establishing module 52, the optimizing module 53, and the second generating module 54 shown in fig. 5). The processor 701 executes various functional applications of the server and data processing by running non-transitory software programs, instructions, and modules stored in the memory 702, that is, implements the spatial position generating method of the vehicle in the above-described method embodiment.
The memory 702 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created according to use of the electronic device, and the like. Further, the memory 702 may include high speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, the memory 702 may optionally include memory located remotely from the processor 701, which may be connected to the electronic device via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The electronic device of the spatial position generating method of the vehicle may further include: an input device 703 and an output device 704. The processor 701, the memory 702, the input device 703 and the output device 704 may be connected by a bus or other means, and fig. 7 illustrates an example of a connection by a bus.
The input device 703 may receive input numeric or character information and generate key signal inputs related to user settings and function control of the electronic apparatus, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, a pointing stick, one or more mouse buttons, a track ball, a joystick, or other input devices. The output devices 704 may include a display device, auxiliary lighting devices (e.g., LEDs), and tactile feedback devices (e.g., vibrating motors), among others. The display device may include, but is not limited to, a Liquid Crystal Display (LCD), a Light Emitting Diode (LED) display, and a plasma display. In some implementations, the display device can be a touch screen.
Various implementations of the systems and techniques described here can be realized in digital electronic circuitry, integrated circuitry, application specific ASICs (application specific integrated circuits), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.
These computer programs (also known as programs, software applications, or code) include machine instructions for a programmable processor, and may be implemented using high-level procedural and/or object-oriented programming languages, and/or assembly/machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and/or device (e.g., magnetic discs, optical disks, memory, Programmable Logic Devices (PLDs)) used to provide machine instructions and/or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal used to provide machine instructions and/or data to a programmable processor.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), Wide Area Networks (WANs), and the Internet.
The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so that the defects of high management difficulty and weak service expansibility in the traditional physical host and VPS service are overcome.
It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present application may be executed in parallel, sequentially, or in different orders, and the present invention is not limited thereto as long as the desired results of the technical solutions disclosed in the present application can be achieved.
The above-described embodiments should not be construed as limiting the scope of the present application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.
Claims (12)
1. A spatial position generation method of a vehicle, comprising:
generating a bounding box projection line of a 2D bounding box of a vehicle from an image of the vehicle;
acquiring size information of the vehicle according to the image;
establishing a 2D coordinate system of the vehicle relative to the ground plane according to the surrounding frame projection line;
optimizing the center point position and the orientation angle of the lower plane of the vehicle in the 2D coordinate system to generate a coordinate optimized value and an orientation angle optimized value; and
and generating the space position of the vehicle according to the coordinate optimization value, the orientation angle optimization value and the size information of the vehicle.
2. The spatial position generating method of a vehicle according to claim 1, wherein the generating a bounding box projection line of a 2D bounding box of the vehicle from an image of the vehicle includes:
detecting a vehicle in an image and generating a 2D surrounding frame position of the vehicle; and
projecting the 2D bounding box location of the vehicle to a ground plane to generate a bounding box projection line.
3. The spatial location generation method of a vehicle of claim 2, wherein the projecting the 2D bounding box location of the vehicle to a ground plane to generate a bounding box projection line comprises:
acquiring a ground equation and camera parameters;
projecting the 2D bounding box position of the vehicle to a ground plane according to the ground equation and camera parameters to generate a bounding box projection line.
4. The spatial position generating method of a vehicle according to claim 1, wherein the projection line of the enclosure frame includes a first projection line of the enclosure framelThe projection line of the second enclosure framerAnd a third bounding box projection linebEstablishing a 2D coordinate system of the vehicle relative to the ground plane according to the bounding box projection lines, comprising:
a hatched line along the third bounding boxbAnd establishing a coordinate axis u in the direction, and establishing a coordinate axis v on the ground plane equation in a vertical direction u.
5. The spatial position generating method of a vehicle according to claim 1, wherein the optimizing the center point position and the orientation angle of the lower plane of the vehicle in the 2D coordinate system to generate a coordinate optimized value and an orientation angle optimized value comprises:
obtaining an orientation angle estimated value of the vehicle according to the image;
acquiring a bottom surface center point of the vehicle in the 2D coordinate system, and sampling in a search space according to the bottom surface center point and the orientation angle predicted value to generate a bottom surface center point sampling value and an orientation angle sampling value;
and optimizing the bottom surface center point sampling value and the orientation angle sampling value according to a loss function to generate the coordinate optimized value and the orientation angle optimized value.
6. A spatial position generating apparatus of a vehicle, comprising:
a first generating module for generating a bounding box projection line of a 2D bounding box of a vehicle from an image of the vehicle;
the acquisition module is used for acquiring the size information of the vehicle according to the image;
the establishing module is used for establishing a 2D coordinate system of the vehicle relative to the ground plane according to the surrounding frame projection line;
an optimization module for optimizing a center point position and an orientation angle of a lower plane of the vehicle in the 2D coordinate system to generate a coordinate optimized value and an orientation angle optimized value; and
and the second generation module is used for generating the spatial position of the vehicle according to the coordinate optimization value, the orientation angle optimization value and the size information of the vehicle.
7. The spatial position generating apparatus of a vehicle according to claim 6, wherein the first generating module includes:
a detection unit for detecting a vehicle in an image and generating a 2D bounding box position of the vehicle; and
a projection unit for projecting the 2D bounding box position of the vehicle to a ground plane to generate a bounding box projection line.
8. The spatial position generating apparatus of a vehicle according to claim 7, wherein the projection unit is specifically configured to:
acquiring a ground equation and camera parameters;
projecting the 2D bounding box position of the vehicle to a ground plane according to the ground equation and camera parameters to generate a bounding box projection line.
9. The spatial position generating apparatus of a vehicle according to claim 6, wherein the projection line of the enclosure frame includes a first projection line of the enclosure framelThe projection line of the second enclosure framerAnd a third bounding box projection linebThe establishing module is specifically configured to:
a hatched line along the third bounding boxbAnd establishing a coordinate axis u in the direction, and establishing a coordinate axis v on the ground plane equation in a vertical direction u.
10. The spatial position generating apparatus of a vehicle according to claim 6, wherein the optimizing module is specifically configured to:
obtaining an orientation angle estimated value of the vehicle according to the image;
acquiring a bottom surface center point of the vehicle in the 2D coordinate system, and sampling in a search space according to the bottom surface center point and the orientation angle predicted value to generate a bottom surface center point sampling value and an orientation angle sampling value;
and optimizing the bottom surface center point sampling value and the orientation angle sampling value according to a loss function to generate the coordinate optimized value and the orientation angle optimized value.
11. An electronic device, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of generating a spatial location of a vehicle of any of claims 1-5.
12. A non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the spatial position generation method of a vehicle of any one of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010605263.0A CN111968071A (en) | 2020-06-29 | 2020-06-29 | Method, device, equipment and storage medium for generating spatial position of vehicle |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010605263.0A CN111968071A (en) | 2020-06-29 | 2020-06-29 | Method, device, equipment and storage medium for generating spatial position of vehicle |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111968071A true CN111968071A (en) | 2020-11-20 |
Family
ID=73360754
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010605263.0A Pending CN111968071A (en) | 2020-06-29 | 2020-06-29 | Method, device, equipment and storage medium for generating spatial position of vehicle |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111968071A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112926378A (en) * | 2021-01-04 | 2021-06-08 | 广州小鹏自动驾驶科技有限公司 | Vehicle side edge determining method and device |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20130024125A (en) * | 2011-08-30 | 2013-03-08 | 삼성전기주식회사 | Apparatus for controlling exposure of stereo-camera, stereo-camera and method for controlling exposure thereof |
| US20170024889A1 (en) * | 2015-07-23 | 2017-01-26 | International Business Machines Corporation | Self-calibration of a static camera from vehicle information |
| US20170169300A1 (en) * | 2015-12-15 | 2017-06-15 | Honda Motor Co., Ltd. | System and method for image based vehicle localization |
| WO2018235300A1 (en) * | 2017-06-23 | 2018-12-27 | 日本電気株式会社 | Object detection device, object detection method, and computer-readable recording medium |
| CN109358622A (en) * | 2018-10-12 | 2019-02-19 | 华北科技学院 | Localization method, electronic equipment and the computer readable storage medium of robot label |
| CN109872372A (en) * | 2019-03-07 | 2019-06-11 | 山东大学 | A kind of small-sized quadruped robot overall Vision localization method and system |
-
2020
- 2020-06-29 CN CN202010605263.0A patent/CN111968071A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20130024125A (en) * | 2011-08-30 | 2013-03-08 | 삼성전기주식회사 | Apparatus for controlling exposure of stereo-camera, stereo-camera and method for controlling exposure thereof |
| US20170024889A1 (en) * | 2015-07-23 | 2017-01-26 | International Business Machines Corporation | Self-calibration of a static camera from vehicle information |
| US20170169300A1 (en) * | 2015-12-15 | 2017-06-15 | Honda Motor Co., Ltd. | System and method for image based vehicle localization |
| WO2018235300A1 (en) * | 2017-06-23 | 2018-12-27 | 日本電気株式会社 | Object detection device, object detection method, and computer-readable recording medium |
| CN109358622A (en) * | 2018-10-12 | 2019-02-19 | 华北科技学院 | Localization method, electronic equipment and the computer readable storage medium of robot label |
| CN109872372A (en) * | 2019-03-07 | 2019-06-11 | 山东大学 | A kind of small-sized quadruped robot overall Vision localization method and system |
Non-Patent Citations (3)
| Title |
|---|
| 伍舜喜: "基于激光雷达的智能车定位技术研究", 中国优秀硕士学位论文全文数据库, 15 June 2008 (2008-06-15) * |
| 洪洋;孙秀霞;王栋;刘树光;王瀚林;: "基于矩形几何特性的小型无人机快速位姿估计方法", 中国激光, no. 05, 10 May 2016 (2016-05-10) * |
| 邹斌;袁宇翔;: "面向智能交通的单目视觉测距方法研究", 交通运输系统工程与信息, no. 04, 15 August 2018 (2018-08-15) * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112926378A (en) * | 2021-01-04 | 2021-06-08 | 广州小鹏自动驾驶科技有限公司 | Vehicle side edge determining method and device |
| CN112926378B (en) * | 2021-01-04 | 2022-06-28 | 广州小鹏自动驾驶科技有限公司 | Vehicle side edge determining method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111311925B (en) | Parking space detection method and device, electronic equipment, vehicle and storage medium | |
| KR20200005999A (en) | Slam method and slam system using dual event camaer | |
| CN112132829A (en) | Vehicle information detection method and device, electronic equipment and storage medium | |
| CN111612753B (en) | Three-dimensional object detection method and device, electronic equipment and readable storage medium | |
| CN110675635B (en) | Method and device for acquiring external parameters of camera, electronic equipment and storage medium | |
| EP3968266B1 (en) | Obstacle three-dimensional position acquisition method and apparatus for roadside computing device | |
| CN111666876B (en) | Method and device for detecting obstacle, electronic equipment and road side equipment | |
| JP2021119507A (en) | Traffic lane determination method, traffic lane positioning accuracy evaluation method, traffic lane determination apparatus, traffic lane positioning accuracy evaluation apparatus, electronic device, computer readable storage medium, and program | |
| CN111578839B (en) | Obstacle coordinate processing method and device, electronic equipment and readable storage medium | |
| CN111079079B (en) | Data correction method, device, electronic equipment and computer readable storage medium | |
| CN112344855B (en) | Obstacle detection method and device, storage medium and drive test equipment | |
| CN111739005A (en) | Image detection method, image detection device, electronic equipment and storage medium | |
| CN111721281B (en) | Position identification method and device and electronic equipment | |
| EP3794312A1 (en) | Indoor location-based service | |
| CN112147632A (en) | Method, device, equipment and medium for testing vehicle-mounted laser radar perception algorithm | |
| CN111402161A (en) | Method, device and equipment for denoising point cloud obstacle and storage medium | |
| CN112184914A (en) | Method and device for determining three-dimensional position of target object and road side equipment | |
| CN111222579A (en) | Cross-camera obstacle association method, device, equipment, electronic system and medium | |
| CN112241718A (en) | Vehicle information detection method, detection model training method and device | |
| CN111353466A (en) | Lane line recognition processing method, lane line recognition processing device, and storage medium | |
| CN111191619A (en) | Method, device and equipment for detecting virtual line segment of lane line and readable storage medium | |
| CN114565908A (en) | Lane line detection method and device, electronic device and storage medium | |
| CN114186007A (en) | High-precision map generation method and device, electronic equipment and storage medium | |
| CN111783611B (en) | Unmanned vehicle positioning method and device, unmanned vehicle and storage medium | |
| CN113126120A (en) | Data annotation method, device, equipment, storage medium and computer program product |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |