CN117095052A - Vehicle relative pose matrix optimization method and device, electronic equipment and readable medium - Google Patents

Vehicle relative pose matrix optimization method and device, electronic equipment and readable medium Download PDF

Info

Publication number
CN117095052A
CN117095052A CN202310820813.4A CN202310820813A CN117095052A CN 117095052 A CN117095052 A CN 117095052A CN 202310820813 A CN202310820813 A CN 202310820813A CN 117095052 A CN117095052 A CN 117095052A
Authority
CN
China
Prior art keywords
sequence
lane marking
lane
coordinate
detection information
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310820813.4A
Other languages
Chinese (zh)
Inventor
胡禹超
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
HoloMatic Technology Beijing Co Ltd
Original Assignee
HoloMatic Technology Beijing Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by HoloMatic Technology Beijing Co Ltd filed Critical HoloMatic Technology Beijing Co Ltd
Priority to CN202310820813.4A priority Critical patent/CN117095052A/en
Publication of CN117095052A publication Critical patent/CN117095052A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30248—Vehicle exterior or interior
    • G06T2207/30252—Vehicle exterior; Vicinity of vehicle
    • G06T2207/30256—Lane; Road marking

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Traffic Control Systems (AREA)

Abstract

The embodiment of the disclosure discloses a vehicle relative pose matrix optimization method, a device, electronic equipment and a readable medium. One embodiment of the method comprises the following steps: acquiring a road image sequence in a preset sliding window and a current vehicle pose matrix sequence corresponding to each road image; carrying out lane marking detection on each road image in the road image sequence to generate a lane marking area detection information group, and obtaining a lane marking area detection information group sequence; determining a center point coordinate set sequence of a short side of the lane marking corresponding to each lane marking region detection information in the lane marking region detection information set sequence; constructing a lane marking constraint equation set; and carrying out optimization processing on each current vehicle pose matrix in the current vehicle pose matrix sequence based on the lane marking constraint equation set so as to generate an optimized vehicle relative pose matrix sequence. The embodiment can improve the accuracy of the generated relative pose matrix of the vehicle.

Description

Vehicle relative pose matrix optimization method and device, electronic equipment and readable medium
Technical Field
Embodiments of the present disclosure relate to the field of computer technology, and in particular, to a vehicle relative pose matrix optimization method, apparatus, electronic device, and readable medium.
Background
The vehicle relative pose matrix is a matrix of the position pose of the vehicle body with respect to the ground in the vicinity of the vehicle body. Currently, in generating a vehicle relative pose matrix, the following methods are generally adopted: on the premise of taking the ground around the vehicle as a plane, determining a relative pose matrix of the vehicle in a projective transformation mode.
However, the inventors have found that when the vehicle relative pose matrix is optimized in the above manner, there are often the following technical problems:
because the ground is not in an ideal plane state, if the vehicle bumps in the actual running process, the relative angle between the vehicle and the ground is offset (such as roll angle and pitch angle offset), so that the optimization of the relative pose matrix of the vehicle is difficult, and the accuracy of the relative pose matrix of the vehicle generated by a projective transformation mode is reduced.
The above information disclosed in this background section is only for enhancement of understanding of the background of the inventive concept and, therefore, may contain information that does not form the prior art that is already known to those of ordinary skill in the art in this country.
Disclosure of Invention
The disclosure is in part intended to introduce concepts in a simplified form that are further described below in the detailed description. The disclosure is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.
Some embodiments of the present disclosure propose a vehicle relative pose matrix optimization method, apparatus, electronic device, and readable medium to solve the technical problems mentioned in the background section above.
In a first aspect, some embodiments of the present disclosure provide a vehicle relative pose matrix optimization method, the method comprising: acquiring a road image sequence in a preset sliding window and a current vehicle pose matrix sequence corresponding to each road image; carrying out lane marking detection on each road image in the road image sequence to generate a lane marking area detection information group, and obtaining a lane marking area detection information group sequence; determining a center point coordinate group sequence of a short side of the lane marking corresponding to each lane marking region detection information in the lane marking region detection information group sequence; constructing a lane marking constraint equation set based on the lane marking region detection information set sequence and the determined lane marking short-side midpoint coordinate set sequence; and carrying out optimization processing on each current vehicle pose matrix in the current vehicle pose matrix sequence based on the lane marking constraint equation set so as to generate an optimized vehicle relative pose matrix sequence.
In a second aspect, some embodiments of the present disclosure provide a vehicle relative pose matrix optimization apparatus, the apparatus comprising: the acquisition unit is configured to acquire a road image sequence in a preset sliding window and a current vehicle pose matrix sequence corresponding to each road image; a lane mark detection unit configured to perform lane mark detection on each road image in the road image sequence to generate a lane mark region detection information set, and obtain a lane mark region detection information set sequence; a determining unit configured to determine a lane-marking short-side midpoint coordinate group sequence corresponding to each lane-marking region detection information in the lane-marking region detection information group sequence; a construction unit configured to construct a lane marking constraint equation set based on the lane marking region detection information set sequence and the determined lane marking short side midpoint coordinate set sequence; and the optimization processing unit is configured to perform optimization processing on each current vehicle pose matrix in the current vehicle pose matrix sequence based on the lane marking constraint equation set so as to generate an optimized vehicle relative pose matrix sequence.
In a third aspect, some embodiments of the present disclosure provide an electronic device comprising: one or more processors; a storage device having one or more programs stored thereon, which when executed by one or more processors causes the one or more processors to implement the method described in any of the implementations of the first aspect above.
In a fourth aspect, some embodiments of the present disclosure provide a computer readable medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the method described in any of the implementations of the first aspect above.
The above embodiments of the present disclosure have the following advantageous effects: by the vehicle relative pose matrix optimization method, the accuracy of the optimized vehicle relative pose matrix can be improved. Specifically, the reduced accuracy of the generated vehicle relative pose matrix is caused by: because the ground is not in an ideal plane state, if the vehicle bumps in the actual running process, the relative angle between the vehicle and the ground is offset (such as roll angle and pitch angle offset), so that the optimization of the relative pose matrix of the vehicle is difficult, and the accuracy of the relative pose matrix of the vehicle generated by a projective transformation mode is reduced. Based on this, in some embodiments of the present disclosure, a vehicle relative pose matrix optimization method first obtains a road image sequence within a preset sliding window and a current vehicle pose matrix sequence corresponding to each road image. By introducing road images of successive frames in the sliding window, extraction of successive features can be facilitated. Meanwhile, the current vehicle pose matrix corresponding to the road image is introduced, so that optimization can be conveniently carried out. Then, the lane marking detection is performed on each road image in the road image sequence to generate a lane marking region detection information set, and a lane marking region detection information set sequence is obtained. Through the lane marking detection, lane marking area detection information for optimizing the current vehicle pose matrix can be obtained. And then, determining a center point coordinate set sequence of the short sides of the lane marking corresponding to each lane marking area detection information in the lane marking area detection information set sequence. The current vehicle pose matrix can be conveniently optimized by utilizing the characteristics of the lane mark by generating the coordinate set sequence of the midpoint of the short side of the lane mark. And then, constructing a lane marking constraint equation set based on the lane marking region detection information set sequence and the determined lane marking short side midpoint coordinate set sequence. By constructing a system of lane-marking constraint equations, features of lane markings can be introduced into the constraint equations for feature constraint in the optimization process. Thereby, it is convenient to correct the position posture of the vehicle with respect to the ground in the bumpy state. And finally, based on the lane marking constraint equation set, carrying out optimization processing on each current vehicle pose matrix in the current vehicle pose matrix sequence so as to generate an optimized vehicle relative pose matrix sequence. The optimization processing can be used for optimizing the relative position and the posture of the vehicle body relative to the ground in a bumpy state. Therefore, the optimization of the relative pose matrix of the vehicle is realized. Further, the accuracy of the generated vehicle relative pose matrix can be improved.
Drawings
The above and other features, advantages, and aspects of embodiments of the present disclosure will become more apparent by reference to the following detailed description when taken in conjunction with the accompanying drawings. The same or similar reference numbers will be used throughout the drawings to refer to the same or like elements. It should be understood that the figures are schematic and that elements and components are not necessarily drawn to scale.
FIG. 1 is a flow chart of some embodiments of a vehicle relative pose matrix optimization method according to the present disclosure;
FIG. 2 is a schematic structural view of some embodiments of a vehicle relative pose matrix optimization device according to the present disclosure;
fig. 3 is a schematic structural diagram of an electronic device suitable for use in implementing some embodiments of the present disclosure.
Detailed Description
Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While certain embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete. It should be understood that the drawings and embodiments of the present disclosure are for illustration purposes only and are not intended to limit the scope of the present disclosure.
It should be noted that, for convenience of description, only the portions related to the present invention are shown in the drawings. Embodiments of the present disclosure and features of embodiments may be combined with each other without conflict.
It should be noted that the terms "first," "second," and the like in this disclosure are merely used to distinguish between different devices, modules, or units and are not used to define an order or interdependence of functions performed by the devices, modules, or units.
It should be noted that references to "one", "a plurality" and "a plurality" in this disclosure are intended to be illustrative rather than limiting, and those of ordinary skill in the art will appreciate that "one or more" is intended to be understood as "one or more" unless the context clearly indicates otherwise.
The names of messages or information interacted between the various devices in the embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.
The present disclosure will be described in detail below with reference to the accompanying drawings in conjunction with embodiments.
FIG. 1 illustrates a flow 100 of some embodiments of a vehicle relative pose matrix optimization method according to the present disclosure. The vehicle relative pose matrix optimization method comprises the following steps:
Step 101, acquiring a road image sequence in a preset sliding window and a current vehicle pose matrix sequence corresponding to each road image.
In some embodiments, the execution body of the vehicle relative pose matrix optimization method may acquire, in a wired manner or a wireless manner, a road image sequence within a preset sliding window and a current vehicle pose matrix sequence corresponding to each road image. The road image sequence may be a sequence of road images of consecutive frames (e.g. 20 consecutive frames) within a preset time window. In addition, the corresponding time point of each road image is different, and the corresponding current vehicle position is also different, so that the current vehicle pose matrix detected by the corresponding positioning system is also different. Thus, the current vehicle pose matrix corresponding to the same point in time as each road image can be acquired. The current vehicle pose matrix may be used to characterize the relative position pose of the vehicle to the ground as detected by the positioning system.
It should be noted that the wireless connection may include, but is not limited to, 3G/4G connections, wiFi connections, bluetooth connections, wiMAX connections, zigbee connections, UWB (ultra wideband) connections, and other now known or later developed wireless connection means.
Step 102, carrying out lane marking detection on each road image in the road image sequence to generate a lane marking area detection information group, and obtaining a lane marking area detection information group sequence.
In some embodiments, the executing body may perform lane marking detection on each road image in the sequence of road images to generate a lane marking area detection information set, so as to obtain a sequence of lane marking area detection information sets. The lane markings may be standard lane markings on the road. Such as crosswalk lines, drain lines, etc. Each set of lane marking region detection information may be information of a lane marking detected from one road image. Each lane-marking area detection information may correspond to a set of lane markings. Such as a set of crosswalk lines in a certain direction. For road images of successive frames, the lane markings detected on the respective road images are mutually corresponding. That is, each of the lane marking detection information in each of the lane marking detection information groups may have the lane marking detection information corresponding to the same group of lane markings as the other respective lane marking detection information groups.
In some optional implementations of some embodiments, the executing body performs lane marking detection on each road image in the sequence of road images to generate a lane marking area detection information set, and may include the steps of:
The first step, carrying out area detection on the lane markings in the road image to obtain a lane marking area detection frame group and a lane marking coordinate set corresponding to each lane marking area detection frame. The method comprises the steps of carrying out region detection on the lane markings in the road image through a preset detection algorithm to obtain a lane marking region detection frame group and a lane marking coordinate set corresponding to each lane marking region detection frame. Here, each of the lane marking region detection frames in the lane marking region detection frame group may be a detection frame corresponding to each of the lane markings within the one group of lane marking regions. The lane marking coordinate set may be each coordinate of the position of the detection frame, or may be all coordinates in the region of the detection frame. In addition, each lane marking area detection frame may also have a unique identification. The lane marking area detection frame corresponding to the same lane marking can be determined from the lane marking area detection frame groups detected in other road images through the unique identification. The lane-marking coordinates may be coordinates in an image coordinate system.
As an example, the detection algorithm described above may include, but is not limited to, at least one of: FCN (Fully Convolutional Networks, full-roll neural Network) model, resnet (Residual neural Network) model, VGG (Visual Geometry Group Network, convolutional neural Network) model, google net (deep neural Network) model, and the like.
And secondly, determining each lane marking area detection frame and a corresponding lane marking coordinate set in the lane marking area detection frame group as lane marking area detection information to obtain a lane marking area detection information group.
And step 103, determining a center point coordinate set sequence of the short sides of the lane marking corresponding to each lane marking area detection information in the lane marking area detection information set sequence.
In some embodiments, the executing entity may determine a sequence of sets of coordinates of a midpoint of a short side of the lane marking corresponding to each of the sets of lane marking region detection information.
In some optional implementations of some embodiments, the executing body determining a sequence of coordinates sets of a midpoint of a short side of the lane marking corresponding to each lane marking region detection information in the sequence of sets of lane marking region detection information may include the steps of:
and determining the midpoint coordinates of two short sides of each lane marking area detection frame in the lane marking area detection frame group included in the lane marking area detection information to generate a lane marking midpoint coordinate group, and obtaining a lane marking short side midpoint coordinate group sequence. The coordinates of the midpoint position on the short side of the lane marking area detection frame may be used as the coordinates of the midpoint of the lane marking. In addition, since each lane-marking area detection frame may correspond to the same lane marking as the lane-marking area detection frames of the other respective frames. Thus, each lane-line short-side midpoint coordinate herein may also be the lane-line short-side midpoint coordinate corresponding to the same position on the same lane-line in other frames (i.e., other lane-line short-side midpoint coordinate sets). Finally, the midpoint coordinates of the two short sides can be respectively used as the midpoint coordinates of the lane marking. Thus, each sequence of sets of short-side midpoint coordinates of a lane marking may correspond to a set of lane markings, and each set of short-side midpoint coordinates of a lane marking may correspond to a lane marking.
And 104, constructing a lane marking constraint equation set based on the lane marking region detection information set sequence and the determined lane marking short-side midpoint coordinate set sequence.
In some embodiments, the executing entity may construct a lane-marking constraint equation set based on the lane-marking region detection information set sequence and the determined lane-marking short-side midpoint coordinate set sequence.
In some optional implementations of some embodiments, the executing body constructs a lane-marking constraint equation set based on the lane-marking area detection information set sequence and the determined lane-marking short-edge midpoint coordinate set sequence, and may include the following steps:
and determining global coordinates of a blanking point corresponding to each road image in the road image sequence by using the determined coordinate set sequence of the midpoint of the short side of the lane marking, a preset camera external parameter matrix and a vehicle body conversion matrix sequence to obtain a global coordinate sequence of the blanking point. The global coordinates of the blanking points may be coordinates of the blanking points in a map coordinate system. Here, the global coordinates of the blanking point may have a correspondence relationship with the blanking point in the image coordinate system. The camera extrinsic matrix may be a pose matrix of the body of the current vehicle relative to the camera. Each body transformation matrix in the body transformation matrix sequence may represent a pose matrix of the body relative to a map coordinate system at a time. First, the coordinates of the midpoints of the short sides of the respective lane markings may be converted into a map coordinate system. And then, fitting the midpoint coordinates of the short sides of the lane marking corresponding to the same side in the midpoint coordinate sets of the short sides of the lane marking in the converted midpoint coordinate set sequence of the short sides of the lane marking into a linear equation. Similarly, another linear equation is generated by fitting. Finally, the intersection of two linear equations may be used as the global coordinate of the blanking point. The coordinates of the global coordinates of the blanking point in the image coordinate system can be generated by the following formula:
Where p represents the coordinate. P is p v Representing the global coordinates of the blanking point. P is p t Representing the coordinates of the global coordinates of the blanking points in the image coordinate system. T (T) 1 Representing the camera extrinsic matrix. T (T) 2 And a vehicle body conversion matrix corresponding to the road image in the vehicle body conversion matrix sequence.Representing a preset projection function for projecting the coordinates in the in-bracket camera coordinate system into the image coordinate system.
In practice, the global coordinates of the blanking points and the coordinates of the blanking points in the corresponding image coordinate system are adjustable parameters in the optimization process, so that the adjusted corresponding coordinates can meet the coordinate conversion constraint of the formula.
And secondly, constructing a lane marking parallel constraint equation based on the blanking point global coordinate sequence, the lane marking region detection information group sequence and the determined lane marking short side midpoint coordinate group sequence. For one lane mark, the line (i.e. the central line) of the central point coordinates of the short sides of the two lane marks and the corresponding line of the lane mark in other frames are parallel lines, and the extension lines of the parallel lines should intersect at the same blanking point coordinates in the image coordinate system of the road image. Therefore, the method can be used for constructing a lane marking parallel constraint equation through the global coordinate sequence of the blanking point, the lane marking area detection information group sequence and the determined lane marking short side midpoint coordinate group sequence. Here, the lane-marking parallel constraint equation constructed can be as follows:
Wherein e 1 A first error value representing a sequence of sets of coordinates at the short sides of respective (i.e., each frame) lane markings of a corresponding set of lane markings output by the lane marking parallel constraint equation. N represents the number of sets of midpoint coordinates of the short sides of a lane marking (i.e., the number of lane markings in a set of lane markings) in a sequence of sets of midpoint coordinates of the short sides of a lane marking. n and k represent serial numbers. e, e 1，k And the first error value of the k-th lane marking short-side midpoint coordinate set sequence in the lane marking short-side midpoint coordinate set sequence corresponding to one group of lane markings output by the lane marking parallel constraint equation is represented. P is p n And the coordinates of the middle point of the short side of the lane marking in the coordinate group sequence of the middle point of the short side of the lane marking corresponding to the global coordinates of the blanking point are expressed. P is p t，k The blanking point coordinates in the image coordinate system are represented for the global coordinates of the blanking point corresponding to the kth frame. P is p n，k And represents the lane-line short-side midpoint coordinates in the nth lane-line short-side midpoint coordinate set in one lane-line short-side midpoint coordinate set sequence (i.e., the kth lane-line short-side midpoint coordinate set sequence) corresponding to the kth blanking point global coordinates. P is p n，k，1 Representing the first (i.e., one side) lane-line short-side midpoint coordinate in the nth lane-line short-side midpoint coordinate set in a sequence of lane-line short-side midpoint coordinate sets corresponding to the kth blanking point global coordinate. P is p n，k，2 Representing the second (i.e., other) lane-line short-side midpoint coordinates in the nth lane-line short-side midpoint coordinate set in the sequence of one lane-line short-side midpoint coordinate set corresponding to the kth blanking point global coordinates.
And thirdly, constructing a lane marking equidistant constraint equation based on the lane marking region detection information group sequence and the determined lane marking short side midpoint coordinate group sequence. Wherein, within the same set of lane markings, adjacent lane markings may be equidistant between each other. Thus, for this characteristic of the lane markings, the lane marking equidistant constraint equation can be constructed by the above-described lane marking region detection information set sequence and the determined lane marking shorter side midpoint coordinate set sequence. Here, the lane-marking equidistant constraint equation constructed can be as follows:
wherein e 2 And representing second error values of the coordinate set sequences of the middle points of the short sides of the lane marks corresponding to one group of lane marks output by the lane mark equidistant constraint equation. e, e 2，k And representing a second error value of the k-th lane marking short-side midpoint coordinate set sequence in the lane marking short-side midpoint coordinate set sequence of each lane marking short-side midpoint coordinate set sequence of a corresponding group of lane markings output by the lane marking equidistant constraint equation. i denotes a sequence number (of the midpoint coordinates of the two lane-marking short sides in the lane-marking short sides midpoint coordinate group). m represents an image coordinate system. m p n，k，i And representing the coordinates of the midpoint coordinates of the short sides of the ith lane marking in the midpoint coordinate set of the short sides of the nth lane marking in the sequence of the midpoint coordinate set of the short sides of one lane marking corresponding to the global coordinates of the kth blanking point in the map coordinate system. m p n+1，k， i represents the coordinates of the midpoint coordinates of the ith lane marking short side in the coordinate set of the midpoint of the (n+1) th lane marking short side in the coordinate set sequence of the midpoint of one lane marking short side corresponding to the global coordinates of the kth blanking point in the map coordinate system. m p n+2，k，i And representing the coordinates of the midpoint coordinates of the ith lane marking short side in the coordinate set of the midpoint of the (n+2) th lane marking short side in the coordinate set of the midpoint of one lane marking short side in the coordinate set sequence corresponding to the global coordinates of the kth blanking point in the map coordinate system. A is used to shorten the formula length.
And step four, determining a ground unit normal vector corresponding to each lane marking area detection information group in the lane marking area detection information group sequence to obtain a ground unit normal vector sequence. The ground unit normal vector may be a unit normal vector of the ground near the current vehicle in a vehicle body coordinate system. Thus, each road image (i.e., the ground around the current vehicle for each frame) may correspond to one ground unit normal vector. Here, the preset initial unit normal vector may be used as each ground unit normal vector to obtain a ground unit normal vector sequence. The initial unit normal vector may be a three-dimensional unit vector in a vehicle body coordinate system.
As an example, the initial unit normal vector may be a transpose matrix of [0, 1 ].
And fifthly, determining a negative value of the distance between each ground unit normal vector in the ground unit normal vector sequence and a coordinate origin in a vehicle body coordinate system of the current vehicle, and obtaining a normal vector distance negative value sequence. Wherein, a negative value of a distance value of each ground unit normal vector from the origin of coordinates of the vehicle body coordinate system may be determined as a normal vector distance negative value.
And a sixth step of constructing a lane marking vector constraint equation based on the normal vector distance negative sequence, the vehicle body conversion matrix sequence, the lane marking area detection information group sequence, the determined lane marking short side midpoint coordinate group sequence and the ground unit normal vector sequence. The lane marking vector constraint equation with the lane marking at the same ground plane can be constructed by introducing the distance value between the unit vector of the ground plane at different frames and the origin coordinates of the vehicle body. Here, the constructed lane marking vector constraint equation may be as follows:
wherein e 3 And the third error value of the coordinate set sequence of the middle point of the short side of each lane marking corresponding to one lane marking output by the lane marking vector constraint equation is represented. e, e 3，k And the third error value of the k-th coordinate set sequence of the midpoint coordinate set of the short sides of the lane marking in the coordinate set sequence of the midpoint of the short sides of the lane marking corresponding to the lane marking in the group of the lane marking outputted by the lane marking vector constraint equation is represented. T represents the transpose matrix. f represents the ground unit normal vector in the above ground unit normal vector sequence. f (f) k And represents the kth ground unit normal vector in the ground unit normal vector sequence. d represents the negative value of the normal vector distance in the sequence of negative values of the normal vector distance. d, d k Representing the kth of the above-mentioned normal vector distances in the above-mentioned normal vector distance negative value sequenceNegative values. T (T) 2，k Representing the kth body transition matrix in the body transition matrix sequence.
Seventh, constructing a lane marking blanking point constraint equation based on the normal vector distance negative sequence, the blanking point global coordinate sequence, the vehicle body conversion matrix sequence and the ground unit normal vector sequence. The vanishing points corresponding to the lane markings are constrained to the same ground plane of the lane markings, so that a lane marking vanishing point constraint equation can be constructed based on the normal vector distance negative value sequence, the vanishing point global coordinate sequence, the vehicle body transformation matrix sequence and the ground unit normal vector sequence. Then, the constructed lane marking blanking point constraint equation may be as follows:
Wherein e 4 And a fourth error value representing global coordinates of a corresponding one of the blanking points output by the lane-marking blanking-point constraint equation. e, e 4，k And a fourth error value representing global coordinates of a kth blanking point output by the lane-marking blanking-point constraint equation. P is p v，k Representing the kth global coordinate of the blanking point in the sequence of global coordinates of the blanking point.
And eighth step, determining a coordinate covariance matrix corresponding to the midpoint coordinates of the short sides of each lane marking. When the lane marking coordinates are generated, the detection algorithm can also output a coordinate covariance matrix corresponding to each lane marking coordinate so as to represent the detection error of the lane marking coordinates.
And ninth, constructing a lane marking coordinate re-projection constraint equation based on the determined lane marking short side midpoint coordinate group sequence and a coordinate covariance matrix corresponding to each lane marking short side midpoint coordinate. Wherein, in the presence of the above-mentioned coordinate covariance matrix, the coordinates of the points in each lane marking should satisfy the reprojection error constraint of the transformation from the camera coordinate system to the image coordinate system. Therefore, a lane-marking coordinate re-projection constraint equation can be constructed by the determined lane-marking short-side midpoint coordinate set sequence and the coordinate covariance matrix corresponding to each lane-marking short-side midpoint coordinate. Here, the constructed lane-marking coordinate re-projection constraint equation may be as follows:
Wherein e 5 And the fifth error value of the coordinate set sequence of the middle point of each short side of the lane marking corresponding to one group of lane markings output by the lane marking coordinate reprojection constraint equation is represented. e, e 5，k And the fifth error value of the k-th lane marking short-side midpoint coordinate set sequence in the lane marking short-side midpoint coordinate set sequence corresponding to one group of lane markings output by the lane marking coordinate reprojection constraint equation is represented. i represents a sequence number. P is p n，k，i And the coordinates of the midpoint of the short sides of the ith lane marking in the midpoint coordinate set of the short sides of the nth lane marking in the coordinate set sequence of the midpoint of the short sides of the one lane marking corresponding to the global coordinates of the kth blanking point are expressed. Sigma (sigma) n，k，i And the coordinate covariance matrix of the midpoint coordinates of the short sides of the ith lane marking in the midpoint coordinate set of the short sides of the nth lane marking in the midpoint coordinate set sequence of the short sides of one lane marking corresponding to the global coordinates of the kth blanking point is represented.
And tenth, determining the lane marking parallel constraint equation, the lane marking equidistant constraint equation, the lane marking vector constraint equation, the lane marking blanking point constraint equation and the lane marking coordinate reprojection constraint equation as lane marking constraint equations respectively to obtain a lane marking constraint equation set.
Step 105, based on the lane marking constraint equation set, performing optimization processing on each current vehicle pose matrix in the current vehicle pose matrix sequence to generate an optimized vehicle relative pose matrix sequence.
In some embodiments, the executing entity may perform optimization processing on each current vehicle pose matrix in the current vehicle pose matrix sequence based on the lane-marking constraint equation set to generate an optimized vehicle relative pose matrix sequence.
In some optional implementations of some embodiments, the executing body performs optimization processing on each current vehicle pose matrix in the current vehicle pose matrix sequence based on the lane marking constraint equation set to generate an optimized vehicle relative pose matrix sequence, and may include the following steps:
and taking the current vehicle pose matrix sequence, the determined lane marking short side midpoint coordinate set sequence, the ground unit normal vector sequence and the normal vector distance negative value sequence as optimization targets, and performing optimization processing on each current vehicle pose matrix in the current vehicle pose matrix sequence by utilizing the lane marking constraint equation set so as to generate an optimized vehicle relative pose matrix sequence. Wherein, the optimization process can be performed by the following formula:
Wherein,and representing the k-th optimized vehicle relative pose matrix in the optimized vehicle relative pose matrix sequence. M represents a sequence number. λ represents a preset error coefficient (e.g. 1). Lambda (lambda) M Representing the mth preset error coefficient. e, e M，k And representing the error value of the coordinate set sequence of the midpoint of the short side of the kth lane marking in the coordinate set sequence of the midpoint of the short side of each lane marking corresponding to one group of lane markings output by the M-th lane marking error constraint equation in the lane marking error constraint equation set.
In addition, the coordinates of the midpoint coordinates of the first lane marking short side in the midpoint coordinate set of the nth lane marking short side in the map coordinate system in the coordinate set sequence of the midpoint coordinate set of the nth lane marking short side in the optimized corresponding k blanking point global coordinates can be synchronized. And synchronizing the coordinates of the midpoint coordinates of the short sides of the second lane marking in the midpoint coordinate set of the short sides of the nth lane marking in the coordinate set sequence of the midpoint coordinates of the short sides of the one lane marking corresponding to the global coordinates of the kth blanking point in the map coordinate system. And synchronously optimizing the kth ground unit normal vector in the ground unit normal vector sequence. And synchronously optimizing the k th normal vector distance negative value in the normal vector distance negative value sequence. In practice, the optimization for the vehicle relative pose matrix may be to optimize only pitch and roll angles.
The above formulas and their related matters serve as an invention point of the embodiments of the present disclosure, and may further solve the technical problem "because the ground is not in an ideal planar state, if a road jolt occurs in an actual driving process of a vehicle, a relative angle between the vehicle and the ground is offset (such as a roll angle and a pitch angle offset), so that optimization of a relative pose matrix of the vehicle is difficult, and thus, accuracy of the relative pose matrix of the vehicle generated by a projective transformation manner is reduced. First, it is considered that if a road jolt occurs during the actual running of the vehicle due to the fact that the ground is not in an ideal planar state, the relative angle between the vehicle and the ground is offset. Thus, a structured lane marking is introduced for vehicle relative pose matrix optimization. Here, it is considered that, for one lane-line, the line (i.e., the center line) of the coordinates of the middle point of the short sides of two lane-lines thereof and the line of the lane-line corresponding thereto in each other frame are parallel lines, and the extension lines of the respective parallel lines should intersect at the same blanking point coordinates in the image coordinate system of the road image. Therefore, the global coordinates of the blanking points corresponding to each road image can be used for constructing a lane marking parallel constraint equation. The characteristic of parallel constraint of the lane markings is introduced into the optimization process of the relative pose matrix of the vehicle. Secondly, considering the characteristic that the adjacent lane markings can be equidistant in the same group of lane markings, the lane marking equidistant constraint equation is constructed. Therefore, the equidistant characteristics of the lane markings can be introduced into the optimization process of the relative pose matrix of the vehicle. Thereafter, considering that the ground unit normal vector may be a characteristic of a unit normal vector of the ground in the vicinity of the current vehicle in the vehicle body coordinate system, it is used to construct a lane marking vector constraint equation. The ground normal vector corresponding to each road image of the continuous frame and the position of the ground normal vector relative to the origin coordinate can be used for restraining each lane marking coordinate on the ground. So as to optimize the relative pose matrix of the vehicle by combining the characteristics of the lane markings. And then, considering that the shadow eliminating points corresponding to the lane markings are constrained to the same ground plane of the lane markings, and constructing a lane marking blanking point constraint equation based on the normal vector distance negative value sequence, the blanking point global coordinate sequence, the vehicle body conversion matrix sequence and the ground unit normal vector sequence. Thus, the characteristics of the lane markings can be further introduced into the optimization process of the relative pose matrix of the vehicle. And then, taking the reprojection error constraint of the coordinates of the points of the lane markings into consideration, and constructing a lane marking coordinate reprojection constraint equation. Therefore, the influence caused by the coordinate detection error is reduced in the optimization process of the relative pose matrix of the vehicle. Finally, the optimization processing formula is combined with each constraint equation, so that the relative pose matrix of the vehicle can be optimized. Therefore, the accuracy of generating the optimized vehicle relative pose matrix can be improved.
Optionally, the executing body may further execute the following steps:
first, detecting the road image at the current moment to obtain detection information. The detection information may include an obstacle detection coordinate set and a road information detection coordinate set. The road image at the current moment can be detected through a preset image detection algorithm, so that detection information can be obtained. Each obstacle detection coordinate set may correspond to an obstacle. Each road information detection coordinate set may correspond to one road information.
As an example, the obstacle may be a vehicle pedestrian or the like. The road information may be lane lines, lane steering identifications, etc. The image detection algorithm may include, but is not limited to, at least one of: a refinnet (Multi-Path Refinement Networks for High-Resolution Semantic Segmentation, multi-path refinement network for high resolution semantic segmentation) algorithm, a SegNet (A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation, a deep convolutional encoding-decoding structure model for image segmentation) algorithm, and the like.
And a second step of converting the obstacle detection coordinates in the obstacle detection coordinate set and the road information detection coordinates in the road information detection coordinate set into a vehicle body coordinate system at the current moment by using the optimized vehicle relative pose matrix corresponding to the current moment in the optimized vehicle relative pose matrix sequence, so as to obtain a converted obstacle coordinate set and a converted road information coordinate set. The method comprises the steps of converting obstacle detection coordinates and road information detection coordinates from an image coordinate system to a vehicle body coordinate system at the current moment by utilizing an optimized vehicle relative pose matrix corresponding to the current moment in the optimized vehicle relative pose matrix sequence in a coordinate conversion mode to obtain a converted obstacle coordinate set and a converted road information coordinate set.
In practice, the method utilizes the relative pose matrix of the optimized vehicle to carry out coordinate conversion, and can be used for further improving the accuracy of the converted obstacle coordinates and the converted road information coordinates.
And thirdly, transmitting the vehicle body coordinate system, the converted obstacle coordinate set and the converted road information coordinate set to a target terminal for display, and storing the optimized vehicle relative pose matrix corresponding to the current moment.
The above embodiments of the present disclosure have the following advantageous effects: by the vehicle relative pose matrix optimization method, the accuracy of the optimized vehicle relative pose matrix can be improved. Specifically, the reduced accuracy of the generated vehicle relative pose matrix is caused by: because the ground is not in an ideal plane state, if the vehicle bumps in the actual running process, the relative angle between the vehicle and the ground is offset (such as roll angle and pitch angle offset), so that the optimization of the relative pose matrix of the vehicle is difficult, and the accuracy of the relative pose matrix of the vehicle generated by a projective transformation mode is reduced. Based on this, in some embodiments of the present disclosure, a vehicle relative pose matrix optimization method first obtains a road image sequence within a preset sliding window and a current vehicle pose matrix sequence corresponding to each road image. By introducing road images of successive frames in the sliding window, extraction of successive features can be facilitated. Meanwhile, the current vehicle pose matrix corresponding to the road image is introduced, so that optimization can be conveniently carried out. Then, the lane marking detection is performed on each road image in the road image sequence to generate a lane marking region detection information set, and a lane marking region detection information set sequence is obtained. Through the lane marking detection, lane marking area detection information for optimizing the current vehicle pose matrix can be obtained. And then, determining a center point coordinate set sequence of the short sides of the lane marking corresponding to each lane marking area detection information in the lane marking area detection information set sequence. The current vehicle pose matrix can be conveniently optimized by utilizing the characteristics of the lane mark by generating the coordinate set sequence of the midpoint of the short side of the lane mark. And then, constructing a lane marking constraint equation set based on the lane marking region detection information set sequence and the determined lane marking short side midpoint coordinate set sequence. By constructing a system of lane-marking constraint equations, features of lane markings can be introduced into the constraint equations for feature constraint in the optimization process. Thereby, it is convenient to correct the position posture of the vehicle with respect to the ground in the bumpy state. And finally, based on the lane marking constraint equation set, carrying out optimization processing on each current vehicle pose matrix in the current vehicle pose matrix sequence so as to generate an optimized vehicle relative pose matrix sequence. The optimization processing can be used for optimizing the relative position and the posture of the vehicle body relative to the ground in a bumpy state. Therefore, the optimization of the relative pose matrix of the vehicle is realized. Further, the accuracy of the generated vehicle relative pose matrix can be improved.
With further reference to fig. 2, as an implementation of the method shown in the above figures, the present disclosure provides embodiments of a vehicle relative pose matrix optimization apparatus, corresponding to those method embodiments shown in fig. 1, which may be particularly applicable in various electronic devices.
As shown in fig. 2, the vehicle relative pose matrix optimization device 200 of some embodiments includes: an acquisition unit 201, a lane marking detection unit 202, a determination unit 203, a construction unit 204, and an optimization processing unit 205. Wherein, the acquiring unit 201 is configured to acquire a road image sequence in a preset sliding window and a current vehicle pose matrix sequence corresponding to each road image; a lane marking detection unit 202 configured to perform lane marking detection on each road image in the above-described road image sequence to generate a lane marking region detection information group, resulting in a lane marking region detection information group sequence; a determining unit 203 configured to determine a lane-marking short-side midpoint coordinate group sequence corresponding to each of the lane-marking region detection information in the above-described lane-marking region detection information group sequence; a construction unit 204 configured to construct a lane marking constraint equation set based on the above-described lane marking region detection information set sequence and the determined lane marking short side midpoint coordinate set sequence; the optimization processing unit 205 is configured to perform optimization processing on each current vehicle pose matrix in the current vehicle pose matrix sequence based on the lane marking constraint equation set, so as to generate an optimized vehicle relative pose matrix sequence.
It will be appreciated that the elements described in the apparatus 200 correspond to the various steps in the method described with reference to fig. 1. Thus, the operations, features and resulting benefits described above for the method are equally applicable to the apparatus 200 and the units contained therein, and are not described in detail herein.
Referring now to fig. 3, a schematic diagram of an electronic device 300 suitable for use in implementing some embodiments of the present disclosure is shown. The electronic device shown in fig. 3 is merely an example and should not impose any limitations on the functionality and scope of use of embodiments of the present disclosure.
As shown in fig. 3, the electronic device 300 may include a processing means 301 (e.g., a central processing unit, a graphics processor, etc.) that may perform various suitable actions and processes in accordance with a program stored in a Read Only Memory (ROM) 302 or a program loaded from a storage means 308 into a Random Access Memory (RAM) 303. In the RAM 303, various programs and data required for the operation of the electronic apparatus 300 are also stored. The processing device 301, the ROM 302, and the RAM 303 are connected to each other via a bus 304. An input/output (I/O) interface 305 is also connected to bus 304.
In general, the following devices may be connected to the I/O interface 305: input devices 306 including, for example, a touch screen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; an output device 307 including, for example, a Liquid Crystal Display (LCD), a speaker, a vibrator, and the like; storage 308 including, for example, magnetic tape, hard disk, etc.; and communication means 309. The communication means 309 may allow the electronic device 300 to communicate with other devices wirelessly or by wire to exchange data. While fig. 3 shows an electronic device 300 having various means, it is to be understood that not all of the illustrated means are required to be implemented or provided. More or fewer devices may be implemented or provided instead. Each block shown in fig. 3 may represent one device or a plurality of devices as needed.
In particular, according to some embodiments of the present disclosure, the processes described above with reference to flowcharts may be implemented as computer software programs. For example, some embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method shown in the flow chart. In such embodiments, the computer program may be downloaded and installed from a network via communications device 309, or from storage device 308, or from ROM 302. The above-described functions defined in the methods of some embodiments of the present disclosure are performed when the computer program is executed by the processing means 301.
It should be noted that, in some embodiments of the present disclosure, the computer readable medium may be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In some embodiments of the present disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In some embodiments of the present disclosure, however, the computer-readable signal medium may comprise a data signal propagated in baseband or as part of a carrier wave, with the computer-readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, fiber optic cables, RF (radio frequency), and the like, or any suitable combination of the foregoing.
In some implementations, the clients, servers may communicate using any currently known or future developed network protocol, such as HTTP (Hyper Text Transfer Protocol ), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), the internet (e.g., the internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future developed networks.
The computer readable medium may be embodied in the apparatus; or may exist alone without being incorporated into the electronic device. The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to: acquiring a road image sequence in a preset sliding window and a current vehicle pose matrix sequence corresponding to each road image; carrying out lane marking detection on each road image in the road image sequence to generate a lane marking area detection information group, and obtaining a lane marking area detection information group sequence; determining a center point coordinate group sequence of a short side of the lane marking corresponding to each lane marking region detection information in the lane marking region detection information group sequence; constructing a lane marking constraint equation set based on the lane marking region detection information set sequence and the determined lane marking short-side midpoint coordinate set sequence; and carrying out optimization processing on each current vehicle pose matrix in the current vehicle pose matrix sequence based on the lane marking constraint equation set so as to generate an optimized vehicle relative pose matrix sequence.
Computer program code for carrying out operations for some embodiments of the present disclosure may be written in one or more programming languages, including an object oriented programming language such as Java, smalltalk, C ++ and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (for example, through the Internet using an Internet service provider).
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units described in some embodiments of the present disclosure may be implemented by means of software, or may be implemented by means of hardware. The described units may also be provided in a processor, for example, described as: a processor includes an acquisition unit, a lane marking detection unit, a determination unit, a construction unit, and an optimization processing unit. The names of these units do not constitute a limitation on the unit itself in some cases, and for example, the optimization processing unit may also be described as "a unit that performs optimization processing on each current vehicle pose matrix in the sequence of current vehicle pose matrices".
The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: a Field Programmable Gate Array (FPGA), an Application Specific Integrated Circuit (ASIC), an Application Specific Standard Product (ASSP), a system on a chip (SOC), a Complex Programmable Logic Device (CPLD), and the like.
The foregoing description is only of the preferred embodiments of the present disclosure and description of the principles of the technology being employed. It will be appreciated by those skilled in the art that the scope of the invention in the embodiments of the present disclosure is not limited to the specific combination of the above technical features, but encompasses other technical features formed by any combination of the above technical features or their equivalents without departing from the spirit of the invention. Such as the above-described features, are mutually substituted with (but not limited to) the features having similar functions disclosed in the embodiments of the present disclosure.

Claims (9)

1. A vehicle relative pose matrix optimization method, comprising:
acquiring a road image sequence in a preset sliding window and a current vehicle pose matrix sequence corresponding to each road image;
carrying out lane marking detection on each road image in the road image sequence to generate a lane marking area detection information group, and obtaining a lane marking area detection information group sequence;
determining a center point coordinate group sequence of a short side of the lane marking corresponding to each lane marking region detection information in the lane marking region detection information group sequence;
constructing a lane marking constraint equation set based on the lane marking region detection information set sequence and the determined lane marking short-side midpoint coordinate set sequence;
and carrying out optimization processing on each current vehicle pose matrix in the current vehicle pose matrix sequence based on the lane marking constraint equation set so as to generate an optimized vehicle relative pose matrix sequence.
2. The method of claim 1, wherein the method further comprises:
detecting a road image at the current moment to obtain detection information, wherein the detection information comprises an obstacle detection coordinate set and a road information detection coordinate set;
Converting obstacle detection coordinates in the obstacle detection coordinate set and road information detection coordinates in the road information detection coordinate set to a vehicle body coordinate system at the current moment by using an optimized vehicle relative pose matrix corresponding to the current moment in the optimized vehicle relative pose matrix sequence, so as to obtain a converted obstacle coordinate set and a converted road information coordinate set;
and sending the vehicle body coordinate system, the converted obstacle coordinate set and the converted road information coordinate set to a target terminal for display, and storing an optimized vehicle relative pose matrix corresponding to the current moment.
3. The method of claim 1, wherein the lane marking detection of each road image in the sequence of road images to generate a lane marking region detection information set comprises:
performing region detection on the lane markings in the road image to obtain a lane marking region detection frame group and a lane marking coordinate set corresponding to each lane marking region detection frame;
and determining each lane marking area detection frame and a corresponding lane marking coordinate set in the lane marking area detection frame group as lane marking area detection information to obtain a lane marking area detection information group.
4. The method of claim 3, wherein the determining a sequence of sets of coordinates of a midpoint of a short side of the lane marking corresponding to each lane marking region detection information in the sequence of sets of lane marking region detection information comprises:
and determining the midpoint coordinates of two short sides of each lane marking area detection frame in the lane marking area detection frame group included in the lane marking area detection information to generate a lane marking midpoint coordinate group, and obtaining a lane marking short side midpoint coordinate group sequence.
5. The method of claim 1, wherein the constructing a set of lane-marking constraint equations based on the set of lane-marking region detection information sequences and the determined set of lane-marking short-side midpoint coordinates comprises:
determining global coordinates of a blanking point corresponding to each road image in the road image sequence by using the determined coordinate set sequence of the middle point of the short side of the lane marking, a preset camera external parameter matrix and a vehicle body conversion matrix sequence, so as to obtain a global coordinate sequence of the blanking point;
constructing a lane marking parallel constraint equation based on the blanking point global coordinate sequence, the lane marking region detection information group sequence and the determined lane marking short side midpoint coordinate group sequence;
Constructing a lane marking equidistant constraint equation based on the lane marking region detection information group sequence and the determined lane marking short-side midpoint coordinate group sequence;
determining a ground unit normal vector corresponding to each lane marking area detection information group in the lane marking area detection information group sequence to obtain a ground unit normal vector sequence;
determining a negative value of the distance between each ground unit normal vector in the ground unit normal vector sequence and a coordinate origin in a vehicle body coordinate system of the current vehicle to obtain a normal vector distance negative value sequence;
constructing a lane marking vector constraint equation based on the normal vector distance negative sequence, the vehicle body conversion matrix sequence, the lane marking area detection information group sequence, the determined lane marking short side midpoint coordinate group sequence and the ground unit normal vector sequence;
constructing a lane marking blanking point constraint equation based on the normal vector distance negative sequence, the blanking point global coordinate sequence, the vehicle body conversion matrix sequence and the ground unit normal vector sequence;
determining a coordinate covariance matrix corresponding to the midpoint coordinates of the short sides of each lane marking;
constructing a lane marking coordinate re-projection constraint equation based on the determined lane marking short-side midpoint coordinate set sequence and a coordinate covariance matrix corresponding to each lane marking short-side midpoint coordinate;
And respectively determining the lane marking parallel constraint equation, the lane marking equidistant constraint equation, the lane marking vector constraint equation, the lane marking blanking point constraint equation and the lane marking coordinate reprojection constraint equation as lane marking constraint equations to obtain a lane marking constraint equation set.
6. The method of claim 5, wherein the optimizing each current vehicle pose matrix in the sequence of current vehicle pose matrices based on the set of lane-marking constraint equations to generate a sequence of optimized vehicle relative pose matrices comprises:
and taking the current vehicle pose matrix sequence, the determined center point coordinate set sequence of the short sides of the lane marks, the ground unit normal vector sequence and the normal vector distance negative value sequence as optimization targets, and performing optimization processing on each current vehicle pose matrix in the current vehicle pose matrix sequence by utilizing the lane mark constraint equation set so as to generate an optimized vehicle relative pose matrix sequence.
7. A vehicle relative pose matrix optimization device, comprising:
the acquisition unit is configured to acquire a road image sequence in a preset sliding window and a current vehicle pose matrix sequence corresponding to each road image;
A lane marking detection unit configured to perform lane marking detection on each road image in the road image sequence to generate a lane marking region detection information set, and obtain a lane marking region detection information set sequence;
a determining unit configured to determine a lane-marking short-side midpoint coordinate group sequence corresponding to each lane-marking region detection information in the lane-marking region detection information group sequence;
a construction unit configured to construct a lane-marking constraint equation set based on the lane-marking region detection information set sequence and the determined lane-marking short-side midpoint coordinate set sequence;
and the optimization processing unit is configured to perform optimization processing on each current vehicle pose matrix in the current vehicle pose matrix sequence based on the lane marking constraint equation set so as to generate an optimized vehicle relative pose matrix sequence.
8. An electronic device, comprising:
one or more processors;
a storage device having one or more programs stored thereon,
when executed by the one or more processors, causes the one or more processors to implement the method of any of claims 1-6.
9. A computer readable medium having stored thereon a computer program, wherein the computer program, when executed by a processor, implements the method of any of claims 1-6.
CN202310820813.4A 2023-07-05 2023-07-05 Vehicle relative pose matrix optimization method and device, electronic equipment and readable medium Pending CN117095052A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310820813.4A CN117095052A (en) 2023-07-05 2023-07-05 Vehicle relative pose matrix optimization method and device, electronic equipment and readable medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310820813.4A CN117095052A (en) 2023-07-05 2023-07-05 Vehicle relative pose matrix optimization method and device, electronic equipment and readable medium

Publications (1)

Publication Number Publication Date
CN117095052A true CN117095052A (en) 2023-11-21

Family

ID=88770408

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310820813.4A Pending CN117095052A (en) 2023-07-05 2023-07-05 Vehicle relative pose matrix optimization method and device, electronic equipment and readable medium

Country Status (1)

Country Link
CN (1) CN117095052A (en)

Similar Documents

Publication Publication Date Title
CN112598762B (en) Three-dimensional lane line information generation method, device, electronic device, and medium
CN111950555B (en) Text recognition method and device, readable medium and electronic equipment
CN114894205B (en) Three-dimensional lane line information generation method, device, equipment and computer readable medium
CN116182878B (en) Road curved surface information generation method, device, equipment and computer readable medium
CN114399588B (en) Three-dimensional lane line generation method and device, electronic device and computer readable medium
CN114993328B (en) Vehicle positioning evaluation method, device, equipment and computer readable medium
CN116164770B (en) Path planning method, path planning device, electronic equipment and computer readable medium
CN116758498B (en) Obstacle information generation method, obstacle information generation device, electronic device, and computer-readable medium
CN116740382B (en) Obstacle information generation method, obstacle information generation device, electronic device, and computer-readable medium
CN117218187A (en) Pedestrian position information generation method, device, equipment and computer readable medium
CN116161040B (en) Parking space information generation method, device, electronic equipment and computer readable medium
CN115937046B (en) Road ground information generation method, device, equipment and computer readable medium
CN115565158B (en) Parking space detection method, device, electronic equipment and computer readable medium
CN116311155A (en) Obstacle information generation method, obstacle information generation device, electronic device, and computer-readable medium
CN112880675B (en) Pose smoothing method and device for visual positioning, terminal and mobile robot
CN117095052A (en) Vehicle relative pose matrix optimization method and device, electronic equipment and readable medium
CN115393826A (en) Three-dimensional lane line generation method and device, electronic device and computer readable medium
CN111383337B (en) Method and device for identifying objects
CN114119973A (en) Spatial distance prediction method and system based on image semantic segmentation network
CN116563817B (en) Obstacle information generation method, obstacle information generation device, electronic device, and computer-readable medium
CN116653931B (en) Vehicle parking control method, device, electronic equipment and computer readable medium
CN116630436B (en) Camera external parameter correction method, camera external parameter correction device, electronic equipment and computer readable medium
US11521331B2 (en) Method and apparatus for generating position information, device, and medium
CN111325712B (en) Method and device for detecting image validity
CN114863025B (en) Three-dimensional lane line generation method and device, electronic device and computer readable medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination