WO2021190167A1 - 一种位姿确定方法、装置、介质和设备 - Google Patents

一种位姿确定方法、装置、介质和设备 Download PDF

Info

Publication number
WO2021190167A1
WO2021190167A1 PCT/CN2021/075016 CN2021075016W WO2021190167A1 WO 2021190167 A1 WO2021190167 A1 WO 2021190167A1 CN 2021075016 W CN2021075016 W CN 2021075016W WO 2021190167 A1 WO2021190167 A1 WO 2021190167A1
Authority
WO
WIPO (PCT)
Prior art keywords
semantic
target
pose
semantic target
movable device
Prior art date
Application number
PCT/CN2021/075016
Other languages
English (en)
French (fr)
Chinese (zh)
Inventor
唐庆
刘余钱
陆潇
Original Assignee
上海商汤临港智能科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 上海商汤临港智能科技有限公司 filed Critical 上海商汤临港智能科技有限公司
Priority to JP2022504292A priority Critical patent/JP2022542082A/ja
Publication of WO2021190167A1 publication Critical patent/WO2021190167A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/08—Control of attitude, i.e. control of roll, pitch, or yaw
    • G05D1/0891—Control of attitude, i.e. control of roll, pitch, or yaw specially adapted for land vehicles
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C11/00—Photogrammetry or videogrammetry, e.g. stereogrammetry; Photographic surveying
    • G01C11/04—Interpretation of pictures
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/20—Instruments for performing navigational calculations
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02—Control of position or course in two dimensions
    • G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0231—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means
    • G05D1/0246—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using a video camera in combination with image processing means

Definitions

  • the present disclosure relates to the field of computer vision technology, and in particular to a method, device, medium, and equipment for determining a pose.
  • Mobile device positioning technology is one of the key technologies of autonomous driving systems.
  • the cost of vision cameras is low, and large wide-angle, high-pixel cameras also provide large-scale, high-precision observation data. Therefore, mobile device positioning technologies that combine vision cameras and high-precision maps are increasingly favored by the autonomous driving industry.
  • the traditional positioning technology has low positioning accuracy.
  • the present disclosure provides a pose determination method, device, medium, device, and computer program.
  • a pose determination method includes: obtaining a first semantic target in a first image of an area where a movable device is located, and obtaining a second semantic target in a semantic map,
  • the first semantic target belongs to at least two categories, and the first semantic targets of at least two of the at least two categories are located in different spatial dimensions; and it is determined that the second semantic target is related to each of the first semantic targets.
  • a matching semantic target that matches the target; the pose of the movable device is determined according to the matching semantic target.
  • the determining a matching semantic target that matches each of the first semantic targets among the second semantic targets includes: generating a second image according to the pixel value of each of the first semantic targets; The second image determines matching semantic targets among the second semantic targets that match each of the first semantic targets.
  • the acquiring the second semantic target in the semantic map includes: acquiring the pose estimation value of the movable device, and determining the target for searching the semantic map according to the pose estimation value Search range; search the second semantic target from the target search range.
  • the obtaining the pose estimation value of the movable device includes: obtaining the first pose of the movable device at the first moment; and determining the movable device according to the first pose The pose estimation value of the device at the second moment, where the first moment is before the second moment.
  • the pose estimation value includes an estimation value of a position and an estimation value of an orientation;
  • the target search range is an area in the orientation and within a preset distance range of the position.
  • the determining the matching semantic target in the second semantic target that matches each of the first semantic targets includes: for each of the first semantic targets, the second semantic target is The second semantic target with the same category as the first semantic target and the shortest distance is determined as the matching semantic target of the first semantic target.
  • the method further includes: before determining a second semantic target with the same category as the first semantic target and the shortest distance among the second semantic targets as the matching semantic target of the first semantic target, For each of the second semantic targets, obtain the projected semantic target of the second semantic target in the second image; the position of the second semantic target in the semantic map and the projected semantic target in the second image The second image is generated based on the pixel value of each of the first semantic targets; the distance between the projected semantic target and the first semantic target is determined as the second semantic target and the first semantic target. A distance between semantic objects.
  • the determining the second semantic target with the same category as the first semantic target and the shortest distance among the second semantic targets as the matching semantic target of the first semantic target includes: determining the first semantic target Among the two semantic targets, a second semantic target with the same category, the shortest distance and the same shape as the first semantic target is determined as the matching semantic target of the first semantic target.
  • the shape of the second semantic target is determined based on the contour information of the projection semantic target of the second semantic target in the second image.
  • the method further includes: determining the first position of each first target point in the at least one first target point of the first semantic target; the first target point is based on the first semantic target Determining the second position of each second target point in at least one second target point of the projected semantic target; the second target point is determined based on the contour information of the projected semantic target; According to the first position of each first target point and the second position of each second target point, the distance between the projected semantic target and the first semantic target is determined.
  • the determining the distance between the projected semantic target and the first semantic target according to the first position of each first target point and the second position of each second target point includes: The first position of each first target point and the second position of each second target point, the distance between each first target point and the corresponding second target point is determined; The average value of the distance between the first target point and the corresponding second target point is determined as the distance between the first semantic target and the projected semantic target.
  • the first target point is the vertex of the bounding box of the first semantic target; and/or the contour of the first semantic target is In the case of a long strip, the first target point is the vertex of the line segment corresponding to the first semantic target.
  • the determining the pose of the movable device according to the matching semantic target includes: establishing a pose constraint condition according to the matching semantic target; determining the movable device according to the pose constraint condition The pose of the device.
  • the pose constraint condition is determined according to the pose change relationship of the movable device within a preset time period.
  • the establishing the pose constraint condition according to the matching semantic target includes: determining the plane where the movable device is located and the normal vector of the plane according to the position of the matching semantic target; At least one of the plane, the normal vector, and the distance between the matching semantic target and the first semantic target determines the pose distribution error of the movable device; and establishes the pose distribution error according to the pose distribution error Pose constraints.
  • the pose distribution error of the movable device is determined according to at least one of the plane, the normal vector, and the distance between the matching semantic target and the first semantic target , Including: determining the height distribution error of the target point of the movable device according to the plane; determining the pitch angle distribution error and the roll angle distribution error of the movable device according to the normal vector; according to the matching semantic target and The distance between the first semantic targets determines the reprojection distance error; the establishing the pose constraint condition according to the pose distribution error includes: according to the height distribution error, the pitch angle distribution error, and the At least one of the roll angle distribution error and the reprojection distance error establishes the pose constraint condition.
  • the pose constraint condition is that the sum of the height distribution error, the pitch angle distribution error, the roll angle distribution error, and the reprojection distance error is minimized.
  • the determining the pose of the movable device according to the matching semantic target includes: determining the three-dimensional pose of the movable device according to the matching semantic target; the method further includes: The three-dimensional pose controls the driving state of the movable device.
  • the embodiments of the present disclosure by acquiring first semantic targets of multiple categories in the first image of the area where the movable device is located, and determining the matching semantic target that matches the first semantic target from the second semantic target in the semantic map, Then determine the pose of the movable device according to the matching semantic target.
  • the embodiments of the present disclosure comprehensively utilize multiple categories of semantic information. Compared with the situation in some areas where the number of semantic targets of a single category is small and the pose cannot be determined based on the semantic targets of a single category, the technical solution of the present disclosure is adopted to reduce The impact on the accuracy of the pose determination process due to the close proximity of semantic targets of a single category in multiple regions improves the accuracy and robustness of pose determination.
  • a pose determination method includes: acquiring a first semantic target in a first image of an area where a movable device is located, and acquiring a second semantic target in a semantic map Determine the matching semantic target of the second semantic target that matches the first semantic target; establish a pose constraint condition according to the matching semantic target, and determine the mobile device's position and pose constraint according to the pose constraint condition Three-dimensional pose.
  • the pose constraint condition is determined according to the pose change relationship of the movable device within a preset time period.
  • the establishing the pose constraint condition according to the matching semantic target includes: determining the plane where the movable device is located and the normal vector of the plane according to the position of the matching semantic target; At least one of the plane, the normal vector, and the distance between the matching semantic target and the first semantic target determines the pose distribution error of the movable device; and establishes the pose distribution error according to the pose distribution error Pose constraints.
  • the pose distribution error of the movable device is determined according to at least one of the plane, the normal vector, and the distance between the matching semantic target and the first semantic target , Including: determining the height distribution error of the target point of the movable device according to the plane; determining the pitch angle distribution error and the roll angle distribution error of the movable device according to the normal vector; according to the matching semantic target and The distance between the first semantic targets determines the reprojection distance error; the establishing the pose constraint condition according to the pose distribution error includes: according to the height distribution error, the pitch angle distribution error, and the At least one of the roll angle distribution error and the reprojection distance error establishes the pose constraint condition.
  • the pose constraint condition is that the sum of the height distribution error, the pitch angle distribution error, the roll angle distribution error, and the reprojection distance error is minimized.
  • the second semantic target matches the first semantic target.
  • Match the semantic target establish the pose constraint condition according to the matched semantic target, and determine the three-dimensional pose of the movable device according to the pose constraint condition. Since constraint conditions are introduced in the pose determination process, therefore, In the process of solving the pose, only when the solved pose satisfies the constraints, can the solved pose be regarded as the pose of the movable device, which realizes the determination of the movable device through the constraint conditions.
  • the three-dimensional pose improves the accuracy of pose determination.
  • a pose determination device the device includes: a first acquisition module for acquiring a first semantic target in a first image of a region where a movable device is located, and acquiring semantics A second semantic target in the map, where the first semantic target belongs to at least two categories, and the first semantic targets of at least two of the at least two categories are located in different spatial dimensions; the first determining module is configured to A matching semantic target that matches each of the first semantic targets among the second semantic targets is determined; a second determination module is configured to determine the pose of the movable device according to the matching semantic target.
  • a pose determination device the device includes: a third acquisition module for acquiring the first semantic target in the first image of the area where the movable device is located, and acquiring the semantics The second semantic target in the map; the seventh determining module is used to determine the matching semantic target in the second semantic target that matches the first semantic target; the eighth determining module is used to determine the matching semantic target according to the matching semantic target A pose constraint condition is established, and the three-dimensional pose of the movable device is determined according to the pose constraint condition.
  • a computer-readable storage medium having a computer program stored thereon, and when the program is executed by a processor, the method described in any of the embodiments is implemented.
  • a computer device including a memory, a processor, and a computer program stored in the memory and capable of running on the processor.
  • the processor implements any implementation when the program is executed. The method described in the example.
  • a computer program is provided, the computer program is stored on a storage medium, and when the computer program is executed by a processor, the method described in any of the embodiments is implemented.
  • FIG. 1 is a flowchart of an implementation manner of a pose determination method according to some embodiments of the present disclosure.
  • Figure 2 is a schematic diagram of semantic targets of some embodiments of the present disclosure.
  • Fig. 3a is a schematic diagram of a target search range of some embodiments of the present disclosure.
  • Fig. 3b is a schematic diagram of the target search range of some embodiments of the present disclosure.
  • Figure 4a is a schematic diagram of a semantic matching process in some embodiments of the present disclosure.
  • Fig. 4b is a schematic diagram of the semantic matching process of other embodiments of the present disclosure.
  • Fig. 5 is a schematic diagram of semantic matching results of some embodiments of the present disclosure.
  • Fig. 6 is a schematic diagram of a pose estimation effect of some embodiments of the present disclosure.
  • Fig. 7 is a schematic diagram of the pose determination principle of some embodiments of the present disclosure.
  • FIG. 8 is a flowchart of another implementation manner of the pose determination method according to some embodiments of the present disclosure.
  • FIG. 9 is a block diagram of an implementation manner of a pose determination device according to some embodiments of the present disclosure.
  • FIG. 10 is a block diagram of another implementation manner of the pose determination apparatus according to some embodiments of the present disclosure.
  • FIG. 11 is a schematic diagram of the structure of a computer device according to some embodiments of the present disclosure.
  • first, second, third, etc. may be used in this disclosure to describe various information, the information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other.
  • first information may also be referred to as second information, and similarly, the second information may also be referred to as first information.
  • word “if” as used herein can be interpreted as "when” or “when” or "in response to determination”.
  • the method includes step S101 to step S103.
  • Step S101 Obtain a first semantic target in a first image of an area where a movable device is located, and acquire a second semantic target in a semantic map, where the first semantic target belongs to at least two categories, and the at least two categories
  • the first semantic targets of at least two categories of are located in different spatial dimensions
  • Step S102 Determine matching semantic targets among the second semantic targets that match each of the first semantic targets
  • Step S103 Determine the pose of the movable device according to the matching semantic target.
  • the movable device may be equipped with a vision camera to collect the first image of the area, and may also be equipped with a positioning device to obtain rough or accurate position information.
  • Movable equipment may include, but is not limited to, vehicles, mobile robots and other equipment.
  • the vehicle may be any type of vehicle, such as a car, bus, truck, etc., accordingly, the area where the movable equipment is located may be a road area.
  • the first image is a road image;
  • the mobile robot can be any type of robot, such as an industrial robot, a sweeping robot, a toy robot, an educational robot, etc., accordingly, the area where the mobile device is located can be all
  • the working area of the mobile robot, the first image is a working area image.
  • the movable device and the area where it is located may also be other types of devices and areas, which are not limited in the present disclosure. Taking the area where the mobile device is located is a road area, and the first image is a road image as an example, the solution of the embodiment of the present disclosure will be described below.
  • the road image can be collected by the vision camera on the mobile device in real time.
  • the road image can include multiple categories of first semantic targets near the current location of the mobile device.
  • the categories can be pre-divided according to actual needs, for example According to the function of the first semantic target, it may include, but is not limited to, a combination of at least two or more of the following: pavement indication line category, road travel direction sign category, traffic sign category, traffic signal lamp category, street lamp category, etc.
  • the road surface indicator line category may include at least one of road solid lines, road dashed lines, double yellow lines and other indicator lines for dividing lanes, as well as at least one of stop lines, zebra crossings, and other indicator lines with specific meanings.
  • the road travel direction sign category may include at least one of a turn sign, a diversion line sign, a center circle sign, and the like.
  • the traffic sign category may include at least one of speed limit sign, height limit sign, prohibition sign sign, and road sign sign.
  • the traffic signal light category may include at least one of traffic lights, flashing warning lights, lane lights, and the like.
  • the street lamp category can include various street lamps used to realize the road lighting function or have the function of decorating and beautifying the road.
  • the categories may also be classified according to the position of the first semantic target. For example, the first semantic target on the road is divided into one category, and the first semantic target above the road is divided into another category.
  • the categories can also be divided according to other division conditions, which are not limited in the present disclosure.
  • the first semantic target of multiple categories can be obtained from the road image, for example, the first semantic target of the road surface indicator category, the first semantic target of the traffic signal category, and the road travel direction sign category can be acquired.
  • the first semantic goal A schematic diagram of the first semantic target of an embodiment of the present disclosure is shown in FIG. 2.
  • the first semantic target of the road travel direction sign category is included, such as a left turn direction sign, a straight go direction sign, and a right turn direction sign; road surface indications
  • the first semantic target of the line category such as solid roads and zebra crossings
  • the first semantic target of the traffic signal category such as traffic lights.
  • a second image may be generated according to the pixel values of the first semantic targets of the multiple categories; the second image may be determined according to the second image.
  • different pixel values represent first semantic targets of different categories. For example, the pixel value of the first semantic target of the road indication line category is 1, and the pixel value of the first semantic target of the traffic signal category is 1. 2. Wait. Since the road image may include a lot of background information irrelevant to the first semantic target, by generating a second image and determining the matching semantic target of the first semantic target based on the second image, the efficiency of pose determination can be improved.
  • determining the matching semantic target that matches each first semantic target among the second semantic targets includes: generating a second image according to the pixel value of each first semantic target; determining the second image according to the second image Matching semantic targets in the second semantic target that match each of the first semantic targets.
  • the second image may include category information and location information of each first semantic object.
  • the road image can be input to a pre-trained machine learning model, for example, a deep learning network, to detect the category information and location information of the first semantic target in the road image.
  • the output of the machine learning model may also be an image (that is, the second image). Further, the detection result of the machine learning model may be analyzed to obtain geometric information of each first semantic target, for example, shape information, bounding box information and/or size information of the first semantic target.
  • the position information of multiple vertices of the bounding box of the first semantic target can be obtained;
  • the first semantic target whose shape is a bar for example, Street lights, stop lines, lane lines, etc.
  • the straight line segment sequence information corresponding to the first semantic target including the position information of the endpoints of multiple straight line segments in the straight line segment sequence.
  • a second image can include multiple categories of first semantic targets, and the geometric information of each first semantic target can be parsed separately.
  • the road image includes three first semantic targets, these three first semantic targets are acquired at the same time.
  • the first semantic targets of at least two categories of the first semantic targets of the plurality of categories are located in different spatial dimensions.
  • the acquired first semantic targets of the multiple categories include the first semantic target of the road indicating line category and the first semantic target of the traffic signal category, where the first semantic target of the road indicating line category is on the ground plane.
  • the first semantic target, the first semantic target of the traffic signal category is the first semantic target perpendicular to the ground plane.
  • the semantic map is used to store each second semantic target in the area where the movable device is located and the location information corresponding to the second semantic target.
  • the semantic map may be a high-precision semantic map.
  • the semantic map can be stored in advance. For example, in the case where the start point and the end point of the movable device are known, a semantic map related to the navigation path between the start point and the end point can be acquired and stored.
  • a request may also be sent to the map server when a semantic map needs to be acquired, so as to obtain the semantic map returned by the map server.
  • the second semantic target in the semantic map When acquiring the second semantic target in the semantic map, all the second semantic targets in the semantic map can be acquired, or the second semantic target can be searched within the target search range in the semantic map. Determine the target search range first, and then search for the second semantic target from the target search range, which can effectively reduce the search range and improve the search efficiency.
  • the pose estimation value of the movable device may be obtained, and the target search range in the semantic map may be determined according to the pose estimation value of the movable device.
  • the movable device may be positioned based on the positioning device on the movable device, and then the target search range in the semantic map is determined according to the positioning result.
  • the first pose of the movable device at the first moment may be acquired; the pose estimation value of the movable device at the second moment is determined according to the first pose, and the first pose The time is before the second time.
  • the positioning device may be a global positioning system (Global Positioning System, GPS), an inertial measurement unit (Inertial Measurement Unit, IMU), and so on. In this way, the pose estimation value of the movable device can be quickly determined, and the efficiency of pose estimation can be improved.
  • the second moment may be the moment when the last frame of the first image is captured (may be referred to as the current moment), and the first moment may be the moment when at least one frame of image before the last frame of the first image is captured, for example, After the positioning device on the movable device is initialized, the time corresponding to the first frame of the first image is captured (may be referred to as the initial time).
  • the first pose may be the pose when the last frame of image was taken (referred to as the last frame pose), or the pose of the movable device at the initial moment (referred to as Is the initial pose). Further, in the case that the previous frame of the road image is found, the previous frame pose can be used as the first pose; in the case that the previous frame of the road image is not found Next, the initial pose may be used as the first pose.
  • the pose estimation value may be determined based on the motion model of the movable device.
  • the motion model may be a uniform velocity model, an acceleration model, and so on. Taking the motion model as a constant speed model, the first moment is the initial moment, and the second moment is the current moment as an example.
  • the time difference can be calculated according to the time difference between the initial moment and the current moment, and the travel speed of the movable device and the time difference. And then calculate the current pose estimation value of the movable device according to the initial pose of the movable device and the driving distance.
  • the driving process can be divided into several segments, and each segment can use a motion model.
  • the pose estimation value may include the estimation value of the position, and accordingly, the target search range may be an area within the preset distance range of the position in the semantic map, as shown in the circular area in FIG. 3a.
  • the preset distance may be determined according to the positioning accuracy of the positioning device on the movable device.
  • the positioning accuracy of a consumer-grade GPS device is generally about 10 meters. Therefore, the preset distance can be set to a value of about 10 meters.
  • the pose estimation value may also include both the estimation value of the position and the estimation value of the orientation.
  • the target search range may be a preset distance on the orientation and at the position in the semantic map. The area within the range, as shown in the fan-shaped area in Figure 3b. Determining the target search range according to the orientation can reduce the search area and improve the search efficiency of the second semantic target.
  • the matching semantic target of each first semantic target may be determined from the second semantic target according to the distance between the second semantic target and each first semantic target. For example, for each first semantic target, a second semantic target with the same category as the first semantic target and the shortest distance among the second semantic targets may be determined as the matching semantic target of the first semantic target. In a theoretical situation, the positions of the first semantic target and its matching semantic target are the same in theory. However, due to the error in the pose estimation value, there is a certain position difference between the first semantic target and its matching semantic target. The second semantic target with the shortest distance from the first semantic target is most likely to be the matching semantic target of the first semantic target. Therefore, the matching semantic target of the first semantic target can be determined relatively accurately by the above method.
  • the process of determining the matching semantic target is described.
  • the first semantic target is O 11
  • the second semantic target obtained from the semantic map includes O 21 , O 22 ,..., O 2n , where n is a positive integer
  • O 21, O 22, ..., O 2n distance O 11 assuming are d 1, d 2, ..., d n , then obtain d 1, d 2, ..., d n of
  • the minimum value is assumed to be d k , where k ⁇ ⁇ 1, 2, ..., n ⁇ .
  • the second target is further determined semantic category corresponding to D k is the same as the first semantic object classes, if identical, the second target semantics corresponding to the determined k is O will match the semantics of the target 11 d.
  • the shape and category of the first semantic target, the shape and category of the second semantic target, and the distance between each second semantic target and the first semantic target may also be used.
  • a second semantic target with the same category, the shortest distance and the same shape among the second semantic target may be determined as the matching semantic target of the first semantic target. In this way, the accuracy of determining the matching semantic target can be improved, especially when the number of second semantic targets with the same category and the shortest distance as the first semantic target is greater than one, the accuracy is high.
  • the second semantic target can be projected to On the same plane as the first semantic target, the projected semantic target corresponding to the second semantic target is obtained, and the distance between the projected semantic target and the first semantic target is determined as the distance between the corresponding second semantic target and the first semantic target distance. In this way, the distance between the second semantic target and the first semantic target can be calculated on the same plane, reducing the computational complexity.
  • the projected semantic target of the second semantic target in the second image can be obtained; the position of the second semantic target in the semantic map and the projected semantic target in the The second image corresponds to the position in the second image, and the second image is generated based on the pixel value of each first semantic target; the distance between the projected semantic target and the first semantic target is determined as the second semantic target and the first semantic target A distance between semantic objects.
  • the first position of each first target point in at least one first target point of the first semantic target can be determined; the first target point is determined based on the contour information of the first semantic target Determining the second position of each second target point in at least one second target point of the projected semantic target; the second target point is determined based on the contour information of the projected semantic target; according to each first The first position of the target point and the second position of each second target point determine the distance between the projected semantic target and the first semantic target.
  • the distance between semantic objects can be calculated according to the contour of the semantic object, which improves the accuracy of distance calculation.
  • the first position of each first target point and the second position of each second target point may be determined; for each first target point, the first target point and the corresponding second target point may be determined The distance of the point; the average value of the distance between each of the first target points and the corresponding second target point is determined as the distance between the first semantic target and the projected semantic target.
  • the above-mentioned distance may adopt Euclidean distance, Chebyshev distance, Mahalanobis distance, Ran's distance, etc., which are not limited in the present disclosure.
  • the first target points of the first semantic target are represented by A1, B1, C1, and D1, respectively, and the projection semantic target
  • the second target points are represented by A2, B2, C2, and D2.
  • the contour information of a semantic target can be used to characterize the shape of the semantic target.
  • the contour of the first semantic target is a polygon
  • the first target point is a vertex of a bounding box of the first semantic target.
  • the contour of the first semantic target is a long strip
  • the first target point is a vertex of a line segment corresponding to the first semantic target.
  • the road image or the second image corresponding to the road image can be input to the deep learning network, and the first output image of the deep learning network can be obtained, and the pixels with the same pixel value can be obtained from the first output image
  • the contour of the formed figure ie, the first semantic target
  • a semantic map can be input to a deep learning network, and a second output image of the deep learning network can be obtained, and a graph composed of pixels with the same pixel value can be obtained from the second output image (ie, the second semantic target ), and then smooth the contour to obtain the bounding box or line segment corresponding to the second semantic target.
  • the coordinates of the four vertices A1, B1, C1, and D1 of the quadrilateral can be obtained, and the four corresponding vertices A2 of the bounding box of the projected semantic target can be obtained , B2, C2 and D2 coordinates, and then calculate the distance between the corresponding vertices according to the coordinates, namely A1 and A2, B1 and B2, C1 and C2, D1 and D2, assuming that they are d1, d2, d3 and d4. Then, averaging d1, d2, d3 and d4 to obtain the distance between the first semantic target and the projected semantic target.
  • the number of straight line segments constituting the first semantic target may be multiple, that is, the first semantic target is composed of a sequence of straight line segments including multiple straight line segments, then each of the first semantic target is determined separately The average distance between the straight line segment and the end point of the corresponding straight line segment constituting the projection semantic target, and the average value of the average distance corresponding to each straight line segment is determined as the distance between the first semantic target and the projection semantic target.
  • the projection semantic target can be determined respectively.
  • the distance between the first part and the first part of the first semantic target also called the first distance
  • determine the distance between the second part of the projected semantic target and the second part of the first semantic target also called Is the second distance
  • the distance between the projected semantic target and the first semantic target is determined according to the first distance and the second distance.
  • the average value of the first distance and the second distance may be determined as the distance between the first semantic target and the projected semantic target.
  • O 11 and O 12 are the two first semantic targets
  • O 21 to O 25 are the projection semantic targets corresponding to the second semantic target, according to the first semantic target and the second semantic target
  • O 22 can be determined as the matching semantic target of O 11
  • O 23 can be determined as the matching semantic target of O 12 .
  • a pose constraint condition may be established according to the matching semantic target; the pose of the movable device may be determined according to the pose constraint condition.
  • the pose constraint condition may be determined according to the pose change relationship of the movable device within a preset time period. Due to the introduction of constraint conditions in the pose determination process, when solving the pose, the pose change of the movable device can be constrained according to the pose change relationship of the movable device within a preset time period. Only when the solved pose satisfies the constraint condition, the solved pose is used as the pose of the movable device, which improves the accuracy of the determination of the pose.
  • the driving process of the movable platform can be divided into many small stages. In each stage, it can be assumed that the movable device is driving on a plane. This way, on the one hand, the computational complexity can be reduced, so that the calculation can be When the pose of a mobile device is only three degrees of freedom components (the component in the direction of movement of the movable device, the component in the direction perpendicular to the direction of movement of the movable device on the horizontal plane, and the yaw angle of the movable device) change On the other hand, it can also use this as a constraint to improve calculation accuracy.
  • the plane of the road surface where the movable device is located and the normal vector of the plane may be determined according to the position of the matching semantic target; according to the plane, the normal vector, and the difference between the matching semantic target and the first semantic target At least one of the distances determines the pose distribution error of the movable device; and establishes the pose constraint condition according to the pose distribution error.
  • the height distribution error of the target point of the movable device may be determined according to the plane.
  • the pitch angle distribution error and the roll angle distribution error of the movable device may be determined according to the normal vector.
  • the re-projection distance error may be determined according to the distance between the matching semantic target and the first semantic target. Therefore, the pose constraint condition may be established according to at least one of the height distribution error, the pitch angle distribution error, the roll angle distribution error, and the reprojection distance error.
  • the reprojection distance error includes a point reprojection distance error, that is, multiple vertices of the bounding box of the first semantic object and the projection The sum of the distances of multiple corresponding vertices of the bounding box of the semantic target. In the embodiment shown in Figure 4a, it is the sum of d1, d2, d3 and d4.
  • the reprojection distance error includes the reprojection distance error of the line, that is, the end points of the straight line segment constituting the first semantic target and the projection The sum of the distances of the corresponding endpoints of the straight line segment.
  • the reprojection distance error is the sum of the reprojection distance error of the point and the reprojection distance error of the line.
  • a mapping table may also be stored in advance, and the mapping table is used to record each position in the area where the mobile device is located, the plane (for example, road surface) corresponding to the position, and the normal vector of the plane. . Then, according to the estimated value of the position of the movable device, the plane and the normal vector where the movable device is located can be searched in the mapping table. As the movable device moves on a plane, the height, roll angle, and pitch angle of each point on the movable device (for example, the center point of the movable device) change very little, so the above plane can be used as a priori Information to determine the height distribution error, pitch angle distribution error, and roll angle distribution error.
  • a distance constraint condition can be established, that is, the sum of the height distribution error, the pitch angle distribution error, the roll angle distribution error, and the reprojection distance error is minimized.
  • d k represents the residual term of the optimization target at the current moment k
  • D l and D p represent the error of the straight line segment and the error of the bounding box vertex respectively
  • Q and P represent the set of currently matched straight line segment and the set of bounding box vertices
  • ⁇ represents the three-dimensional information (e.g., straight line segment L i and the bounding box of the vertex P j) by projecting the position and orientation of the current time k T k to the projection function on the road image, the image line features and L i matches as l i, and P j
  • the matching image point feature is p j , the priori information plane of the road plane where the mobile device is located is known.
  • D plane represents the plane constraint
  • the difference between the height z perpendicular to the road surface and the height H of the camera from the road surface Represents the roll angle when the axis around the yaw angle is perpendicular to the road surface in the pose under the plane constraint
  • the distance between the pitch angle ⁇ and the corresponding rotation component of the plane plane, ⁇ , ⁇ , and ⁇ respectively represent the coefficients of each item.
  • the embodiments of the present disclosure can adopt traditional optimization methods (such as optional Gauss-Newton method, Levenberg Marquard method or DogLeg method, etc.), or use filters to iteratively solve the above steps to obtain the pose estimation of the movable device value.
  • the effect of pose estimation is shown in Figure 6.
  • Figure 6 is the right rear view when the vehicle is moving. From bottom to top, the front of the vehicle is a dotted line, a solid line at the edge of the road, a road surface turning sign, a zebra crossing, a traffic light, and a large scale above the road. Road signs. These semantic goals are expressed in the final geometric form.
  • Fig. 7 is a schematic diagram of the principle of pose determination according to an embodiment of the present disclosure.
  • road images can be detected to obtain first semantic targets in multiple categories, and then, the first semantic target of the first semantic target can be obtained.
  • you can query the local semantic map within the preset range obtain the position information of the vertices of the bounding box and the end points of the straight line segments of each second semantic target in the local semantic map, and obtain the plane where the movable device is located.
  • Construct constraint conditions based on the above point, line, and surface information.
  • the determined pose of the movable device is a three-dimensional pose, that is, includes the height of the movable device in the traveling direction of the movable device, the direction perpendicular to the traveling direction of the movable device on the road surface, and the height of the movable device.
  • the coordinates of the direction, and the pitch angle, yaw angle and roll angle of the movable device are included in the traveling direction of the movable device.
  • the embodiments of the present disclosure utilize multiple semantics and solution constraints to achieve the solution of the three-dimensional orientation in the pose estimation process of the movable platform; comprehensively utilize multiple categories of semantic targets and prefabricated semantic maps and visual cameras, even in a single Stable and continuous pose estimation results can also be obtained in regions with relatively sparse semantics; at the same time, pose estimation is solved based on point, line, and surface constraints, which improves the accuracy of pose estimation.
  • the driving state of the movable device may be controlled according to the three-dimensional pose.
  • the driving state includes the speed, acceleration, angle (including at least any one of pitch angle, yaw angle, and roll angle) of the movable device, and the like.
  • the embodiments of the present disclosure can accurately determine the current pose of a movable device in fields such as automatic driving.
  • intelligent driving systems such as ADAS
  • the current mobile device's pose estimation is performed through the embodiments of the present disclosure to improve the accuracy of the current mobile device's pose estimation, and further help ADAS (Advanced Driving Assistant System, Advanced Driving Assistant System) and other systems Carry out more accurate assisted driving (such as emergency avoidance, automatic parking, etc.).
  • ADAS Advanced Driving Assistant System
  • other systems Carry out more accurate assisted driving (such as emergency avoidance, automatic parking, etc.).
  • the method includes step S801 to step S803.
  • Step S801 Obtain the first semantic target in the first image of the area where the movable device is located, and acquire the second semantic target in the semantic map;
  • Step S802 Determine a matching semantic target that matches the first semantic target among the second semantic targets;
  • Step S803 Establish a pose constraint condition according to the matching semantic target, and determine the three-dimensional pose of the movable device according to the pose constraint condition.
  • the first semantic target may include multiple categories of first semantic targets.
  • the pose constraint condition is determined according to the pose change relationship of the movable device within a preset time period.
  • the establishing the pose constraint condition according to the matching semantic target includes: determining the plane where the movable device is located and the normal vector of the plane according to the position of the matching semantic target; At least one of the plane, the normal vector, and the distance between the matching semantic target and the first semantic target determines the pose distribution error of the movable device; and establishes the pose distribution error according to the pose distribution error Pose constraints.
  • the pose distribution error of the movable device is determined according to at least one of the plane, the normal vector, and the distance between the matching semantic target and the first semantic target , Including: determining the height distribution error of the target point of the movable device according to the plane; determining the pitch angle distribution error and the roll angle distribution error of the movable device according to the normal vector; according to the matching semantic target and The distance between the first semantic targets determines the reprojection distance error; the establishing the pose constraint condition according to the pose distribution error includes: according to the height distribution error, the pitch angle distribution error, and the At least one of the roll angle distribution error and the reprojection distance error establishes the pose constraint condition.
  • the pose constraint condition is that the sum of the height distribution error, the pitch angle distribution error, the roll angle distribution error, and the reprojection distance error is minimized.
  • the second semantic target matches the first semantic target.
  • the writing order of the steps does not mean a strict execution order but constitutes any limitation on the implementation process.
  • the specific execution order of each step should be based on its function and possibility.
  • the inner logic is determined.
  • an embodiment of the present disclosure also provides a pose estimation device.
  • the device includes a first acquisition module 901, a first determination module 902, and a second determination module 903.
  • the first acquisition module 901 is configured to acquire a first semantic target in a first image of the area where the movable device is located, and acquire a second semantic target in the semantic map, where the first semantic target belongs to at least two categories, so The first semantic targets of at least two of the at least two categories are located in different spatial dimensions.
  • the first determining module 902 is configured to determine matching semantic targets among the second semantic targets that match each of the first semantic targets.
  • the second determining module 903 is configured to determine the pose of the movable device according to the matching semantic target.
  • the first determining module 902 includes: an image generating unit, configured to generate a second image according to the pixel value of each of the first semantic objects; a first determining unit, configured to generate a second image according to the second image A matching semantic target that matches each of the first semantic targets among the second semantic targets is determined.
  • the first acquiring module 901 includes: a first acquiring unit, configured to acquire the pose estimation value of the movable device, and determine to search the semantic map according to the pose estimation value The search range of the target; the search unit is used to search the second semantic target from the search range of the target.
  • the first obtaining unit includes: an obtaining subunit, configured to obtain a first pose of the movable device at a first moment; a first determining subunit, configured to obtain a first pose according to the first position The pose determines the pose estimate of the movable device at a second moment, and the first moment is before the second moment.
  • the pose estimation value includes an estimation value of a position and an estimation value of an orientation;
  • the target search range is an area on the orientation and within a preset distance range of the position.
  • the first determining module 902 is configured to: for each of the first semantic targets, determine a second semantic target with the same category as the first semantic target and the shortest distance among the second semantic targets Is the matching semantic target of the first semantic target.
  • the device further includes: a second acquisition module, configured to determine a second semantic target with the same category as the first semantic target and the shortest distance among the second semantic targets as the first semantic target Before matching the semantic target of the target, for each of the second semantic targets, obtain the projection semantic target of the second semantic target in the second image; the position of the second semantic target in the semantic map and the projection semantics The position of the target in the second image is corresponding, and the second image is generated based on the pixel value of each of the first semantic targets; the third determining module is used to compare the projected semantic target with the first semantic target The distance between is determined as the distance between the second semantic target and the first semantic target.
  • a second acquisition module configured to determine a second semantic target with the same category as the first semantic target and the shortest distance among the second semantic targets as the first semantic target Before matching the semantic target of the target, for each of the second semantic targets, obtain the projection semantic target of the second semantic target in the second image; the position of the second semantic target in the semantic map and the projection semantics The position of the target
  • the first determining module 902 is configured to: determine a second semantic target with the same category, the shortest distance and the same shape as the first semantic target among the second semantic targets as the first semantic target The matching semantic target.
  • the device further includes: a fourth determining module, configured to determine the first position of each first target point in the at least one first target point of the first semantic target; The target point is determined based on the contour information of the first semantic target; the fifth determining module is configured to determine the second position of each second target point in the at least one second target point of the projected semantic target; The second target point is determined based on the contour information of the projection semantic target; the sixth determining module is configured to determine the projection according to the first position of each first target point and the second position of each second target point The distance between the semantic target and the first semantic target.
  • a fourth determining module configured to determine the first position of each first target point in the at least one first target point of the first semantic target
  • the target point is determined based on the contour information of the first semantic target
  • the fifth determining module is configured to determine the second position of each second target point in the at least one second target point of the projected semantic target
  • the second target point is determined based on the contour information of the projection semantic target
  • the sixth determining module is
  • the sixth determining module includes: a second determining unit configured to determine the first position of each first target point and the second position of each second target point The distance between each first target point and the corresponding second target point; a third determining unit, configured to determine the average value of the distance between each first target point and the corresponding second target point as the first target point The distance between a semantic target and the projected semantic target.
  • the first target point when the contour of the first semantic target is a polygon, the first target point is a vertex of a bounding box of the first semantic target. In some embodiments, when the contour of the first semantic target is a long strip, the first target point is a vertex of a line segment corresponding to the first semantic target.
  • the second determining module 903 includes: a first establishing unit, configured to establish a pose constraint condition according to the matching semantic target; a fourth determining unit, configured to determine a pose constraint condition according to the pose constraint condition Describe the pose of the movable device.
  • the pose constraint condition is determined according to the pose change relationship of the movable device within a preset time period.
  • the first establishing unit includes: a second determining subunit, configured to determine the plane where the movable device is located and the normal vector of the plane according to the position of the matching semantic target; A subunit, configured to determine the pose distribution error of the movable device according to at least one of the plane, the normal vector, and the distance between the matching semantic target and the first semantic target; The unit is used to establish the pose constraint condition according to the pose distribution error.
  • the third determining subunit is configured to: determine the height distribution error of the target point of the movable device according to the plane; determine the pitch angle distribution error of the movable device according to the normal vector And roll angle distribution error; determining the reprojection distance error according to the distance between the matching semantic target and the first semantic target; the establishing subunit is used to: according to the height distribution error and the pitch angle distribution error , At least one of the roll angle distribution error and the reprojection distance error establishes the pose constraint condition.
  • the pose constraint condition is that the sum of the height distribution error, the pitch angle distribution error, the roll angle distribution error, and the reprojection distance error is minimized.
  • the second determining module 903 is configured to determine the three-dimensional pose of the movable device according to the matching semantic target; the apparatus further includes: a control module, configured to determine the three-dimensional pose according to the three-dimensional pose Control the driving state of the movable device.
  • an embodiment of the present disclosure also provides a pose determination device.
  • the device includes a third acquisition module 1001, a seventh determination module 1002, and an eighth determination module 1003.
  • the third acquisition module 1001 is configured to acquire the first semantic target in the first image of the area where the movable device is located, and acquire the second semantic target in the semantic map.
  • the seventh determining module 1002 is configured to determine a matching semantic target in the second semantic target that matches the first semantic target.
  • the eighth determining module 1003 is configured to establish a pose constraint condition according to the matching semantic target, and determine the three-dimensional pose of the movable device according to the pose constraint condition.
  • the pose constraint condition is determined according to the pose change relationship of the movable device within a preset time period.
  • the eighth determining module 1003 includes: a fifth determining unit, configured to determine the plane where the movable device is located and the normal vector of the plane according to the position of the matching semantic target; A unit for determining the pose distribution error of the movable device according to at least one of the plane, the normal vector, and the distance between the matching semantic target and the first semantic target; second establishment The unit is used to establish the pose constraint condition according to the pose distribution error.
  • the sixth determining unit includes: a fourth determining subunit, configured to determine the height distribution error of the target point of the movable device according to the plane; and a fifth determining subunit, configured to determine the height distribution error of the target point according to the plane
  • the normal vector determines the pitch angle distribution error and the roll angle distribution error of the movable device; the sixth determining subunit is used to determine the reprojection distance error according to the distance between the matching semantic target and the first semantic target
  • the second establishing unit is configured to establish the pose constraint condition according to at least one of the height distribution error, the pitch angle distribution error, the roll angle distribution error, and the reprojection distance error.
  • the pose constraint condition is that the sum of the height distribution error, the pitch angle distribution error, the roll angle distribution error, and the reprojection distance error is minimized.
  • the functions or modules contained in the device provided in the embodiments of the present disclosure can be used to execute the methods described in the above method embodiments.
  • the functions or modules contained in the device provided in the embodiments of the present disclosure can be used to execute the methods described in the above method embodiments.
  • the device embodiments described above are merely illustrative, and the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, that is, they may be located in One place, or it can be distributed to multiple network modules. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution in this specification. Those of ordinary skill in the art can understand and implement without creative work.
  • an embodiment of the present disclosure also provides a computer device, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor.
  • the processor executes the program described in any of the embodiments. The method described.
  • the embodiments of the apparatus in this specification can be applied to computer equipment, such as servers or terminal equipment.
  • the device embodiments can be implemented by software, or can be implemented by hardware or a combination of software and hardware.
  • Taking software implementation as an example as a logical device, it is formed by reading the corresponding computer program instructions in the non-volatile memory into the memory by the processor that processes the file where it is located.
  • FIG. 11 it is a hardware structure diagram of the computer equipment where the device of this specification is located, except for the processor 1101, the memory 1102, the network interface 1103, and the non-volatile memory 1104 shown in FIG.
  • the server or electronic device where the device is located in the embodiment may also include other hardware according to the actual function of the computer device, which will not be repeated here.
  • an embodiment of the present disclosure also provides a computer storage medium on which a computer program is stored, and when the program is executed by a processor, the method described in any of the embodiments is implemented.
  • an embodiment of the present disclosure further provides a computer program, the computer program is stored on a storage medium, and when the computer program is executed by a processor, the method described in any of the embodiments is implemented.
  • the present disclosure may take the form of a computer program product implemented on one or more storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing program codes.
  • Computer usable storage media include permanent and non-permanent, removable and non-removable media, and information storage can be realized by any method or technology.
  • the information can be computer-readable commands, data structures, program modules, or other data.
  • Examples of computer storage media include, but are not limited to: phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, CD-ROM, digital versatile disc (DVD) or other optical storage, Magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media can be used to store information that can be accessed by computing devices.
  • PRAM phase change memory
  • SRAM static random access memory
  • DRAM dynamic random access memory
  • RAM random access memory
  • ROM read-only memory
  • EEPROM electrically erasable programmable read-only memory
  • flash memory or other memory technology
  • CD-ROM compact disc
  • DVD digital versatile disc
  • Magnetic cassettes magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media can be used to store information that can be accessed by computing devices.

Landscapes

  • Engineering & Computer Science (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Automation & Control Theory (AREA)
  • Multimedia (AREA)
  • Aviation & Aerospace Engineering (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Electromagnetism (AREA)
  • Image Analysis (AREA)
  • Control Of Driving Devices And Active Controlling Of Vehicle (AREA)
PCT/CN2021/075016 2020-03-24 2021-02-03 一种位姿确定方法、装置、介质和设备 WO2021190167A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP2022504292A JP2022542082A (ja) 2020-03-24 2021-02-03 ポーズ特定方法、ポーズ特定装置、コンピュータ可読記憶媒体、コンピュータ機器及びコンピュータプログラム

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010213807.9 2020-03-24
CN202010213807.9A CN111427373B (zh) 2020-03-24 2020-03-24 一种位姿确定方法、装置、介质和设备

Publications (1)

Publication Number Publication Date
WO2021190167A1 true WO2021190167A1 (zh) 2021-09-30

Family

ID=71549104

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/075016 WO2021190167A1 (zh) 2020-03-24 2021-02-03 一种位姿确定方法、装置、介质和设备

Country Status (3)

Country Link
JP (1) JP2022542082A (ja)
CN (1) CN111427373B (ja)
WO (1) WO2021190167A1 (ja)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102022212704A1 (de) 2022-11-28 2024-05-29 Robert Bosch Gesellschaft mit beschränkter Haftung Verfahren zum Abgleichen einer digitalen Straßenkarte

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111427373B (zh) * 2020-03-24 2023-11-24 上海商汤临港智能科技有限公司 一种位姿确定方法、装置、介质和设备
CN114063091A (zh) * 2020-07-30 2022-02-18 北京四维图新科技股份有限公司 高精度的定位方法及产品
CN114543819B (zh) * 2021-09-16 2024-03-26 北京小米移动软件有限公司 车辆定位方法、装置、电子设备及存储介质

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107144285A (zh) * 2017-05-08 2017-09-08 深圳地平线机器人科技有限公司 位姿信息确定方法、装置和可移动设备
CN109544629A (zh) * 2018-11-29 2019-03-29 南京人工智能高等研究院有限公司 摄像头位姿确定方法和装置以及电子设备
CN109870689A (zh) * 2019-01-08 2019-06-11 武汉中海庭数据技术有限公司 毫米波雷达与高精矢量地图匹配的车道级定位方法与系统
CN110147705A (zh) * 2018-08-28 2019-08-20 北京初速度科技有限公司 一种基于视觉感知的车辆定位方法及电子设备
US10437252B1 (en) * 2017-09-08 2019-10-08 Perceptln Shenzhen Limited High-precision multi-layer visual and semantic map for autonomous driving
CN111427373A (zh) * 2020-03-24 2020-07-17 上海商汤临港智能科技有限公司 一种位姿确定方法、装置、介质和设备

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190141481A1 (en) * 2016-04-28 2019-05-09 Nec Corporation Management system, mobile body, management device, velocity notification method, management method, and storage medium having program stored thereon
EP3551967A2 (en) * 2016-12-09 2019-10-16 TomTom Global Content B.V. Method and system for video-based positioning and mapping
US10963800B2 (en) * 2017-07-25 2021-03-30 General Electric Company Service layer augmentation of response to semantically-informed query of arbitrary external data sources
US10546387B2 (en) * 2017-09-08 2020-01-28 Qualcomm Incorporated Pose determination with semantic segmentation
US10657388B2 (en) * 2018-03-13 2020-05-19 Honda Motor Co., Ltd. Robust simultaneous localization and mapping via removal of dynamic traffic participants
CN109002837A (zh) * 2018-06-21 2018-12-14 网易（杭州）网络有限公司 一种图像语义分类方法、介质、装置和计算设备
JP2019219988A (ja) * 2018-06-21 2019-12-26 日本電信電話株式会社 意味情報付与装置、意味情報付与方法、およびプログラム
CN109345574B (zh) * 2018-08-31 2020-10-09 西安电子科技大学 基于语义点云配准的激光雷达三维建图方法
CN109815847B (zh) * 2018-12-30 2020-12-01 中国电子科技集团公司信息科学研究院 一种基于语义约束的视觉slam方法
CN110335319B (zh) * 2019-06-26 2022-03-18 华中科技大学 一种语义驱动的相机定位与地图重建方法和系统

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107144285A (zh) * 2017-05-08 2017-09-08 深圳地平线机器人科技有限公司 位姿信息确定方法、装置和可移动设备
US10437252B1 (en) * 2017-09-08 2019-10-08 Perceptln Shenzhen Limited High-precision multi-layer visual and semantic map for autonomous driving
CN110147705A (zh) * 2018-08-28 2019-08-20 北京初速度科技有限公司 一种基于视觉感知的车辆定位方法及电子设备
CN109544629A (zh) * 2018-11-29 2019-03-29 南京人工智能高等研究院有限公司 摄像头位姿确定方法和装置以及电子设备
CN109870689A (zh) * 2019-01-08 2019-06-11 武汉中海庭数据技术有限公司 毫米波雷达与高精矢量地图匹配的车道级定位方法与系统
CN111427373A (zh) * 2020-03-24 2020-07-17 上海商汤临港智能科技有限公司 一种位姿确定方法、装置、介质和设备

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102022212704A1 (de) 2022-11-28 2024-05-29 Robert Bosch Gesellschaft mit beschränkter Haftung Verfahren zum Abgleichen einer digitalen Straßenkarte

Also Published As

Publication number Publication date
CN111427373B (zh) 2023-11-24
JP2022542082A (ja) 2022-09-29
CN111427373A (zh) 2020-07-17

Similar Documents

Publication Publication Date Title
WO2021190167A1 (zh) 一种位姿确定方法、装置、介质和设备
US11113544B2 (en) Method and apparatus providing information for driving vehicle
US11175145B2 (en) System and method for precision localization and mapping
CN112105893B (zh) 用于自动驾驶车辆的实时地图生成系统
CN111771229B (zh) 用于自动驾驶车辆的点云重影效果检测系统
WO2020154967A1 (en) Map partition system for autonomous vehicles
US9495602B2 (en) Image and map-based detection of vehicles at intersections
WO2020154966A1 (en) A rgb point clouds based map generation system for autonomous vehicles
CN111582189B (zh) 交通信号灯识别方法、装置、车载控制终端及机动车
CN108981730A (zh) 用于为操作自动驾驶车辆生成参考路径的方法和系统
EP3707469A1 (en) A point clouds registration system for autonomous vehicles
WO2020043081A1 (zh) 定位技术
CN111220143B (zh) 一种成像设备的位置和姿态确定方法及装置
CN111780771A (zh) 一种定位方法、装置、电子设备及计算机可读存储介质
Burnett et al. Building a winning self-driving car in six months
Li et al. Robust localization for intelligent vehicles based on compressed road scene map in urban environments
Laflamme et al. Driving datasets literature review
JP7321035B2 (ja) 物体位置検出方法及び物体位置検出装置
CN112735163B (zh) 确定目标物体静止状态的方法、路侧设备、云控平台
CN113189610A (zh) 地图增强的自动驾驶多目标追踪方法和相关设备
Deusch Random finite set-based localization and SLAM for highly automated vehicles
Yan [Retracted] Vehicle Safety‐Assisted Driving Technology Based on Computer Artificial Intelligence Environment
Evlampev et al. Map relative localization based on road lane matching with Iterative Closest Point algorithm
Buczko et al. Efficient global localization using vision and digital offline map
US20240239368A1 (en) Systems and methods for navigating a vehicle by dynamic map creation based on lane segmentation

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21774089

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2022504292

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205N DATED 15/11/2022)

122 Ep: pct application non-entry in european phase

Ref document number: 21774089

Country of ref document: EP

Kind code of ref document: A1