WO2020253842A1 - 一种车辆位姿确定方法、装置及电子设备 - Google Patents

一种车辆位姿确定方法、装置及电子设备 Download PDF

Info

Publication number
WO2020253842A1
WO2020253842A1 PCT/CN2020/097180 CN2020097180W WO2020253842A1 WO 2020253842 A1 WO2020253842 A1 WO 2020253842A1 CN 2020097180 W CN2020097180 W CN 2020097180W WO 2020253842 A1 WO2020253842 A1 WO 2020253842A1
Authority
WO
WIPO (PCT)
Prior art keywords
vehicle
semantic information
pose
scene
semantic
Prior art date
Application number
PCT/CN2020/097180
Other languages
English (en)
French (fr)
Inventor
胡兵
吕吉鑫
孟超
Original Assignee
杭州海康威视数字技术股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from CN201910537501.6A external-priority patent/CN112116654B/zh
Application filed by 杭州海康威视数字技术股份有限公司 filed Critical 杭州海康威视数字技术股份有限公司
Priority to EP20826498.6A priority Critical patent/EP3989170A4/en
Publication of WO2020253842A1 publication Critical patent/WO2020253842A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
    • G06V20/586—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads of parking space
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • G06V20/588—Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/70—Labelling scene content, e.g. deriving syntactic or semantic representations
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10028—Range image; Depth image; 3D point clouds
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30248—Vehicle exterior or interior
    • G06T2207/30252—Vehicle exterior; Vicinity of vehicle

Definitions

  • This application relates to the field of image analysis technology, and in particular to a method, device and electronic device for determining the pose of a vehicle.
  • high-precision, high-frequency pose determination of the vehicle.
  • high-precision, high-frequency vehicle The pose is determined.
  • GPS Global Position System
  • GPS sensors can be installed on the vehicle to determine the vehicle's pose.
  • GPS sensors need to receive GPS signals sent by GPS base stations to work properly.
  • the vehicle may be in a scene with a poor signal, such as an underground parking lot, and the GPS sensor may not receive the GPS signal normally, resulting in the inability to determine the vehicle's pose.
  • the purpose of the embodiments of the present application is to provide a vehicle pose determination method, device, and electronic equipment, so as to accurately determine the pose in a scene where GPS signals cannot be received normally.
  • the specific technical solutions are as follows:
  • a method for determining the pose of a vehicle is provided.
  • the vehicle to be positioned is provided with a surround view camera, and the method includes:
  • the global semantic information of the preset area including the scene where the vehicle to be located determines the pose matching the local semantic information in the preset area as the location of the vehicle to be located. posture.
  • the semantic segmentation of the bird's-eye view to obtain local semantic information of the scene where the vehicle to be located is located includes:
  • the pose matching the local semantic information is determined in the preset area as the location where the vehicle to be located is located.
  • the poses include:
  • the local semantic information is matched with the global semantic information of a preset area to obtain a pose in the preset area that matches the local semantic information ,
  • the global semantic information is the semantic point cloud of the preset area.
  • the identification type includes: lane line, parking space frame, parking line, speed bump, road arrow, and parking space number.
  • the surround-view camera includes a plurality of fish-eye cameras, and the plurality of fish-eye cameras are respectively set in different directions of the vehicle to be positioned for shooting images in corresponding directions;
  • the method further includes:
  • the images captured by the multiple fisheye cameras are transformed and spliced to obtain a bird's-eye view of the scene where the vehicle to be located is located.
  • the vehicle to be located is further provided with a pose sensor for measuring the relative pose of the vehicle to be located at different time nodes;
  • the semantic segmentation of the bird's-eye view to obtain local semantic information of the scene where the vehicle to be located is located includes:
  • the superimposed result is filtered to obtain the semantic information of the scene in which the vehicle to be located is located in a time window including the current time as local semantic information.
  • a device for determining a vehicle pose is provided.
  • the vehicle to be positioned is provided with a surround view camera, and the surround view camera is used to take a bird's eye view of the scene where the vehicle to be positioned is located.
  • a semantic segmentation module configured to perform semantic segmentation on the bird's-eye view to obtain local semantic information of the scene where the vehicle to be located is located;
  • the semantic matching module is used to determine a pose matching the local semantic information in the preset area according to the global semantic information of the preset area including the scene where the vehicle to be located is located, as the pending The pose of the vehicle.
  • the semantic segmentation module is specifically configured to perform semantic segmentation on the bird's-eye view to obtain the semantic point cloud of the scene in which the vehicle to be located is located, as the local semantic information, where the semantic point
  • the cloud is used to represent geometric information and semantic information of each spatial point, and the semantic information represents the identification type corresponding to each spatial point;
  • the semantic matching module is specifically configured to match the local semantic information with the global semantic information of the preset area based on the geometric information and semantic information represented by the local semantic information to obtain the middle and The pose matched by the local semantic information is used as the pose of the vehicle to be located, and the global semantic information is the semantic point cloud of the preset area.
  • the semantic segmentation module is specifically configured to perform semantic segmentation on the bird's-eye view to obtain the semantic point cloud of the scene in which the vehicle to be located is located, as the local semantic information, where the semantic point
  • the cloud is used to represent geometric information and semantic information of each spatial point, and the semantic information represents the identification type corresponding to each spatial point;
  • the semantic matching module is specifically configured to match the local semantic information with the global semantic information of the preset area based on the geometric information and semantic information represented by the local semantic information to obtain the middle and The pose matched by the local semantic information is used as the pose of the vehicle to be located, and the global semantic information is the semantic point cloud of the preset area.
  • the identification type includes: lane line, parking space frame, parking line, speed bump, road arrow, and parking space number.
  • the surround view camera includes a plurality of fish-eye cameras, and the plurality of fish-eye cameras are respectively set in different orientations of the vehicle to be positioned and used for shooting images in corresponding directions;
  • the device further includes an image stitching module, configured to perform semantic segmentation on the bird's-eye view to obtain local semantic information of the scene where the vehicle to be located is located, the method further includes:
  • the images captured by the multiple fisheye cameras are transformed and spliced to obtain a bird's-eye view of the scene where the vehicle to be located is located.
  • the vehicle to be located is further provided with a pose sensor for measuring the relative pose of the vehicle to be located at different time nodes;
  • the semantic segmentation module is specifically configured to perform semantic segmentation on a bird's-eye view captured at multiple time nodes including the current time, to obtain semantic information of the scene where the vehicle to be located is located at the multiple time nodes ;
  • the superimposed result is filtered to obtain the semantic information of the scene in which the vehicle to be located is located in a time window including the current time as local semantic information.
  • the semantic segmentation module is further configured to perform semantic segmentation on the bird's-eye view captured at multiple time nodes including the current moment, to obtain that the vehicle to be located is in the Obtaining the visual relative poses on the multiple time nodes after the semantic information of the scenes at the multiple time nodes, where the visual relative poses are obtained by matching the semantic information of the multiple time nodes;
  • the semantic segmentation module is specifically configured to superimpose the semantic information of the scene in which the vehicle to be located is located on the multiple time nodes based on the fused relative pose of the vehicle to be located on the multiple time nodes , Get the superposition result.
  • the semantic matching module is specifically configured to estimate the current location of the vehicle to be located based on the determined location of the vehicle to be located at the current moment, as a prediction Estimate location
  • the prior semantic information in the preset range of the estimated position in the preset area determine the pose matching the local semantic information within the preset range of the preset position as the vehicle to be located The pose.
  • an electronic device including:
  • Memory used to store computer programs
  • the processor is configured to implement the method steps in any one of the foregoing first aspects when executing the program stored in the memory.
  • a computer-readable storage medium is provided, and a computer program is stored in the computer-readable storage medium. The method steps described.
  • a vehicle pose surround view system including: a surround view camera and at least one processor, the surround view camera is used to take a bird's eye view of the scene where the vehicle to be located is located, so The processor is used to: perform semantic segmentation on the bird's-eye view to obtain local semantic information of the scene where the vehicle to be located is located; according to the global semantic information of a preset area including the scene where the vehicle to be located is located, A pose matching the local semantic information is determined in the preset area as the pose of the vehicle to be located.
  • the vehicle pose determination method, device, and electronic equipment provided by the embodiments of the present application can extract semantic information from a bird’s-eye view captured by a surround-view camera to obtain the characteristics of the scene in which the vehicle to be located is located, and then determine through map matching The pose of the vehicle to be located.
  • the pose can be determined without the aid of GPS signals. Therefore, the pose can be accurately determined even in the scene where GPS signals cannot be received normally.
  • implementing any product or method of the present application does not necessarily need to achieve all the advantages described above at the same time.
  • FIG. 1 is a schematic flowchart of a method for determining a vehicle pose provided by an embodiment of the application
  • FIG. 2 is a schematic diagram of another flow chart of a method for determining a vehicle pose provided by an embodiment of the application;
  • FIG. 3 is a schematic diagram of a principle of determining a vehicle pose provided by an embodiment of the application.
  • FIG. 4 is a schematic diagram of a structure of a vehicle pose determination device provided by an embodiment of the application.
  • Fig. 5 is a schematic structural diagram of an electronic device for determining a vehicle pose provided by an embodiment of the application.
  • Fig. 1 is a schematic flowchart of a method for determining a vehicle pose provided by an embodiment of the application, which may include:
  • S101 Perform semantic segmentation on the bird's-eye view to obtain local semantic information of the scene where the vehicle to be located is located.
  • the local semantic information can be used to indicate the identification type and location information of the identification in the scene where the vehicle to be located is located.
  • the local semantic information may indicate that in the scene where the vehicle to be located is located, the spatial point with the spatial coordinate (1,2,3) belongs to the parking line.
  • the representation of local semantic information can be different according to different application scenarios, which is not limited in this embodiment.
  • the local semantic information may be expressed in the form of a semantic point cloud.
  • the semantic point cloud is used to represent the geometric information and semantic information of each spatial point, and the semantic information represents the identification type corresponding to each spatial point.
  • the identification type can be different.
  • the identification type may include one or more of ground identifications such as lane lines, parking space frames, parking lines, speed bumps, road arrows, and parking space numbers.
  • the semantic point cloud can be regarded as a collection of multiple points, and each point can be represented in the form of (loc, label), where loc represents the spatial coordinates of the point, and label represents the label corresponding to the identification type corresponding to the point, such as existence A point with spatial coordinates (3, 2, 0), and the point belongs to the parking line. Assuming that the label corresponding to the parking line is 2, the point in the semantic point cloud can be represented as (3, 2, 0, 2) .
  • S102 According to the global semantic information of the preset area including the scene where the vehicle to be located is located, determine a pose matching the local semantic information in the preset area as the pose of the vehicle to be located.
  • the representation of global semantic information is the same as that of local semantic information.
  • the local semantic information is expressed in the form of a semantic point cloud
  • the global semantic information is also expressed in the form of a semantic point cloud.
  • the matching of the pose and the local semantic information may refer to that based on the global semantic information, assuming that the vehicle to be located is in the pose, the theoretically measured local semantic information can be matched with the actual local semantic information measured in S101.
  • the way of determining the pose matching the local semantic confidence can also be different. It can be based on the geometric information and semantic information represented by the local semantic information, the local semantic information is matched with the global semantic information of the preset area, and the pose matching the local semantic information in the preset area is obtained as the vehicle to be located. Posture.
  • the global semantic information and local semantic information in the form of semantic point cloud as an example for description, which may include the following steps:
  • Step 1 Based on the geometric information and semantic information represented by the local semantic information, the local semantic information is matched with the preset global semantic information to obtain the matching point pair of the local semantic information and the global semantic information.
  • a point in the local semantic information matches the geometric information of a point in the global semantic information, and the semantic information also matches, it can be considered that these two points correspond to the same spatial point, that is, the two points constitute connection relation.
  • the current position of the vehicle to be located based on the determined position of the vehicle to be located at the historical moment as the estimated location.
  • the relative pose change within 5s can be read from the pose sensor to obtain the estimated position A'.
  • the semantic information in the preset range near the position A ⁇ is intercepted as the prior semantic information, and by matching the prior semantic information with the local semantic information, the difference between the local semantic information and the prior semantic information is obtained. Matching point pairs can be understood.
  • the prior semantic information is part of the global semantic information
  • the matching point pairs between the local semantic information and the prior semantic information can be regarded as the matching point pairs between the local semantic information and the global semantic information.
  • the prior semantic information is a part of the global semantic information, so the amount of calculation required to complete the matching of local semantic information and prior semantic information is lower than the amount of calculation required to complete the matching of local semantic information and global semantic information.
  • Step 2 Determine the pose of the vehicle to be located in the spatial coordinate system of the global semantic information based on the coordinate conversion relationship of the matching point pair.
  • the spatial coordinate system of global semantic information is the global coordinate system
  • the spatial coordinate system of local semantic information is the local coordinate system.
  • a matching point pair includes a point in the global semantic information and a matching point in the local semantic information. As in the previous analysis, these two points theoretically represent the same point in the space.
  • the coordinates in the global coordinate system and the coordinates in the local coordinate system are expressed. Therefore, by integrating multiple matching point pairs, the coordinate transformation relationship between the global coordinate system and the local coordinate system can be determined.
  • the relative positional relationship between the vehicle to be positioned and the surround view camera can be considered to be fixed and known, and the local semantic information is the semantic point cloud of the scene where the vehicle to be positioned is constructed based on the bird's-eye view captured by the surround view camera. It is considered that the local coordinates and heading angle of the vehicle to be positioned in the local coordinate system are known. Furthermore, based on the coordinate conversion relationship between the global coordinate system and the local coordinate system reflected by the matching point pairs, the global coordinates and heading angles of the vehicle to be located in the global coordinate system can be determined to determine the pose of the vehicle to be located .
  • semantic information can be extracted from the bird's-eye view captured by the surrounding camera to obtain the characteristics of the scene where the vehicle to be located is located, and then the pose of the vehicle to be located can be determined through map matching.
  • the pose can be determined without the aid of GPS signals. Therefore, the pose can be determined even in a scene where GPS signals cannot be received normally.
  • a laser localizer and/or a sonic localizer may be provided on the vehicle, and the position and attitude can be determined by the laser localizer and/or the sonic localizer.
  • the cost of laser positioners and acoustic wave positioners is often high.
  • the pose determination can be achieved without the use of a laser localizer and a sonic localizer, and the cost of image acquisition equipment required to take a bird's-eye view is often low Due to the cost of the laser localizer and the acoustic wave localizer, compared with the related technology, the solution of using the laser localizer and/or the acoustic wave localizer to determine the pose can effectively reduce the cost of the pose determination.
  • FIG. 2 is a schematic diagram of another flow chart of the vehicle pose determination method provided by an embodiment of the application, wherein the vehicle to be positioned is provided with a pose sensor and a surround view camera, and the pose sensor is used to measure the vehicle to be positioned Relative poses at different time nodes.
  • Methods can include:
  • S201 Perform semantic segmentation on a bird's-eye view captured at multiple time nodes including the current time to obtain a semantic point cloud of a scene where the vehicle to be located is located at multiple time nodes.
  • the current time is t1
  • multiple time nodes are ⁇ t1, t2..., tn ⁇
  • the bird's-eye view captured on ti is called bird's-eye view i
  • the vehicle to be located is located on ti
  • the semantic point cloud of the scene is called the semantic point cloud i, where i is any positive integer in [1,n].
  • Different bird's-eye views can be semantically segmented at different times, or they can be semantically segmented in parallel at the same time. For example, after each bird's-eye view is captured, the captured bird's-eye view can be segmented separately. Semantic segmentation may also be to perform semantic segmentation on the aerial view 1-aerial view n in parallel after the aerial view 1-aerial view n is captured, which is not limited in this embodiment.
  • S202 Obtain the relative pose of the vehicle to be located measured by the pose sensor at multiple time nodes, as a sensed relative pose.
  • the pose sensor may include an IMU (Inertial Measurement Unit) and a wheel speed counter.
  • the IMU is used to measure the three-axis acceleration and angular velocity of the vehicle
  • the wheel speed counter is used to measure the number of rotations of the vehicle tires, and based on the measured number of rotations, calculate the vehicle's movement distance.
  • the relative pose of the vehicle to be located at a time node can be the pose of the vehicle to be located at that time node, relative to the pose of the vehicle to be located at a time node before the time node The amount of change. Given the three-axis acceleration, angular velocity, and distance of the vehicle, the amount of change in the movement of the vehicle within a specified time window can be calculated, so the pose sensor can measure the relative pose of the vehicle to be positioned.
  • the sampling frequency of the pose sensor and the surround view camera may be different, so a time synchronization unit can be set to synchronize the bird's-eye view collected by the surround view camera and the relative pose measured by the pose sensor.
  • S203 Based on the sensed relative poses of the vehicle to be located at multiple time nodes, superimpose the semantic point clouds of the scene where the vehicle to be located is located at multiple time nodes to obtain an overlay result.
  • the sensing relative pose of the vehicle to be positioned on ti is called loci, then the semantic point cloud 1 can be set in loc1, and the semantic point cloud 2 can be superimposed on loc2...and so on, until the semantic point Set the cloud n superposition at locn to get the superposition result.
  • the pose sensor may have certain errors. Taking the pose sensor including IMU and wheel speed counter as an example, there may be accumulated errors and random fluctuations in the IMU and wheel speed counter, resulting in the measured three-axis acceleration, angular velocity and The vehicle movement distance is not accurate enough. Based on the inaccurate three-axis acceleration, angular velocity, and vehicle movement distance, the relative pose obtained is not accurate enough, that is, there may be a certain error in sensing the relative pose.
  • the semantic point cloud of the scene where the vehicle to be located is located at multiple time nodes can be matched to obtain the relative pose of the vehicle to be located at multiple time nodes, as Visual relative pose. And fusion sense relative pose and visual relative pose, get fused relative pose.
  • the fusion relative pose can be regarded as a correction result obtained by correcting the sensing relative pose based on the visual relative pose. Therefore, the fused relative pose obtained by fusion is more accurate than the sensed relative pose. Therefore, based on the fusion of relative poses, superimposing the semantic point cloud of the scene where the vehicle to be located at multiple time nodes is located can make the obtained superimposition result more accurate.
  • S204 Filter the superimposition result to obtain the semantic point cloud of the scene where the vehicle to be located is located in the time window including the current moment, as local semantic information.
  • the algorithm used for filtering can be different according to different application scenarios, which is not limited in this embodiment. Since the superimposed result is the semantic point cloud of the scene where the vehicle to be located is located at multiple time nodes including the current moment, the result obtained after filtering can be regarded as the location of the vehicle to be located in the time window including the current moment.
  • the semantic point cloud of the scene can also be regarded as the semantic point cloud of the scene within a certain distance of the most recent movement of the vehicle to be located.
  • S205 According to the global semantic information of the preset area including the scene where the vehicle to be located is located, determine a pose matching the local semantic information in the preset area as the pose of the vehicle to be located.
  • This step is the same as S102, and can refer to the related description of S102, which will not be repeated here.
  • the geometric information and semantic information included in the semantic point cloud of the scene where the vehicle to be positioned at the current moment is often limited, which may result in multiple poses matching the local semantic information in the global semantic information, which may lead to Local semantics cannot accurately match global semantic information.
  • the semantic point cloud of the scene at different time nodes can be superimposed to increase the geometric information and semantic information contained in the local semantic information, and reduce the presence of multiple poses and local semantics in the global semantic information. The possibility of information matching enables local semantic information to be more accurately matched with global semantic information, thereby improving the accuracy of the determined pose.
  • the surround view camera includes four fisheye cameras (included in other optional embodiments).
  • the number of fisheye cameras can also be different), respectively set around the vehicle to be located.
  • the vehicle to be located is also provided with an IMU and a wheel speed counter. See Figure 3, including:
  • S301 Four fisheye cameras capture images in corresponding directions, and send the captured images and time stamps to the data collection platform.
  • the data collection platform may be set on the vehicle, or may be set on a network device that has a communication connection with the vehicle.
  • the image sent by the fisheye camera contains a timestamp indicating the time of shooting.
  • the data acquisition platform transforms and stitches the images captured by multiple fisheye cameras to obtain a bird's-eye view of the scene where the vehicle to be located is located.
  • the data collection platform performs semantic segmentation on the bird's-eye view to obtain a semantic point cloud of the scene where the vehicle to be located is located.
  • This step is the same as S101, please refer to the foregoing description of S101, which will not be repeated here.
  • S304 The data collection platform synchronously reads the IMU data and wheel speed counter data corresponding to each image according to the received time stamp.
  • the IMU data is the data measured by the IMU
  • the wheel speed counter data is the data measured by the wheel speed counter.
  • the data collection platform sends the obtained IMU data, wheel speed counter data, and semantic point cloud to the data analysis platform.
  • the data analysis platform may be set on the vehicle, or may be set on a network device that has a communication connection with the vehicle.
  • the data analysis platform determines the sensing relative pose based on IMU data and wheel speed counter data, and determines the visual relative pose based on semantic point clouds at different time nodes.
  • sensing relative pose and the visual relative pose please refer to the relevant descriptions in the foregoing S202 and S203, which will not be repeated here.
  • S307 The data analysis platform fuses and senses the relative pose and the visual relative pose to obtain the fused relative pose.
  • the data analysis platform superimposes the semantic point cloud of the scene where the vehicle to be located is located at multiple time nodes based on the fused relative pose of the vehicle to be located at multiple time nodes to obtain a superimposed result.
  • the data analysis platform filters the superimposition result to obtain a semantic point cloud of the scene where the vehicle to be located is located in a time window including the current moment, as local semantic information.
  • the data analysis platform Based on the geometric information and semantic information represented by the local semantic map, the data analysis platform matches the local semantic information with the preset global semantic information to obtain the matching point pair of the local semantic information and the global semantic information.
  • the data analysis platform determines the pose of the vehicle to be located in the spatial coordinate system of the global semantic information based on the coordinate transformation relationship of the matching point pair.
  • the data acquisition platform and the data analysis platform in this embodiment may be two independent physical devices, or two different virtual devices integrated on the same electronic device, which is not limited in this embodiment.
  • a data display platform may also be included for displaying the vehicle according to the pose determined by the data analysis platform.
  • Fig. 4 is a schematic structural diagram of a vehicle pose determination device provided by an embodiment of the application, which may include:
  • the semantic segmentation module 401 is used to perform semantic segmentation on the bird's-eye view to obtain local semantic information of the scene where the vehicle to be located is located;
  • the semantic matching module 402 is used to determine the pose matching the local semantic information in the preset area according to the global semantic information of the preset area including the scene where the vehicle to be located is located, as the pose of the vehicle to be located .
  • the semantic segmentation module 401 is specifically configured to perform semantic segmentation on the bird's-eye view to obtain the semantic point cloud of the scene where the vehicle to be located is located, as local semantic information, where the semantic point cloud is used to represent The geometric information and semantic information of each spatial point, and the semantic information indicates the identification type corresponding to each spatial point;
  • the semantic matching module 402 is specifically used to match the local semantic information with the global semantic information of the preset area based on the geometric information and semantic information represented by the local semantic information, to obtain the pose matching the local semantic information in the preset area , As the pose of the vehicle to be located, the global semantic information is the semantic point cloud of the preset area.
  • the identification type includes: lane line, parking space frame, parking line, speed bump, road arrow, and parking space number.
  • the surround view camera includes a plurality of fish-eye cameras, and the plurality of fish-eye cameras are respectively arranged in different directions of the vehicle to be positioned, and used to shoot images in corresponding directions;
  • the device also includes an image splicing module, which is used to perform semantic segmentation on the bird's-eye view to obtain local semantic information of the scene where the vehicle to be located is located, and the method further includes:
  • the images captured by multiple fisheye cameras are transformed and spliced to obtain a bird's-eye view of the scene where the vehicle to be located is located.
  • the vehicle to be located is further provided with a pose sensor, which is used to measure the relative pose of the vehicle to be located at different time nodes;
  • the semantic segmentation module 401 is specifically configured to perform semantic segmentation on the bird's-eye view taken at multiple time nodes including the current time, to obtain semantic information of the scene where the vehicle to be located is located at multiple time nodes;
  • the superposition result is filtered to obtain the semantic information of the scene where the vehicle to be located is located in the time window including the current moment, as the local semantic information.
  • the semantic segmentation module 401 is also used to perform semantic segmentation on the bird's-eye view captured at multiple time nodes including the current moment, to obtain the location of the vehicle to be located at multiple time nodes. After the semantic information of the scene, the visual relative poses at multiple time nodes are obtained, and the visual relative poses are obtained by matching the semantic information of multiple time nodes;
  • the semantic segmentation module 401 is specifically configured to superimpose the semantic information of the scene where the vehicle to be located is located at multiple time nodes based on the fused relative pose of the vehicle to be located at multiple time nodes to obtain the superimposed result.
  • the semantic matching module 402 is specifically configured to estimate the current location of the vehicle to be located based on the determined location of the vehicle to be located at the historical moment, as the estimated location;
  • a pose matching the local semantic information is determined within the preset range of the preset location as the pose of the vehicle to be located.
  • An embodiment of the present application also provides an electronic device, as shown in FIG. 5, including:
  • the memory 501 is used to store computer programs
  • the processor 502 is configured to implement the following steps when executing the program stored in the memory 501:
  • a pose matching the local semantic information is determined in the preset area as the pose of the vehicle to be located.
  • performing semantic segmentation on the bird's-eye view to obtain local semantic information of the scene where the vehicle to be located is located includes:
  • the semantic point cloud is used to represent the geometric information and semantic information of each spatial point, and the semantic information represents the corresponding spatial point.
  • the pose that matches the local semantic information is determined in the preset area as the pose of the vehicle to be located, including:
  • the local semantic information is matched with the global semantic information of the preset area, and the pose matching the local semantic information in the preset area is obtained as the position of the vehicle to be located.
  • the global semantic information is the semantic point cloud of the preset area.
  • the identification type includes: lane line, parking space frame, parking line, speed bump, road arrow, and parking space number.
  • the surround view camera includes a plurality of fish-eye cameras, and the plurality of fish-eye cameras are respectively set in different directions of the vehicle to be positioned for shooting images in corresponding directions;
  • the method further includes:
  • the images captured by multiple fisheye cameras are transformed and spliced to obtain a bird's-eye view of the scene where the vehicle to be located is located.
  • the vehicle to be located is further provided with a pose sensor, which is used to measure the relative pose of the vehicle to be located at different time nodes;
  • Semantic segmentation of the bird's-eye view taken at multiple time nodes including the current time to obtain semantic information of the scene where the vehicle to be located is located at multiple time nodes;
  • the superposition result is filtered to obtain the semantic information of the scene where the vehicle to be located is located in the time window including the current moment, as the local semantic information.
  • the method is used to obtain semantic information of the scene where the vehicle to be located is located at multiple time nodes. Also includes:
  • the semantic information of the scene where the vehicle to be located is located at multiple time nodes is superimposed to obtain the superimposed result, including:
  • the semantic information of the scene where the vehicle to be located is located at multiple time nodes is superimposed to obtain the superimposed result.
  • a pose matching the local semantic information is determined in the preset area as the vehicle to be located.
  • Posture including:
  • the current location of the vehicle to be located is estimated as the estimated location
  • a pose matching the local semantic information is determined within the preset range of the preset location as the pose of the vehicle to be located.
  • the memory mentioned in the above electronic device may include random access memory (Random Access Memory, RAM), and may also include non-volatile memory (Non-Volatile Memory, NVM), such as at least one disk storage.
  • RAM Random Access Memory
  • NVM Non-Volatile Memory
  • the memory may also be at least one storage device located far away from the foregoing processor.
  • the above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (Network Processor, NP), etc.; it can also be a digital signal processor (DSP), a dedicated Circuit (Application Specific Integrated Circuit, ASIC), Field-Programmable Gate Array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.
  • CPU central processing unit
  • NP Network Processor
  • DSP digital signal processor
  • ASIC Application Specific Integrated Circuit
  • FPGA Field-Programmable Gate Array
  • a computer-readable storage medium stores instructions that, when run on a computer, cause the computer to execute any of the above embodiments. Method for determining vehicle pose.
  • a computer program product containing instructions is also provided, which when running on a computer, causes the computer to execute any vehicle pose determination method in the foregoing embodiments.
  • the computer program product includes one or more computer instructions.
  • the computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.
  • the computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website, computer, server, or data center.
  • the computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server or a data center integrated with one or more available media.
  • the usable medium may be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid state disk (SSD)).
  • An embodiment of the present application provides a vehicle pose surround view system, including: a surround view camera and at least one processor, the surround view camera is used to shoot a bird's eye view of the scene where the vehicle to be located is located, and the processor is used to: Perform semantic segmentation on the bird's-eye view to obtain local semantic information of the scene where the vehicle to be located is located; according to the global semantic information of a preset area including the scene where the vehicle to be located is located, in the preset area The pose matching the local semantic information is determined as the pose of the vehicle to be located.
  • the system may also include a display for displaying a bird's-eye view of the scene where the vehicle to be located is located, and or, a map.
  • the map displayed on the display may include at least one of the following information: semantic information, the pose of the vehicle to be located, and the image of the vehicle where the camera is located.
  • the bird's-eye view may also include at least one of the following information: semantic information, the pose of the vehicle to be located, and the image of the vehicle where the camera is located.
  • the vehicle pose can be determined more accurately, which further makes map navigation more accurate, enables more accurate positioning and navigation, and realizes assisted driving.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Computational Linguistics (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)
  • Traffic Control Systems (AREA)

Abstract

本申请实施例提供了一种车辆位姿确定方法、装置及电子设备。其中，待定位车辆设置有环视相机，所述环视相机用于拍摄所述待定位车辆所处场景的鸟瞰图，所述方法包括：对所述鸟瞰图进行语义分割，得到所述待定位车辆所处场景的局部语义信息；根据包括所述待定位车辆所处场景在内的预设区域的全局语义信息，在所述预设区域内确定与所述局部语义信息匹配的位姿，作为所述待定位车辆所处的位姿。可以通过从环视相机拍摄到的鸟瞰图中提取语义信息，以获取待定位车辆所处场景的特征，进而通过地图匹配，确定出待定位车辆的位姿。可以在不借助GPS信号的前提下，实现位姿的确定。因此即使在无法正常接收到GPS信号的场景中也能够准确确定位姿。

Description

一种车辆位姿确定方法、装置及电子设备
本申请要求于2019年6月20日提交中国专利局、申请号为201910537501.6发明名称为“一种车辆位姿确定方法、装置及电子设备”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及图像分析技术领域，特别是涉及一种车辆位姿确定方法、装置及电子设备。
背景技术
在一些应用场景中，可能需要对车辆进行高精度、高频率的位姿(位置和朝向角)确定，例如无人驾驶系统中出于安全性的考虑，需要对车辆进行高精度、高频率的位姿确定。
相关技术中，可以通过在车辆上设置GPS(Global Position System，全球定位系统)传感器，以确定车辆的位姿。但是，GPS传感器需要接收GPS基站发送的GPS信号才能够正常工作。而在一些应用场景中车辆可能处于信号较差的场景中，如地下停车场，GPS传感器可能无法正常接收到GPS信号，导致无法对车辆进行位姿确定。
发明内容
本申请实施例的目的在于提供一种车辆位姿确定方法、装置及电子设备，以实现在无法正常接收到GPS信号的场景中，准确确定位姿。具体技术方案如下：
在本申请实施例的第一方面，提供了一种车辆位姿确定方法，待定位车辆设置有环视相机，所述方法包括：
对所述鸟瞰图进行语义分割，得到所述待定位车辆所处场景的局部语义信息；
根据包括所述待定位车辆所处场景在内的预设区域的全局语义信息，在所述预设区域内确定与所述局部语义信息匹配的位姿，作为所述待定位车辆 所处的位姿。
在一种可能的实施例中，所述对所述鸟瞰图进行语义分割，得到所述待定位车辆所处场景的局部语义信息，包括：
对所述鸟瞰图进行语义分割，得到所述待定位车辆所处场景的语义点云，作为局部语义信息，其中，语义点云用于表示各空间点的几何信息和语义信息，所述语义信息表示各空间点对应的标识类型；
所述根据包括所述待定位车辆所处场景在内的预设区域的全局语义信息，在所述预设区域内确定与所述局部语义信息匹配的位姿，作为所述待定位车辆所处的位姿，包括：
基于所述局部语义信息所表示的几何信息以及语义信息，将所述局部语义信息与预设区域的全局语义信息进行匹配，得到所述预设区域内中与所述局部语义信息匹配的位姿，作为所述待定位车辆所处的位姿，所述全局语义信息为所述预设区域的语义点云。
在一种可能的实施例中，所述标识类型包括：车道线、车位框、停车线、减速带、道路箭头、车位编号。
在一种可能的实施例中，所述环视相机包括多个鱼眼相机，所述多个鱼眼相机分别设置在待定位车辆的不同方位，用于拍摄所对应方向上的图像；
在所述对所述鸟瞰图进行语义分割，得到所述待定位车辆所处场景的局部语义信息之前，所述方法还包括：
根据逆透视变换原理，对所述多个鱼眼相机拍摄到的图像进行变换并拼接，得到所述待定位车辆所处场景的鸟瞰图。
在一种可能的实施例中，所述待定位车辆还设置有位姿传感器，用于测量所述待定位车辆在不同时间节点上的相对位姿；
所述对所述鸟瞰图进行语义分割，得到所述待定位车辆所处场景的局部语义信息，包括：
对包括当前时刻在内的多个时间节点上拍摄到的鸟瞰图进行语义分割，得到所述待定位车辆在所述多个时间节点上所处场景的语义信息；
获取所述多个时间节点上的感应相对位姿，所述感应相对位姿为读取所述位姿传感器得到的；
基于所述待定位车辆在所述多个时间节点上的感应相对位姿，对所述多个时间节点上所述待定位车辆所处场景的语义信息进行叠加，得到叠加结果；
对所述叠加结果进行滤波，得到所述待定位车辆在包括当前时刻在内的时间窗口内所处场景的语义信息，作为局部语义信息。
在本申请实施例的第二方面，提供了一种车辆位姿确定装置，待定位车辆设置有环视相机，所述环视相机用于拍摄所述待定位车辆所处场景的鸟瞰图，所述装置包括：
语义分割模块，用于对所述鸟瞰图进行语义分割，得到所述待定位车辆所处场景的局部语义信息；
语义匹配模块，用于根据包括所述待定位车辆所处场景在内的预设区域的全局语义信息，在所述预设区域内确定与所述局部语义信息匹配的位姿，作为所述待定位车辆所处的位姿。
在一种可能的实现方式中，所述语义分割模块，具体用于对所述鸟瞰图进行语义分割，得到所述待定位车辆所处场景的语义点云，作为局部语义信息，其中，语义点云用于表示各空间点的几何信息和语义信息，所述语义信息表示各空间点对应的标识类型；
所述语义匹配模块，具体用于基于所述局部语义信息所表示的几何信息以及语义信息，将所述局部语义信息与预设区域的全局语义信息进行匹配，得到所述预设区域内中与所述局部语义信息匹配的位姿，作为所述待定位车辆所处的位姿，所述全局语义信息为所述预设区域的语义点云。
在一种可能的实现方式中，所述语义分割模块，具体用于对所述鸟瞰图进行语义分割，得到所述待定位车辆所处场景的语义点云，作为局部语义信息，其中，语义点云用于表示各空间点的几何信息和语义信息，所述语义信息表示各空间点对应的标识类型；
所述语义匹配模块，具体用于基于所述局部语义信息所表示的几何信息 以及语义信息，将所述局部语义信息与预设区域的全局语义信息进行匹配，得到所述预设区域内中与所述局部语义信息匹配的位姿，作为所述待定位车辆所处的位姿，所述全局语义信息为所述预设区域的语义点云。
在一种可能的实现方式中，所述标识类型包括：车道线、车位框、停车线、减速带、道路箭头、车位编号。
在一种可能的实现方式中，所述环视相机包括多个鱼眼相机，所述多个鱼眼相机分别设置在待定位车辆的不同方位，用于拍摄所对应方向上的图像；
所述装置还包括图像拼接模块，用于在所述对所述鸟瞰图进行语义分割，得到所述待定位车辆所处场景的局部语义信息之前，所述方法还包括：
根据逆透视变换原理，对所述多个鱼眼相机拍摄到的图像进行变换并拼接，得到所述待定位车辆所处场景的鸟瞰图。
在一种可能的实现方式中，所述待定位车辆还设置有位姿传感器，用于测量所述待定位车辆在不同时间节点上的相对位姿；
所述语义分割模块，具体用于对包括当前时刻在内的多个时间节点上拍摄到的鸟瞰图进行语义分割，得到所述待定位车辆在所述多个时间节点上所处场景的语义信息；
获取所述多个时间节点上的感应相对位姿，所述感应相对位姿为读取所述位姿传感器得到的；
基于所述待定位车辆在所述多个时间节点上的感应相对位姿，对所述多个时间节点上所述待定位车辆所处场景的语义信息进行叠加，得到叠加结果；
对所述叠加结果进行滤波，得到所述待定位车辆在包括当前时刻在内的时间窗口内所处场景的语义信息，作为局部语义信息。
在一种可能的实现方式中，所述语义分割模块还用于在所述对包括当前时刻在内的多个时间节点上拍摄到的鸟瞰图进行语义分割，得到所述待定位车辆在所述多个时间节点上所处场景的语义信息之后，获取所述多个时间节点上的视觉相对位姿，所述视觉相对位姿为通过所述多个时间节点的语义信息匹配得到的；
融合所述感应相对位姿和所述视觉相对位姿，得到融合相对位姿；
所述语义分割模块，具体用于基于所述待定位车辆在所述多个时间节点上的融合相对位姿，对所述多个时间节点上所述待定位车辆所处场景的语义信息进行叠加，得到叠加结果。
在一种可能的实现方式中，所述语义匹配模块，具体用于根据确定得到的所述待定位车辆历史时刻所处的位置，预估所述待定位车辆当前时刻所处的位置，作为预估位置；
根据预设区域中所述预估位置的预设范围内的先验语义信息，在所述预设位置的预设范围内确定与所述局部语义信息匹配的位姿，作为所述待定位车辆所处的位姿。
在本申请实施例的第三方面，提供了一种电子设备，包括：
存储器，用于存放计算机程序；
处理器，用于执行存储器上所存放的程序时，实现上述第一方面任一所述的方法步骤。
在本申请实施例的第四方面，提供了一种计算机可读存储介质，所述计算机可读存储介质内存储有计算机程序，所述计算机程序被处理器执行时实现上述第一方面任一所述的方法步骤。
在本申请实施例的第五方面，提供了一种车辆位姿环视系统，包括：环视相机和至少一个处理器，所述环视相机用于拍摄所述待定位车辆所处场景的鸟瞰图，所述处理器用于：对所述鸟瞰图进行语义分割，得到所述待定位车辆所处场景的局部语义信息；根据包括所述待定位车辆所处场景在内的预设区域的全局语义信息，在所述预设区域内确定与所述局部语义信息匹配的位姿，作为所述待定位车辆所处的位姿。
本申请实施例提供的车辆位姿确定方法、装置及电子设备，可以通过从环视相机拍摄到的鸟瞰图中提取语义信息，以获取待定位车辆所处场景的特征，进而通过地图匹配，确定出待定位车辆的位姿。可以在不借助GPS信号的前提下，实现位姿的确定。因此即使在无法正常接收到GPS信号的场景中 也能够准确确定位姿。当然，实施本申请的任一产品或方法并不一定需要同时达到以上所述的所有优点。
附图说明
为了更清楚地说明本申请实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1为本申请实施例提供的车辆位姿确定方法的一种流程示意图；
图2为本申请实施例提供的车辆位姿确定方法的另一种流程示意图；
图3为本申请实施例提供的车辆位姿确定的一种原理示意图；
图4为本申请实施例提供的车辆位姿确定装置的一种结构示意图；
图5为本申请实施例提供的用于车辆位姿确定的电子设备的一种结构示意图。
具体实施方式
下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
在本申请实施例中，待定位车辆设置有环视相机，用于拍摄待定位车辆所处场景的鸟瞰图。环视相机的设置方式，根据应用场景的不同可以不同，本实施例对此不做限制。参见图1，图1所示为本申请实施例提供的车辆位姿确定方法的一种流程示意图，可以包括：
S101，对鸟瞰图进行语义分割，得到待定位车辆所处场景的局部语义信息。
局部语义信息可以用于表示待定位车辆所处场景中标识的标识类型以及位置信息。示例性的，局部语义信息可以表示待定位车辆所处场景中，空间坐标为(1,2,3)的空间点属于停车线。局部语义信息的表示方式根据应用场 景的不同可以不同，本实施例对此不做限制。
在一种可能的实施例中，局部语义信息可以是以语义点云的形式表示的。其中，语义点云用于表示各空间点的几何信息和语义信息，语义信息表示各空间点对应的标识类型。根据应用场景不同，标识类型可以不同。示例性的，在一种可能的应用场景中，标识类型可以包括车道线、车位框、停车线、减速带、道路箭头、车位编号等地面标识中的一个或多个。语义点云可以视为多个点的集合，每个点可以用(loc，label)的形式表示，其中loc表示该点的空间坐标，label表示该点对应的标识类型所对应的标签，例如存在一个空间坐标为(3，2，0)的点，并且该点属于停车线，假设停车线所对应的标签为2，则语义点云中该点可以表示为(3，2，0，2)。
S102，根据待定位车辆所处场景在内的预设区域的全局语义信息，在预设区域内确定与局部语义信息匹配的位姿，作为待定位车辆所处的位姿。
全局语义信息的表示方式与局部语义信息的表示方式相同，示例性的，假设局部语义信息是以语义点云的形式表示的，则全局语义信息也是以语义点云的形式表示的。
位姿与局部语义信息匹配，可以是指基于全局语义信息，假设待定位车辆处于该位姿时，理论上可以测量得到的局部语义信息与S101中实际测量得到的局部语义信息能够匹配。
根据语义信息表示方式的不同，确定与局部语义信心匹配的位姿的方式也可以不同。可以是基于局部语义信息所表示的几何信息以及语义信息，将局部语义信息与预设区域的全局语义信息匹配，得到预设区域内与局部语义信息匹配的位姿，作为待定位车辆所处的位姿。为描述方便，下面将以全局语义信息和局部语义信息是以语义点云的形式表示的为例进行说明，可以包括以下步骤：
步骤1、基于局部语义信息所表示的几何信息以及语义信息，将局部语义信息与预设的全局语义信息进行匹配，得到局部语义信息与全局语义信息的匹配点对。
如果局部语义信息中的一个点，与全局语义信息中的一个点的几何信息 相匹配，并且语义信息也相匹配，则可以认为这两个点对应于同一个空间点，即这两个点构成关联关系。
在其他可选的实施例中，也可以是根据确定得到的待定位车辆历史时刻所处的位置，预估待定位车辆当前时刻所处的位置，作为预估位置。示例性的，假设待定位车辆在t＝10s时，处于位置A，当前时刻为t＝15s，可以从位姿传感器读取这5s内的相对位姿变化，得到预估位置A`。进而从全局语义信息中，截取位置A`附近预设范围内的语义信息，作为先验语义信息，并且通过将先验语义信息与局部语义信息进行匹配，得到局部语义信息与先验语义信息的匹配点对，可以理解，由于先验语义信息为全局语义信息的一部分，因此局部语义信息与先验语义信息的匹配点对，可以视为局部语义信息与全局语义信息的匹配点对。同时，先验语义信息为全局语义信息的一部分，因此完成局部语义信息与先验语义信息的匹配所需要的计算量，低于完成局部语义信息与全局语义信息的匹配所需要的计算量。
步骤2、基于匹配点对的坐标转换关系，确定待定位车辆在全局语义信息的空间坐标系中的位姿。
为描述方便，假设全局语义信息的空间坐标系为全局坐标系，而局部语义信息的空间坐标系为局部坐标系。可以理解的是，一个匹配点对中包括一个全局语义信息中的点，以及局部语义信息中相匹配的点，如前述分析，这两个点理论上表示空间中的同一个点，只是分别以全局坐标系中的坐标和局部坐标系中的坐标进行表示。因此，综合多个匹配点对，可以确定出全局坐标系与局部坐标系之间的坐标变换关系。
待定位车辆与环视相机之间的相对位置关系可以认为是固定并且已知的，而局部语义信息为基于环视相机拍摄到的鸟瞰图所构建的待定位车辆所处场景的语义点云，因此可以认为待定位车辆在局部坐标系中的局部坐标和朝向角是已知的。进而可以基于匹配点对所反映出的全局坐标系与局部坐标系之间的坐标转换关系，确定待定位车辆在全局坐标系中的全局坐标以及朝向角，即可以确定得到待定位车辆的位姿。
选用该实施例，可以通过从环视相机拍摄到的鸟瞰图中提取语义信息，以获取待定位车辆所处场景的特征，进而通过地图匹配，确定出待定位车辆 的位姿。可以在不借助GPS信号的前提下，实现位姿的确定。因此即使在无法正常接收到GPS信号的场景中也能够确定位姿。
在一种可能的实施例中，可以在车辆上设置激光定位器和/或声波定位器，通过激光定位器和/或声波定位器进行位姿的确定。但是，激光定位器和声波定位器的成本往往较高。而选用本申请实施例提供的车辆位姿确定方法，可以在不借助激光定位器和声波定位器的前提下，实现位姿的确定，并且由于拍摄鸟瞰图所需要的图像采集设备的成本往往低于激光定位器和声波定位器的成本，因此相比于相关技术中，利用激光定位和/或声波定位器进行位姿确定的方案，可以有效降低位姿确定的成本。
参见图2，图2所示为本申请实施例提供的车辆位姿确定方法的另一种流程示意图，其中，待定位车辆设置有位姿传感器和环视相机，位姿传感器用于测量待定位车辆在不同时间节点上的相对位姿。方法可以包括：
S201，对包括当前时刻在内的多个时间节点上拍摄到的鸟瞰图进行语义分割，得到待定位车辆在多个时间节点上所处场景的语义点云。
为描述方便，假设当前时刻为t1,，并且多个时间节点为{t1，t2…，tn}，并且将在ti上拍摄到的鸟瞰图称为鸟瞰图i，待定位车辆在ti上所处场景的语义点云称为语义点云i，其中，i为[1,n]中的任意正整数。
对不同的鸟瞰图可以是在不同时间分别进行语义分割的，也可以是在相同时间并行语义分割的，示例性的，可以是在每拍摄到一个鸟瞰图后，对该拍摄到的鸟瞰图进行语义分割，也可以是在拍摄到鸟瞰图1-鸟瞰图n后，并行对鸟瞰图1-鸟瞰图n进行语义分割，本实施例对此不做限制。
S202，获取位姿传感器在多个时间节点上测量得到的待定位车辆的相对位姿，作为感应相对位姿。
位姿传感器可以包括IMU(Inertial Measurement Unit，惯性测量单元)和轮速计数器。其中，IMU用于测量车辆三轴加速度和角速度，轮速计数器用于测量车辆轮胎的旋转圈数，并且基于测量得到的旋转圈数，计算车辆的运动距离。待定位车辆在一个时间节点上的相对位姿，可以是待定位车辆在该一个时间节点上所处的位姿，相对于该时间节点之前的某一时间节点上待定 位车辆所处位姿的变化量。在已知车辆三轴加速度、角速度以及车辆运动距离的情况下，可以计算出车辆在指定时间窗口内运动的变化量，因此位姿传感器可以测量得到待定位车辆的相对位姿。
可以理解的是，在一些实施例中，位姿传感器和环视相机的采样频率可能不同，因此可以设置时间同步单元，以同步环视相机采集到的鸟瞰图和位姿传感器测量得到的相对位姿。
S203，基于待定位车辆在多个时间节点上的感应相对位姿，对多个时间节点上待定位车辆所处场景的语义点云进行叠加，得到叠加结果。
为描述方便，将待定位车辆在ti上的感应相对位姿称为loci，则可以是将语义点云1设置在loc1，将语义点云2叠加的设置在loc2…依次类推，直至将语义点云n叠加的设置在locn，以得到叠加结果。
可以理解的是，位姿传感器可能存在一定误差，以位姿传感器包括IMU和轮速计数器为例，IMU和轮速计数器可能存在累计误差和随机涨落，导致测量得到的三轴加速度、角速度以及车辆运动距离不够准确，基于不准确的三轴加速度、角速度以及车辆运动距离，得到的相对位姿不够准确，即感应相对位姿可能存在一定误差。
有鉴于此，在一种可选的实施例中，可以对待定位车辆在多个时间节点上所处场景的语义点云进行匹配，得到待定位车辆在多个时间节点上的相对位姿，作为视觉相对位姿。并融合感应相对位姿与视觉相对位姿，得到融合相对位姿。
由于融合位姿中综合了基于语义点云匹配得到的视觉相对位姿，因此可以将融合相对位姿视为基于视觉相对位姿，对感应相对位姿进行修正得到的修正结果。因此融合得到的融合相对位姿相比于感应相对位姿更加准确。因此基于融合相对位姿，对多个时间节点上待定位车辆所处场景的语义点云进行叠加，可以使得得到的叠加结果更加准确。
S204，对叠加结果进行滤波，得到待定位车辆在包括当前时刻在内的时间窗口内所处场景的语义点云，作为局部语义信息。
滤波所使用的算法根据应用场景的不同可以不同，本实施例对此不做限 制。由于叠加结果为包括当前时刻在内的多个时间节点上待定位车辆所处场景的语义点云，因此经过滤波得到的结果，可以视为待定位车辆在包括当前时刻在内的时间窗口内所处场景的语义点云，也可以视为待定位车辆最近运动的一段距离内的场景的语义点云。
S205，根据包括待定位车辆所处场景在内的预设区域的全局语义信息，在预设区域内确定与局部语义信息匹配的位姿，作为待定位车辆所处的位姿。
该步骤与S102相同，可以参见前述S102的相关描述，在此不再赘述。
可以理解的是，待定位车辆在当前时刻所处场景的语义点云中所包括的几何信息以及语义信息往往有限，可能导致全局语义信息中存在多个位姿与局部语义信息匹配，即可能导致局部语义无法准确与全局语义信息匹配。而选用该实施例，可以通过叠加多个不同时间节点上所处场景的语义点云，增加局部语义信息中所包含的几何信息以及语义信息，降低全局语义信息中存在多个位姿与局部语义信息匹配的可能性，使得局部语义信息可以与全局语义信息更准确地进行匹配，从而提高确定得到的位姿的准确性。
下面将结合具体的应用场景的对本申请实施例提供的车辆位姿确定方法进行说明，在一种可能的实施例中，环视相机包括四个鱼眼相机(在其他可选的实施例中所包括的鱼眼相机的数目也可以不同)，分别设置在待定位车辆的四周。该待定位车辆还设置有IMU和轮速计数器。可以参见图3，包括：
S301，四个鱼眼相机拍摄对应方向上的图像，并将拍摄到的图像和时间戳发送至数据采集平台。
其中，数据采集平台可以是设置在该车辆上的，也可以是设置在与该车辆存在通信连接的网络设备上。鱼眼相机所发送的图像包含有用于表示拍摄时间的时间戳。
S302，数据采集平台根据逆透视变换原理，对多个鱼眼相机拍摄到的图像进行变换并拼接，得到待定位车辆所处场景的鸟瞰图。
例如，假设四个鱼眼相机分别在t＝10s时拍摄到对应方向上的图像，根据逆透视变换原理，对这四个鱼眼相机在t＝10s时拍摄到的对应方向的图像进行变化和拼接，可以得到t＝10s时待定位车辆所处场景的鸟瞰图。
S303，数据采集平台对鸟瞰图进行语义分割，得到待定位车辆所处场景的语义点云。
该步骤与S101相同，可以参见前述关于S101的描述，在此不再赘述。
S304，数据采集平台根据接收到的时间戳，同步读取各个图像所对应的IMU数据和轮速计数器数据。
其中，IMU数据为IMU测量得到的数据，轮速计数器数据为轮速计数器测量得到的数据。示例性的，假设一个图像的时间戳为t＝10s，则数据采集平台可以读取t＝10s时IMU的读数，作为IMU数据，并读取t＝10s时轮速计数器的读数，作为轮速计数器读数。
S305，数据采集平台将得到的IMU数据、轮速计数器数据、语义点云发送至数据分析平台。
其中，数据分析平台可以是设置于车辆上，也可以是设置于与车辆建立有通信连接的网络设备上。
S306，数据分析平台基于IMU数据、轮速计数器数据确定感应相对位姿，并基于不同时间节点的语义点云确定视觉相对位姿。
关于感应相对位姿和视觉相对位姿可以参见前述S202和S203中的相关描述，在此不再赘述。
S307，数据分析平台融合感应相对位姿和视觉相对位姿，得到融合相对位姿。
关于融合相对位姿可以参见前述S203中的相关描述，在此不再赘述。
S308，数据分析平台基于待定位车辆在多个时间节点上的融合相对位姿，对多个时间节点上待定位车辆所处场景的语义点云进行叠加，得到叠加结果。
S309，数据分析平台对叠加结果进行滤波，得到待定位车辆在包括当前时刻在内的时间窗口内所处场景的语义点云，作为局部语义信息。
S310，数据分析平台基于局域语义地图所表示的几何信息和语义信息，将局部语义信息与预设的全局语义信息进行匹配，得到局部语义信息与全局 语义信息的匹配点对。
可以参见前述S102中的相关描述，在此不再赘述。
S311，数据分析平台基于匹配点对的坐标变换关系，确定待定位车辆在全局语义信息的空间坐标系中的位姿。
可以参见前述S102中的相关描述，在此不再赘述。
该实施例中数据采集平台和数据分析平台可以是相互独立的两个实体设备，也可以是集成在同一个电子设备上的两个不同虚拟设备，本实施例对此不做限制。在一种可选的实施例中，还可以包括数据显示平台，用于根据数据分析平台所确定的位姿显示该车辆。
参见图4，图4所示为本申请实施例提供的车辆位姿确定装置的一种结构示意图，可以包括：
语义分割模块401，用于对鸟瞰图进行语义分割，得到待定位车辆所处场景的局部语义信息；
语义匹配模块402，用于根据包括待定位车辆所处场景在内的预设区域的全局语义信息，在预设区域内确定与局部语义信息匹配的位姿，作为待定位车辆所处的位姿。
在一种可选的实施例中，语义分割模块401，具体用于对鸟瞰图进行语义分割，得到待定位车辆所处场景的语义点云，作为局部语义信息，其中，语义点云用于表示各空间点的几何信息和语义信息，语义信息表示各空间点对应的标识类型；
语义匹配模块402，具体用于基于局部语义信息所表示的几何信息以及语义信息，将局部语义信息与预设区域的全局语义信息进行匹配，得到预设区域内中与局部语义信息匹配的位姿，作为待定位车辆所处的位姿，全局语义信息为预设区域的语义点云。
在一种可选的实施例中，标识类型包括：车道线、车位框、停车线、减速带、道路箭头、车位编号。
在一种可选的实施例中，环视相机包括多个鱼眼相机，多个鱼眼相机分 别设置在待定位车辆的不同方位，用于拍摄所对应方向上的图像；
装置还包括图像拼接模块，用于在对鸟瞰图进行语义分割，得到待定位车辆所处场景的局部语义信息之前，方法还包括：
根据逆透视变换原理，对多个鱼眼相机拍摄到的图像进行变换并拼接，得到待定位车辆所处场景的鸟瞰图。
在一种可选的实施例中，待定位车辆还设置有位姿传感器，用于测量待定位车辆在不同时间节点上的相对位姿；
语义分割模块401，具体用于对包括当前时刻在内的多个时间节点上拍摄到的鸟瞰图进行语义分割，得到待定位车辆在多个时间节点上所处场景的语义信息；
获取多个时间节点上的感应相对位姿，感应相对位姿为读取位姿传感器得到的；
基于待定位车辆在多个时间节点上的感应相对位姿，对多个时间节点上待定位车辆所处场景的语义信息进行叠加，得到叠加结果；
对叠加结果进行滤波，得到待定位车辆在包括当前时刻在内的时间窗口内所处场景的语义信息，作为局部语义信息。
在一种可能的实施例中，语义分割模块401还用于在对包括当前时刻在内的多个时间节点上拍摄到的鸟瞰图进行语义分割，得到待定位车辆在多个时间节点上所处场景的语义信息之后，获取多个时间节点上的视觉相对位姿，视觉相对位姿为通过多个时间节点的语义信息匹配得到的；
融合感应相对位姿和视觉相对位姿，得到融合相对位姿；
语义分割模块401，具体用于基于待定位车辆在多个时间节点上的融合相对位姿，对多个时间节点上待定位车辆所处场景的语义信息进行叠加，得到叠加结果。
在一种可能的实施例中，语义匹配模块402，具体用于根据确定得到的待定位车辆历史时刻所处的位置，预估待定位车辆当前时刻所处的位置，作为预估位置；
根据预设区域中预估位置的预设范围内的先验语义信息，在预设位置的预设范围内确定与局部语义信息匹配的位姿，作为待定位车辆所处的位姿。
本申请实施例还提供了一种电子设备，如图5所示，包括：
存储器501，用于存放计算机程序；
处理器502，用于执行存储器501上所存放的程序时，实现如下步骤：
对鸟瞰图进行语义分割，得到待定位车辆所处场景的局部语义信息；
根据包括待定位车辆所处场景在内的预设区域的全局语义信息，在预设区域内确定与局部语义信息匹配的位姿，作为待定位车辆所处的位姿。
在一种可选的实施例中，对鸟瞰图进行语义分割，得到待定位车辆所处场景的局部语义信息，包括：
对鸟瞰图进行语义分割，得到待定位车辆所处场景的语义点云，作为局部语义信息，其中，语义点云用于表示各空间点的几何信息和语义信息，语义信息表示各空间点对应的标识类型；
根据包括待定位车辆所处场景在内的预设区域的全局语义信息，在预设区域内确定与局部语义信息匹配的位姿，作为待定位车辆所处的位姿，包括：
基于局部语义信息所表示的几何信息以及语义信息，将局部语义信息与预设区域的全局语义信息进行匹配，得到预设区域内与局部语义信息匹配的位姿，作为待定位车辆所处的位姿，全局语义信息为预设区域的语义点云。
在一种可选的实施例中，标识类型包括：车道线、车位框、停车线、减速带、道路箭头、车位编号。
在一种可选的实施例中，环视相机包括多个鱼眼相机，多个鱼眼相机分别设置在待定位车辆的不同方位，用于拍摄所对应方向上的图像；
在对鸟瞰图进行语义分割，得到待定位车辆所处场景的局部语义信息之前，方法还包括：
根据逆透视变换原理，对多个鱼眼相机拍摄到的图像进行变换并拼接，得到待定位车辆所处场景的鸟瞰图。
在一种可选的实施例中，待定位车辆还设置有位姿传感器，用于测量待定位车辆在不同时间节点上的相对位姿；
对鸟瞰图进行语义分割，得到待定位车辆所处场景的局部语义信息，包括：
对包括当前时刻在内的多个时间节点上拍摄到的鸟瞰图进行语义分割，得到待定位车辆在多个时间节点上所处场景的语义信息；
获取多个时间节点上的感应相对位姿，感应相对位姿为读取位姿传感器得到的；
基于待定位车辆在多个时间节点上的感应相对位姿，对多个时间节点上待定位车辆所处场景的语义信息进行叠加，得到叠加结果；
对叠加结果进行滤波，得到待定位车辆在包括当前时刻在内的时间窗口内所处场景的语义信息，作为局部语义信息。
在一种可能的实施例中，在对包括当前时刻在内的多个时间节点上拍摄到的鸟瞰图进行语义分割，得到待定位车辆在多个时间节点上所处场景的语义信息之后，方法还包括：
获取多个时间节点上的视觉相对位姿，视觉相对位姿为通过多个时间节点的语义信息匹配得到的；
融合感应相对位姿和视觉相对位姿，得到融合相对位姿；
基于待定位车辆在多个时间节点上的感应相对位姿，对多个时间节点上待定位车辆所处场景的语义信息进行叠加，得到叠加结果，包括：
基于待定位车辆在多个时间节点上的融合相对位姿，对多个时间节点上待定位车辆所处场景的语义信息进行叠加，得到叠加结果。
在一种可能的实施例中，根据包括待定位车辆所处场景在内的预设区域的全局语义信息，在预设区域内确定与局部语义信息匹配的位姿，作为待定位车辆所处的位姿，包括：
根据确定得到的待定位车辆历史时刻所处的位置，预估待定位车辆当前 时刻所处的位置，作为预估位置；
根据预设区域中预估位置的预设范围内的先验语义信息，在预设位置的预设范围内确定与局部语义信息匹配的位姿，作为待定位车辆所处的位姿。
上述电子设备提到的存储器可以包括随机存取存储器(Random Access Memory，RAM)，也可以包括非易失性存储器(Non-Volatile Memory，NVM)，例如至少一个磁盘存储器。可选的，存储器还可以是至少一个位于远离前述处理器的存储装置。
上述的处理器可以是通用处理器，包括中央处理器(Central Processing Unit，CPU)、网络处理器(Network Processor，NP)等；还可以是数字信号处理器(Digital Signal Processing，DSP)、专用集成电路(Application Specific Integrated Circuit，ASIC)、现场可编程门阵列(Field-Programmable Gate Array，FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。
在本申请提供的又一实施例中，还提供了一种计算机可读存储介质，该计算机可读存储介质中存储有指令，当其在计算机上运行时，使得计算机执行上述实施例中任一车辆位姿确定方法。
在本申请提供的又一实施例中，还提供了一种包含指令的计算机程序产品，当其在计算机上运行时，使得计算机执行上述实施例中任一车辆位姿确定方法。
在上述实施例中，可以全部或部分地通过软件、硬件、固件或者其任意组合来实现。当使用软件实现时，可以全部或部分地以计算机程序产品的形式实现。所述计算机程序产品包括一个或多个计算机指令。在计算机上加载和执行所述计算机程序指令时，全部或部分地产生按照本申请实施例所述的流程或功能。所述计算机可以是通用计算机、专用计算机、计算机网络、或者其他可编程装置。所述计算机指令可以存储在计算机可读存储介质中，或者从一个计算机可读存储介质向另一个计算机可读存储介质传输，例如，所述计算机指令可以从一个网站站点、计算机、服务器或数据中心通过有线(例如同轴电缆、光纤、数字用户线(DSL))或无线(例如红外、无线、微波等) 方式向另一个网站站点、计算机、服务器或数据中心进行传输。所述计算机可读存储介质可以是计算机能够存取的任何可用介质或者是包含一个或多个可用介质集成的服务器、数据中心等数据存储设备。所述可用介质可以是磁性介质，(例如，软盘、硬盘、磁带)、光介质(例如，DVD)、或者半导体介质(例如固态硬盘Solid State Disk(SSD))等。
本申请实施例提供了一种车辆位姿环视系统，包括：环视相机和至少一个处理器，所述环视相机用于拍摄所述待定位车辆所处场景的鸟瞰图，所述处理器用于：对所述鸟瞰图进行语义分割，得到所述待定位车辆所处场景的局部语义信息；根据包括所述待定位车辆所处场景在内的预设区域的全局语义信息，在所述预设区域内确定与所述局部语义信息匹配的位姿，作为所述待定位车辆所处的位姿。
该系统还可以包括显示器，用于显示所述待定位车辆所处场景的鸟瞰图，和或，地图。
显示器显示的地图可以包含如下信息的至少一个：语义信息，待定位车辆的位姿，环视相机所在的车辆的图像。
鸟瞰图也可以包含如下信息的至少一个：语义信息，待定位车辆的位姿，环视相机所在的车辆的图像。
利用本申请实施例在地下室、室内、多层建筑、GPS信号弱的场景中，可以更准确的确定车辆位姿，进一步使得地图导航更为精准，能够更精准的定位、导航，实现辅助驾驶。
需要说明的是，在本文中，诸如第一和第二等之类的关系术语仅仅用来将一个实体或者操作与另一个实体或操作区分开来，而不一定要求或者暗示这些实体或操作之间存在任何这种实际的关系或者顺序。而且，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者设备不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者设备所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括所述要素的过程、方法、物品或者设备中还存在另外的相同要素。
本说明书中的各个实施例均采用相关的方式描述，各个实施例之间相同相似的部分互相参见即可，每个实施例重点说明的都是与其他实施例的不同之处。尤其，对于装置、电子设备、系统、计算机可读存储介质以及计算机程序产品的实施例而言，由于其基本相似于方法实施例，所以描述的比较简单，相关之处参见方法实施例的部分说明即可。
以上所述仅为本申请的较佳实施例而已，并不用以限制本申请，凡在本申请的精神和原则之内，所做的任何修改、等同替换、改进等，均应包含在本申请保护的范围之内。

Claims (17)

  1. 一种车辆位姿确定方法，其特征在于，待定位车辆设置有环视相机，所述环视相机用于拍摄所述待定位车辆所处场景的鸟瞰图，所述方法包括：
    对所述鸟瞰图进行语义分割，得到所述待定位车辆所处场景的局部语义信息；
    根据包括所述待定位车辆所处场景在内的预设区域的全局语义信息，在所述预设区域内确定与所述局部语义信息匹配的位姿，作为所述待定位车辆所处的位姿。
  2. 根据权利要求1所述的方法，其特征在于，所述对所述鸟瞰图进行语义分割，得到所述待定位车辆所处场景的局部语义信息，包括：
    对所述鸟瞰图进行语义分割，得到所述待定位车辆所处场景的语义点云，作为局部语义信息，其中，语义点云用于表示各空间点的几何信息和语义信息，所述语义信息表示各空间点对应的标识类型；
    所述根据包括所述待定位车辆所处场景在内的预设区域的全局语义信息，在所述预设区域内确定与所述局部语义信息匹配的位姿，作为所述待定位车辆所处的位姿，包括：
    基于所述局部语义信息所表示的几何信息以及语义信息，将所述局部语义信息与预设区域的全局语义信息进行匹配，得到所述预设区域内与所述局部语义信息匹配的位姿，作为所述待定位车辆所处的位姿，所述全局语义信息为所述预设区域的语义点云。
  3. 根据权利要求2所述的方法，其特征在于，所述标识类型包括：车道线、车位框、停车线、减速带、道路箭头、车位编号。
  4. 根据权利要求1所述的方法，其特征在于，所述环视相机包括多个鱼眼相机，所述多个鱼眼相机分别设置在待定位车辆的不同方位，用于拍摄所对应方向上的图像；
    在所述对所述鸟瞰图进行语义分割，得到所述待定位车辆所处场景的局部语义信息之前，所述方法还包括：
    根据逆透视变换原理，对所述多个鱼眼相机拍摄到的图像进行变换并拼接，得到所述待定位车辆所处场景的鸟瞰图。
  5. 根据权利要求1所述的方法，其特征在于，所述待定位车辆还设置有位姿传感器，用于测量所述待定位车辆在不同时间节点上的相对位姿；
    所述对所述鸟瞰图进行语义分割，得到所述待定位车辆所处场景的局部语义信息，包括：
    对包括当前时刻在内的多个时间节点上拍摄到的鸟瞰图进行语义分割，得到所述待定位车辆在所述多个时间节点上所处场景的语义信息；
    获取所述多个时间节点上的感应相对位姿，所述感应相对位姿为读取所述位姿传感器得到的；
    基于所述待定位车辆在所述多个时间节点上的感应相对位姿，对所述多个时间节点上所述待定位车辆所处场景的语义信息进行叠加，得到叠加结果；
    对所述叠加结果进行滤波，得到所述待定位车辆在包括当前时刻在内的时间窗口内所处场景的语义信息，作为局部语义信息。
  6. 根据权利要求5所述的方法，其特征在于，在所述对包括当前时刻在内的多个时间节点上拍摄到的鸟瞰图进行语义分割，得到所述待定位车辆在所述多个时间节点上所处场景的语义信息之后，所述方法还包括：
    获取所述多个时间节点上的视觉相对位姿，所述视觉相对位姿为通过所述多个时间节点的语义信息匹配得到的；
    融合所述感应相对位姿和所述视觉相对位姿，得到融合相对位姿；
    所述基于所述待定位车辆在所述多个时间节点上的感应相对位姿，对所述多个时间节点上所述待定位车辆所处场景的语义信息进行叠加，得到叠加结果，包括：
    基于所述待定位车辆在所述多个时间节点上的融合相对位姿，对所述多个时间节点上所述待定位车辆所处场景的语义信息进行叠加，得到叠加结果。
  7. 根据权利要求1所述的方法，其特征在于，所述根据包括所述待定位 车辆所处场景在内的预设区域的全局语义信息，在所述预设区域内确定与所述局部语义信息匹配的位姿，作为所述待定位车辆所处的位姿，包括：
    根据确定得到的所述待定位车辆历史时刻所处的位置，预估所述待定位车辆当前时刻所处的位置，作为预估位置；
    根据预设区域中所述预估位置的预设范围内的先验语义信息，在所述预设位置的预设范围内确定与所述局部语义信息匹配的位姿，作为所述待定位车辆所处的位姿。
  8. 一种车辆位姿确定装置，其特征在于，待定位车辆设置有环视相机，所述环视相机用于拍摄所述待定位车辆所处场景的鸟瞰图，所述装置包括：
    语义分割模块，用于对所述鸟瞰图进行语义分割，得到所述待定位车辆所处场景的局部语义信息；
    语义匹配模块，用于根据包括所述待定位车辆所处场景在内的预设区域的全局语义信息，在所述预设区域内确定与所述局部语义信息匹配的位姿，作为所述待定位车辆所处的位姿。
  9. 根据权利要求8所述的装置，其特征在于，所述语义分割模块，具体用于对所述鸟瞰图进行语义分割，得到所述待定位车辆所处场景的语义点云，作为局部语义信息，其中，语义点云用于表示各空间点的几何信息和语义信息，所述语义信息表示各空间点对应的标识类型；
    所述语义匹配模块，具体用于基于所述局部语义信息所表示的几何信息以及语义信息，将所述局部语义信息与预设区域的全局语义信息进行匹配，得到所述预设区域内中与所述局部语义信息匹配的位姿，作为所述待定位车辆所处的位姿，所述全局语义信息为所述预设区域的语义点云。
  10. 根据权利要求9所述的装置，其特征在于，所述标识类型包括：车道线、车位框、停车线、减速带、道路箭头、车位编号。
  11. 根据权利要求8所述的装置，其特征在于，所述环视相机包括多个鱼眼相机，所述多个鱼眼相机分别设置在待定位车辆的不同方位，用于拍摄所对应方向上的图像；
    所述装置还包括图像拼接模块，用于在所述对所述鸟瞰图进行语义分割，得到所述待定位车辆所处场景的局部语义信息之前，所述方法还包括：
    根据逆透视变换原理，对所述多个鱼眼相机拍摄到的图像进行变换并拼接，得到所述待定位车辆所处场景的鸟瞰图。
  12. 根据权利要求8所述的装置，其特征在于，所述待定位车辆还设置有位姿传感器，用于测量所述待定位车辆在不同时间节点上的相对位姿；
    所述语义分割模块，具体用于对包括当前时刻在内的多个时间节点上拍摄到的鸟瞰图进行语义分割，得到所述待定位车辆在所述多个时间节点上所处场景的语义信息；
    获取所述多个时间节点上的感应相对位姿，所述感应相对位姿为读取所述位姿传感器得到的；
    基于所述待定位车辆在所述多个时间节点上的感应相对位姿，对所述多个时间节点上所述待定位车辆所处场景的语义信息进行叠加，得到叠加结果；
    对所述叠加结果进行滤波，得到所述待定位车辆在包括当前时刻在内的时间窗口内所处场景的语义信息，作为局部语义信息。
  13. 根据权利要求12所述的装置，其特征在于，所述语义分割模块还用于在所述对包括当前时刻在内的多个时间节点上拍摄到的鸟瞰图进行语义分割，得到所述待定位车辆在所述多个时间节点上所处场景的语义信息之后，获取所述多个时间节点上的视觉相对位姿，所述视觉相对位姿为通过所述多个时间节点的语义信息匹配得到的；
    融合所述感应相对位姿和所述视觉相对位姿，得到融合相对位姿；
    所述语义分割模块，具体用于基于所述待定位车辆在所述多个时间节点上的融合相对位姿，对所述多个时间节点上所述待定位车辆所处场景的语义信息进行叠加，得到叠加结果。
  14. 根据权利要求8所述的装置，其特征在于，所述语义匹配模块，具体用于根据确定得到的所述待定位车辆历史时刻所处的位置，预估所述待定位车辆当前时刻所处的位置，作为预估位置；
    根据预设区域中所述预估位置的预设范围内的先验语义信息，在所述预设位置的预设范围内确定与所述局部语义信息匹配的位姿，作为所述待定位车辆所处的位姿。
  15. 一种车辆位姿环视系统，其特征在于，包括：环视相机和至少一个处理器，所述环视相机用于拍摄所述待定位车辆所处场景的鸟瞰图，所述处理器用于：对所述鸟瞰图进行语义分割，得到所述待定位车辆所处场景的局部语义信息；根据包括所述待定位车辆所处场景在内的预设区域的全局语义信息，在所述预设区域内确定与所述局部语义信息匹配的位姿，作为所述待定位车辆所处的位姿。
  16. 一种电子设备，其特征在于，包括：
    存储器，用于存放计算机程序；
    处理器，用于执行存储器上所存放的程序时，实现权利要求1-7任一所述的方法步骤。
  17. 一种计算机可读存储介质，其特征在于，所述计算机可读存储介质内存储有计算机程序，所述计算机程序被处理器执行时实现权利要求1-7任一所述的方法步骤。
PCT/CN2020/097180 2019-06-20 2020-06-19 一种车辆位姿确定方法、装置及电子设备 WO2020253842A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
EP20826498.6A EP3989170A4 (en) 2019-06-20 2020-06-19 METHOD AND DEVICE FOR DETERMINING THE POSITION OF A VEHICLE, AS WELL AS DEVICE AND ELECTRONIC DEVICE

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910537501.6A CN112116654B (zh) 2019-06-20 一种车辆位姿确定方法、装置及电子设备
CN201910537501.6 2019-06-20

Publications (1)

Publication Number Publication Date
WO2020253842A1 true WO2020253842A1 (zh) 2020-12-24

Family

ID=73796634

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/097180 WO2020253842A1 (zh) 2019-06-20 2020-06-19 一种车辆位姿确定方法、装置及电子设备

Country Status (2)

Country Link
EP (1) EP3989170A4 (zh)
WO (1) WO2020253842A1 (zh)

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112966622A (zh) * 2021-03-15 2021-06-15 广州小鹏自动驾驶科技有限公司 一种停车场语义地图完善方法、装置、设备和介质
CN114648639A (zh) * 2022-05-19 2022-06-21 魔视智能科技(武汉)有限公司 一种目标车辆的检测方法、系统及装置
CN114913491A (zh) * 2021-02-08 2022-08-16 广州汽车集团股份有限公司 一种车辆定位方法及系统、计算机可读存储介质
CN115164906A (zh) * 2022-09-02 2022-10-11 深圳鹏行智能研究有限公司 定位方法、机器人和计算机可读存储介质
CN115923847A (zh) * 2023-03-15 2023-04-07 安徽蔚来智驾科技有限公司 自动驾驶车辆的感知信息的预处理方法、装置及车辆
CN116105603A (zh) * 2023-04-13 2023-05-12 安徽蔚来智驾科技有限公司 用于确定移动物体在场所中的位置的方法及系统
CN116358573A (zh) * 2023-05-31 2023-06-30 小米汽车科技有限公司 地图建立方法、装置、存储介质以及车辆

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106403964A (zh) * 2016-08-30 2017-02-15 北汽福田汽车股份有限公司 一种定位导航系统及车辆
CN106407315A (zh) * 2016-08-30 2017-02-15 长安大学 一种基于街景图像数据库的车辆自主定位方法
CN107144285A (zh) * 2017-05-08 2017-09-08 深圳地平线机器人科技有限公司 位姿信息确定方法、装置和可移动设备
CN107292818A (zh) * 2017-07-08 2017-10-24 上海交通大学 一种基于环视相机的捕线器自动定位系统及方法
CN107339996A (zh) * 2017-06-30 2017-11-10 百度在线网络技术（北京）有限公司 车辆自定位方法、装置、设备及存储介质
US20180225524A1 (en) * 2017-02-07 2018-08-09 Fujitsu Limited Moving-object position estimating system, information processing apparatus and moving-object position estimating method
CN110147094A (zh) * 2018-11-08 2019-08-20 北京初速度科技有限公司 一种基于车载环视系统的车辆定位方法及车载终端
CN111508258A (zh) * 2020-04-17 2020-08-07 北京三快在线科技有限公司 一种定位方法及装置

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB2568286B (en) * 2017-11-10 2020-06-10 Horiba Mira Ltd Method of computer vision based localisation and navigation and system for performing the same

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106403964A (zh) * 2016-08-30 2017-02-15 北汽福田汽车股份有限公司 一种定位导航系统及车辆
CN106407315A (zh) * 2016-08-30 2017-02-15 长安大学 一种基于街景图像数据库的车辆自主定位方法
US20180225524A1 (en) * 2017-02-07 2018-08-09 Fujitsu Limited Moving-object position estimating system, information processing apparatus and moving-object position estimating method
CN107144285A (zh) * 2017-05-08 2017-09-08 深圳地平线机器人科技有限公司 位姿信息确定方法、装置和可移动设备
CN107339996A (zh) * 2017-06-30 2017-11-10 百度在线网络技术（北京）有限公司 车辆自定位方法、装置、设备及存储介质
CN107292818A (zh) * 2017-07-08 2017-10-24 上海交通大学 一种基于环视相机的捕线器自动定位系统及方法
CN110147094A (zh) * 2018-11-08 2019-08-20 北京初速度科技有限公司 一种基于车载环视系统的车辆定位方法及车载终端
CN111508258A (zh) * 2020-04-17 2020-08-07 北京三快在线科技有限公司 一种定位方法及装置

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP3989170A4

Cited By (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114913491A (zh) * 2021-02-08 2022-08-16 广州汽车集团股份有限公司 一种车辆定位方法及系统、计算机可读存储介质
CN112966622A (zh) * 2021-03-15 2021-06-15 广州小鹏自动驾驶科技有限公司 一种停车场语义地图完善方法、装置、设备和介质
CN112966622B (zh) * 2021-03-15 2024-03-29 广州小鹏汽车科技有限公司 一种停车场语义地图完善方法、装置、设备和介质
CN114648639A (zh) * 2022-05-19 2022-06-21 魔视智能科技(武汉)有限公司 一种目标车辆的检测方法、系统及装置
CN115164906A (zh) * 2022-09-02 2022-10-11 深圳鹏行智能研究有限公司 定位方法、机器人和计算机可读存储介质
CN115164906B (zh) * 2022-09-02 2022-11-15 深圳鹏行智能研究有限公司 定位方法、机器人和计算机可读存储介质
CN115923847A (zh) * 2023-03-15 2023-04-07 安徽蔚来智驾科技有限公司 自动驾驶车辆的感知信息的预处理方法、装置及车辆
CN115923847B (zh) * 2023-03-15 2023-06-02 安徽蔚来智驾科技有限公司 自动驾驶车辆的感知信息的预处理方法、装置及车辆
CN116105603A (zh) * 2023-04-13 2023-05-12 安徽蔚来智驾科技有限公司 用于确定移动物体在场所中的位置的方法及系统
CN116105603B (zh) * 2023-04-13 2023-09-19 安徽蔚来智驾科技有限公司 用于确定移动物体在场所中的位置的方法及系统
CN116358573A (zh) * 2023-05-31 2023-06-30 小米汽车科技有限公司 地图建立方法、装置、存储介质以及车辆
CN116358573B (zh) * 2023-05-31 2023-08-29 小米汽车科技有限公司 地图建立方法、装置、存储介质以及车辆

Also Published As

Publication number Publication date
CN112116654A (zh) 2020-12-22
EP3989170A1 (en) 2022-04-27
EP3989170A4 (en) 2022-08-17

Similar Documents

Publication Publication Date Title
WO2020253842A1 (zh) 一种车辆位姿确定方法、装置及电子设备
JP6812404B2 (ja) 点群データを融合させるための方法、装置、コンピュータ読み取り可能な記憶媒体、及びコンピュータプログラム
EP3967972A1 (en) Positioning method, apparatus, and device, and computer-readable storage medium
US10788830B2 (en) Systems and methods for determining a vehicle position
CN110033489B (zh) 一种车辆定位准确性的评估方法、装置及设备
EP2399239B1 (en) Estimation of panoramic camera orientation relative to a vehicle coordinate frame
WO2020038285A1 (zh) 车道线的定位方法和装置、存储介质、电子装置
CN111127563A (zh) 联合标定方法、装置、电子设备及存储介质
CN108810473B (zh) 一种在移动平台上实现gps映射摄像机画面坐标的方法及系统
KR102016636B1 (ko) 카메라와 레이더의 캘리브레이션 장치 및 방법
KR101444685B1 (ko) 영상기반 멀티센서 데이터를 이용한 차량의 위치자세 결정 방법 및 장치
CN111311902B (zh) 一种数据处理方法、装置、设备和机器可读介质
CN113406682B (zh) 一种定位方法、装置、电子设备及存储介质
US20110292166A1 (en) North Centered Orientation Tracking in Uninformed Environments
JP2011242207A (ja) 端末位置特定システム、移動端末及び端末位置特定方法
CN110927708A (zh) 智能路侧单元的标定方法、装置及设备
WO2020039937A1 (ja) 位置座標推定装置、位置座標推定方法およびプログラム
CN107885763B (zh) 室内图中兴趣点信息的更新方法、装置及计算机可读介质
US20160169662A1 (en) Location-based facility management system using mobile device
CN115164918B (zh) 语义点云地图构建方法、装置及电子设备
JP2010190647A (ja) 車両位置測定装置及び車両位置測定プログラム
CN117197775A (zh) 对象标注方法、对象标注装置及计算机可读存储介质
TW201621273A (zh) 行動定位裝置及其定位方法
Bakuła et al. Capabilities of a smartphone for georeferenced 3dmodel creation: An evaluation
JP2018017652A (ja) 測量情報管理装置および測量情報管理方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20826498

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2020826498

Country of ref document: EP

Effective date: 20220120