CN112836698A - Positioning method, positioning device, storage medium and electronic equipment - Google Patents
Positioning method, positioning device, storage medium and electronic equipment Download PDFInfo
- Publication number
- CN112836698A CN112836698A CN202011633826.3A CN202011633826A CN112836698A CN 112836698 A CN112836698 A CN 112836698A CN 202011633826 A CN202011633826 A CN 202011633826A CN 112836698 A CN112836698 A CN 112836698A
- Authority
- CN
- China
- Prior art keywords
- grid semantic
- positioning
- grid
- map
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 42
- 239000003550 marker Substances 0.000 claims abstract description 58
- 230000007613 environmental effect Effects 0.000 claims abstract description 23
- 239000002245 particle Substances 0.000 claims description 47
- 238000004422 calculation algorithm Methods 0.000 claims description 43
- 238000004590 computer program Methods 0.000 claims description 11
- 238000005516 engineering process Methods 0.000 claims description 9
- 238000001914 filtration Methods 0.000 claims description 9
- 230000033001 locomotion Effects 0.000 claims description 9
- 230000008569 process Effects 0.000 claims description 9
- 238000007781 pre-processing Methods 0.000 claims description 8
- 238000000605 extraction Methods 0.000 claims description 6
- 238000013135 deep learning Methods 0.000 claims description 5
- 238000001514 detection method Methods 0.000 claims description 5
- 230000008447 perception Effects 0.000 claims description 5
- 238000006243 chemical reaction Methods 0.000 claims description 4
- 230000011218 segmentation Effects 0.000 claims description 4
- 238000012952 Resampling Methods 0.000 claims description 3
- 238000012216 screening Methods 0.000 claims description 3
- 230000009466 transformation Effects 0.000 claims description 3
- 238000013507 mapping Methods 0.000 claims 1
- 239000000126 substance Substances 0.000 claims 1
- 238000005286 illumination Methods 0.000 abstract description 5
- 230000000694 effects Effects 0.000 abstract description 4
- 230000002411 adverse Effects 0.000 abstract description 3
- 230000008859 change Effects 0.000 abstract description 3
- 238000010586 diagram Methods 0.000 description 5
- 230000000007 visual effect Effects 0.000 description 5
- 230000007547 defect Effects 0.000 description 3
- 230000004438 eyesight Effects 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 230000003044 adaptive effect Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 230000004807 localization Effects 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000035945 sensitivity Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 230000016776 visual perception Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/24—Aligning, centring, orientation detection or correction of the image
- G06V10/245—Aligning, centring, orientation detection or correction of the image by locating a pattern; Special marks for positioning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a positioning method, a positioning device, a storage medium and electronic equipment. The positioning method comprises the following steps: acquiring environmental image information of a to-be-positioned place; extracting the grid semantic information of a specific marker in the environment image information; acquiring a grid semantic map; and matching the grid semantic information with the grid semantic map, and determining a positioning result of the to-be-positioned location according to a matching result. The positioning solution of the invention has high stability and lower cost, and can effectively avoid the adverse effect of illumination and environmental change on positioning.
Description
Technical Field
The present invention relates to the field of positioning technologies, and in particular, to a positioning method, an apparatus, a storage medium, and an electronic device.
Background
The realization of positioning by means of vision and laser radar is a hot problem studied by current autonomous robots and autopilots.
The mainstream visual positioning scheme is to extract feature points from a picture, establish a feature point map, and perform positioning based on the feature points. However, the extraction of feature points and the calculation of feature point descriptors are susceptible to the influence of lighting and environment, i.e., feature points and descriptors extracted under different lighting conditions and in different seasons are also different. The difference between the characteristic points and the descriptors has a great influence on the stability of the positioning result, and has a great defect in practical application.
The laser radar-based positioning scheme senses the geometric structure of the surrounding environment by using the laser radar, establishes a laser point cloud map, and performs positioning based on the geometric structure of the laser point cloud map. The positioning accuracy of the laser radar is high, but the positioning cost of the laser radar is also high, which limits the application of the laser radar in practical positioning.
Therefore, there is a need to provide a new positioning solution to improve the positioning stability and reduce the positioning cost.
Disclosure of Invention
In view of the above-mentioned shortcomings of the prior art, an object of the present invention is to provide a positioning method, an apparatus, a storage medium and an electronic device, which are used to solve the problems of insufficient stability and high cost of the prior positioning technology.
To achieve the above and other related objects, the present invention provides a positioning method, including: acquiring environmental image information of a to-be-positioned place; extracting the grid semantic information of a specific marker in the environment image information; acquiring a grid semantic map; and matching the grid semantic information with the grid semantic map, and determining a positioning result of the to-be-positioned location according to a matching result.
In an embodiment of the present invention, the acquiring of the environmental image information of the location to be positioned includes preprocessing the acquired environmental image information, where the preprocessing includes gaussian denoising and image stitching, where the image stitching includes 3D virtual projection and viewpoint conversion technologies, and the stitched image can be arbitrarily converted into a viewing angle.
In an embodiment of the present invention, extracting the grid semantic information of the specific identifier in the environment image information includes: obtaining semantic information of all pixel points of one or more specific markers in the image based on a deep learning target detection algorithm or a semantic segmentation algorithm; and extracting pixel point semantic information of the specific marker as the grid semantic information of the specific marker.
In one embodiment of the present invention, the specific marker includes a road surface marker and a non-road surface space marker, in which: when the specific marker is a road marker, converting the grid semantic information of the specific marker into a road plane for matching in a overlooking manner; and when the specific marker is a non-road surface marker, the non-road surface space marker in the local grid semantic map of the object to be positioned is subjected to perspective transformation to an image plane for matching.
In an embodiment of the present invention, the grid semantic map may be a global grid semantic map or a local grid semantic map, where the local grid semantic map is a grid semantic map of a specific range determined based on a position of an object to be predicted to be positioned in the global grid semantic map.
In an embodiment of the present invention, the global grid semantic map building process includes: establishing a point cloud map based on the perception point cloud data and the positioning data; screening specific marker point clouds based on the point cloud map; extracting semantic information of all pixel points of the specific marker based on the specific marker point cloud; extracting pixel point semantic information of the specific marker as grid semantic information of the specific marker; and forming the global grid semantic map based on the grid semantic information of each specific identifier.
In an embodiment of the present invention, the local grid semantic map building process includes: acquiring the global grid semantic map; determining an initial positioning estimation value of an object to be positioned based on an external positioning source or a motion trend; and acquiring a grid semantic map within a specific range of the initial positioning estimation value based on the initial positioning estimation value.
In an embodiment of the present invention, matching the grid semantic information with the grid semantic map, and determining a positioning result of the to-be-positioned location according to a matching result includes: based on the prediction of the position of the object to be positioned, the current grid semantic information is superposed on the grid semantic map, and the optimal result of the current position of the object to be positioned is obtained by utilizing a probability model algorithm, wherein the probability model algorithm comprises a Kalman filtering algorithm, a particle filtering algorithm and a self-adaptive particle filtering algorithm.
In an embodiment of the present invention, matching the grid semantic information with the grid semantic map based on a particle filtering algorithm, and determining a positioning result of the to-be-positioned location according to the matching result, includes the following steps:
step 1), initializing a particle set, and when an initial time k is 0, determining an approximate position x of a location to be located0Approximate distribution p (x)0) Generating N particle poses
Wherein i is 1,2, … N;
step 2), predicting the particle pose at the next moment k equal to 1 according to the motion trend of the object to be positioned
Step 3), according to the predicted pose of each particle, matching the grid semantic information with a global grid semantic map or a local grid semantic map, and taking the distance value of the position information of the two as the weight of the particle
Step 4), according to the weight of the particles
Resampling the particles according to the size to obtain the position and posture of the resampled particles
Step 5), calculating all particles
The optimal position of the to-be-positioned location at the moment k is obtained according to the weight and the pose of the target location, wherein,
to achieve the above and other related objects, the present invention provides a positioning device, comprising: the image information acquisition module is used for acquiring the environmental image information of the to-be-positioned location; the semantic information extraction module is used for extracting the grid semantic information of the specific marker in the environment image information; the semantic map acquisition module is used for acquiring a grid semantic map; and the map matching and positioning module is used for matching the grid semantic information with the grid semantic map and determining a positioning result of the to-be-positioned place according to a matching result.
To achieve the above and other related objects, the present invention provides a computer-readable storage medium, in which a computer program is stored, and the computer program is loaded and executed by a processor to implement the positioning method.
To achieve the above and other related objects, the present invention provides an electronic device, comprising: a processor and a memory; wherein the memory is for storing a computer program; the processor is used for loading and executing the computer program to enable the electronic equipment to execute the positioning method.
As described above, according to the positioning method, the positioning apparatus, the storage medium, and the electronic device of the present invention, the environmental image information of the location to be positioned is obtained; extracting the grid semantic information of a specific marker in the environment image information; acquiring a grid semantic map; matching the grid semantic information with the grid semantic map, and determining a positioning result of the to-be-positioned location according to a matching result; high stability and low cost, and can effectively avoid the adverse effect of illumination and environmental change on positioning.
Drawings
Fig. 1 is a flowchart illustrating a positioning method according to an embodiment of the invention.
Fig. 2 is a schematic diagram illustrating a process of constructing a global grid semantic map according to an embodiment of the present invention.
Fig. 3 is a schematic diagram illustrating a process of obtaining a local grid semantic map according to an embodiment of the present invention.
Fig. 4A to 4D are schematic diagrams illustrating a principle of obtaining an optimal result of a current position of a target object by using a particle filtering algorithm according to an embodiment of the present invention.
Fig. 5 is a block diagram of a positioning device according to an embodiment of the invention.
Fig. 6 is a schematic structural diagram of an electronic device according to an embodiment of the invention.
Detailed Description
The embodiments of the present invention are described below with reference to specific embodiments, and other advantages and effects of the present invention will be easily understood by those skilled in the art from the disclosure of the present specification. The invention is capable of other and different embodiments and of being practiced or of being carried out in various ways, and its several details are capable of modification in various respects, all without departing from the spirit and scope of the present invention. It is to be noted that the features in the following embodiments and examples may be combined with each other without conflict.
It should be noted that the drawings provided in the following embodiments are only for illustrating the basic idea of the present invention, and the drawings only show the components related to the present invention rather than the number, shape and size of the components in actual implementation, and the type, quantity and proportion of the components in actual implementation may be changed freely, and the layout of the components may be more complicated. In addition, the present application does not limit the execution sequence of each step in the following embodiments, and the sequence between each step in the practical application is not limited to the embodiments provided in the present application.
The realization of positioning by means of vision and laser radar in the prior art is a hot problem studied by autonomous robots and autopilots. However, these two positioning methods have some disadvantages, respectively.
Based on the visual positioning technology, the matching relation of the feature points between the frames is obtained mainly through feature point matching, the pose of the corresponding time of the frames and the three-dimensional coordinates of the matched feature points are calculated through a multi-view geometric equation to form a feature point cloud, the feature point cloud is then re-projected onto the current image, and the optimal position estimation and the feature point coordinates are obtained through minimizing the projection error square sum of all the feature points. In the feature point-based positioning algorithm, commonly used feature point detection algorithms include SIFT, SURF, FAST, ORB, and the like. The SIFT and SURF algorithms keep invariance to rotation, scale scaling, brightness and the like, but the operation speed is slow, and real-time extraction and matching cannot be realized. The FAST algorithm only detects the gray value of a pixel, and is FAST, but has no direction and scale information. The ORB algorithm uses BRIEF descriptors on the basis of the FAST algorithm and solves the problems of scale and rotation by using an image pyramid and gray centroid method, but the ORB algorithm still has the problem of sensitivity to illumination and environmental changes. In addition, the optical flow-based positioning algorithm calculates the camera motion using the pixel gradation information of the image, but the optical flow method assumes that the pixel gradation value of the same spatial point is constant in each image. Since the assumption of invariant gray scale is difficult to satisfy in practice, the vision-based localization algorithm has a great disadvantage in practical application.
The laser radar-based positioning algorithm mainly extracts line and surface features from each frame of laser point cloud, obtains the relative poses of two frames of laser point cloud by utilizing geometric feature matching to form a laser point cloud map, and matches the laser point cloud features of the current frame with the laser point cloud map to obtain the optimal position, namely the generated point cloud map can only be positioned by using the laser radar. However, the data volume of the laser point cloud map is large, the cost of the laser radar is high, and each device using the laser point cloud map for positioning needs to be equipped with the laser radar, which is difficult to realize in practical application, and the application based on the laser radar positioning is limited to a great extent.
In view of the defects of the prior art, the application provides a positioning method based on a semantic map, which has high stability compared with the visual positioning technology, has lower cost compared with the laser radar positioning technology, and can effectively avoid the adverse effects of illumination and environmental changes on positioning.
The semantic map-based positioning method can be independently executed by electronic equipment such as a robot, a car machine, a smart phone, a tablet computer and a server, and can also be executed by combining a plurality of electronic equipment, and the method is not limited in the application.
As shown in fig. 1, the positioning method of the present application includes the following steps:
s11: and acquiring the environmental image information of the to-be-positioned position.
In an embodiment, it is preferable that the image information of the environment of the location to be positioned is acquired by an image information acquiring device of the object to be positioned. The environment image information can be acquired by a single image information acquisition device, or can be acquired by a plurality of image information acquisition devices in different directions and then spliced by images. Wherein the image stitching comprises: 3D virtual projection, viewpoint conversion technology and the like, and the spliced images can be displayed by converting the visual angle at will.
In an embodiment, preferably, the acquiring the environmental image information of the location to be located includes a step of preprocessing the acquired environmental image information, where the preprocessing includes: gaussian denoising, image splicing and the like.
For example, in step S11, the object to be positioned obtains an environmental picture at the location to be positioned by a single image information obtaining device, and performs gaussian denoising preprocessing on the environmental picture.
For another example, in step S11, the object to be positioned acquires multiple environmental pictures at the location to be positioned through multiple image information acquisition devices in different directions, the multiple environmental pictures can be spliced into one picture through technologies such as 3D virtual projection and viewpoint conversion, and similarly, gaussian denoising preprocessing can be performed on the environmental pictures.
S12: and extracting the grid semantic information of the specific marker in the environment image information.
Specifically, the specific identifier includes: pavement markers and non-pavement space markers. The pavement markers mainly comprise relatively standard and stable ground artificial markers such as lane lines, arrows, speed bumps, vehicle lines and the like, and the non-pavement markers comprise other markers except the pavement markers, such as telegraph poles, traffic signs, buildings, landscapes and the like, and can be defined by the skilled person.
In one embodiment, preferably, extracting the grid semantic information of the specific identifier in the environment image information includes: firstly, acquiring semantic information of all pixel points of one or more specific markers in an image based on a deep learning target detection algorithm or a semantic segmentation algorithm; and then, extracting pixel point semantic information of the specific marker as the grid semantic information of the specific marker.
It should be noted that, a person skilled in the art may use the existing deep learning target detection algorithm and semantic segmentation algorithm to perform semantic extraction on an image, and may also use other algorithms to perform semantic extraction on an image, which is not limited in this application.
S13: acquiring a grid semantic map;
the grid semantic map may be a global grid semantic map or a local grid semantic map.
Specifically, the global grid semantic map is a grid semantic map including a global scope. The global grid semantic map acquisition mode can be self-established, or can be acquired in a cloud downloading mode, an advanced built-in mode and other modes.
As shown in fig. 2, the global grid semantic map building process includes the following steps:
s21: establishing a point cloud map based on the perception point cloud data and the positioning data;
the sensing point cloud data can be generated by a laser radar, and the positioning data can be generated by a Beidou device and a GPS device. Since how to build the point cloud map is not a key invention point of the present application, it is not expanded in detail here, and those skilled in the art can build the point cloud map based on the sensing point cloud data and the positioning data by using the existing algorithm or software.
S22: screening specific marker point clouds based on the point cloud map;
in order to reduce the data volume of the global grid semantic map, the global grid semantic map is constructed by adopting the specific marker point clouds in the point cloud map. The specific identifier point cloud includes: the point clouds of the pavement markers and the point clouds of the non-pavement space markers can be selected by a person skilled in the art according to actual needs.
S23: extracting semantic information of all pixel points of the specific marker based on the specific marker point cloud;
for example, if the specific marker is a lane line, the semantic information of all the pixel points of the lane line is extracted from the point cloud of the lane line marker.
S24: extracting pixel point semantic information of the specific marker as grid semantic information of the specific marker;
s25: and extracting the grid semantic information of each specific marker to form a global grid semantic map.
Specifically, the local grid semantic map is a grid semantic map of a specific range determined based on the position of the object to be predicted to be positioned in the global grid semantic map.
As shown in fig. 3, the local grid semantic map building process includes the following steps:
s31: acquiring the global grid semantic map;
s32: determining an initial positioning estimation value of an object to be positioned based on an external positioning source or a motion trend;
wherein the external positioning source is, for example, GPS, beidou, visual SLAM, lidar SLAM, or the like. The movement trend can be comprehensively judged by the driving distance and the driving direction of the vehicle odometer, for example: on the basis of obtaining the global grid semantic map, obtaining the initial position of the vehicle according to a GPS (global positioning system), or performing clock synchronization on the obtained environment image information and vehicle body data by utilizing hardware equipment, and when a deep learning perception result of one frame (extracted grid semantic information of a specific marker in the environment image information) arrives, searching the corresponding vehicle body information according to a timestamp. Obtaining the current car body mileage value according to the time stamp
According to the position of the vehicle body at the moment k-1
And car body odometer
Predicting the current body position:
s33: and acquiring a grid semantic map within a specific range of the initial positioning estimation value based on the initial positioning estimation value.
In connection with the above example, in this step, the corresponding portion of the specific range of the initial positioning estimation value is extracted from the global grid semantic map, so as to form a local grid semantic map of a certain range including the position of the initial positioning estimation value. The size of the specific range value can be preset according to actual needs. For example, when the specific range value is set to 10 meters, the grid semantic map within a range of 10 meters around the current localization estimation value is selected from the global grid semantic map as the local grid semantic map.
S14: and matching the grid semantic information with the grid semantic map, and determining a positioning result of the to-be-positioned location according to a matching result.
Specifically, matching the grid semantic information with the grid semantic map, and determining a positioning result of the to-be-positioned location according to a matching result, includes: and based on the prediction of the position of the object to be positioned, superposing the current grid semantic information on the grid semantic map, and obtaining the optimal result of the current position of the object to be positioned by utilizing a probability model algorithm. The probabilistic model algorithm includes, but is not limited to, a kalman filter algorithm, a particle filter algorithm, an adaptive monte carlo particle filter algorithm.
Matching the grid semantic information with the grid semantic map based on a particle filter algorithm, and determining a positioning result of the to-be-positioned location according to the matching result, wherein the specific process comprises the following steps:
step 1), initializing a particle set, and when an initial time k is 0, determining an approximate position x of a location to be located0Approximate distribution p (x)0) Generating N particle poses
Wherein i is 1,2, … N;
step 2), predicting the particle pose at the next moment k equal to 1 according to the motion trend of the object to be positioned
Step 3), according to the predicted pose of each particle, matching the grid semantic information with a global grid semantic map or a local grid semantic map, and taking the distance value of the position information of the two as the weight of the particle
Step 4), according to the weight of the particles
Resampling the particles according to the size to obtain the position and posture of the resampled particles
Step 5) calculating all particles
The optimal position of the to-be-positioned location at the moment k is obtained according to the weight and the pose of the target location, wherein,
for example, referring to fig. 4A, first, at an initial time, 4 particles distributed near an initial position are obtained according to information provided by GPS; and secondly, acquiring the pose of the particle at the current moment according to the motion trend of the object to be positioned. Referring to fig. 4B, if a marker is found 3m ahead of the current position, as shown by the dashed circular frame, the black square in the figure is shown at 3m ahead of each particle, where the smaller the distance between the black block and the square enclosed by the dashed circular frame, the closer the particle is to the real position, and the larger the weight. Referring to fig. 4C, the particles are resampled according to the weights, and the particles with larger weights are sampled for a plurality of times, so as to obtain the particle distribution shown in fig. 4C. Referring to fig. 4D, the optimal result at the time k is obtained according to the weights and poses of all the particles, as shown by the arrows marked by the dashed oval line in fig. 4D.
And (4) after the optimal position is obtained through matching, positioning the place to be positioned is realized, and the steps S11-S14 are repeatedly executed, so that the target to be positioned can be continuously positioned in real time.
In one embodiment, when the specific marker is a road surface marker, the grid semantic information of the specific marker is transformed to a road plane for matching in a overlooking manner; and when the specific marker is a non-road surface marker, the non-road surface space marker in the local grid semantic map of the object to be positioned is subjected to perspective transformation to an image plane for matching.
It is worth noting that, compared with the matching of the grid semantic information and the global grid semantic map, the matching of the grid semantic information and the local grid semantic map is faster and more accurate.
In an embodiment, the grid semantic information may be matched with a global grid semantic map or a local grid semantic map based on probabilistic model algorithms including, but not limited to: kalman filter algorithm, particle filter algorithm, adaptive particle filter algorithm, etc.
All or part of the steps for implementing the above method embodiments may be performed by hardware associated with a computer program. Based upon such an understanding, the present invention also provides a computer program product comprising one or more computer instructions. The computer instructions may be stored in a computer readable storage medium. The computer-readable storage medium can be any available medium that a computer can store or a data storage device, such as a server, a data center, etc., that is integrated with one or more available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., Solid State Disk (SSD)), among others.
As shown in fig. 5, the present application further provides a positioning device 50, and since the specific implementation of the present device is the same as that of the foregoing method embodiment, the same contents are not repeated herein. The positioning device 50 mainly comprises the following modules:
an image information obtaining module 51, configured to obtain environment image information of a location to be positioned;
a semantic information extracting module 52, configured to extract grid semantic information of a specific identifier in the environment image information;
a semantic map obtaining module 53, configured to obtain a grid semantic map;
and the map matching and positioning module 54 is configured to match the grid semantic information with the grid semantic map, and determine a positioning result of the to-be-positioned location according to a matching result.
Referring to fig. 6, the embodiment provides an electronic device, which may be a desktop, a tablet computer, a smart phone, a car machine, or the like. In detail, the electronic device comprises at least, connected by a bus: the system comprises a memory and a processor, wherein the memory is used for storing computer programs, and the processor is used for executing the computer programs stored by the memory so as to execute all or part of the steps in the method embodiment.
The above-mentioned system bus may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The system bus may be divided into an address bus, a data bus, a control bus, and the like. For ease of illustration, only one thick line is shown, but this does not mean that there is only one bus or one type of bus. The communication interface is used for realizing communication between the database access device and other equipment (such as a client, a read-write library and a read-only library). The Memory may include a Random Access Memory (RAM), and may further include a non-volatile Memory (non-volatile Memory), such as at least one disk Memory.
The Processor may be a general-purpose Processor, and includes a Central Processing Unit (CPU), a Network Processor (NP), and the like; the Integrated Circuit may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, a discrete Gate or transistor logic device, or a discrete hardware component.
In summary, according to the positioning method, the positioning device, the storage medium and the electronic device of the present invention, the point cloud map is used to extract the grid semantic map, so as to ensure the accuracy of the grid semantic map, reduce the data volume of the map, and perform positioning by using the visual perception grid semantic information during real-time positioning, so that the cost is low, the map reusability is high, and the perception grid semantic information is not affected by illumination and environmental changes, thereby being beneficial to improving the positioning stability, effectively overcoming various defects in the prior art, and having high industrial utilization value.
The foregoing embodiments are merely illustrative of the principles and utilities of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or change the above-mentioned embodiments without departing from the spirit and scope of the present invention. Accordingly, it is intended that all equivalent modifications or changes which can be made by those skilled in the art without departing from the spirit and technical spirit of the present invention be covered by the claims of the present invention.
Claims (12)
1. A method of positioning, comprising:
acquiring environmental image information of a to-be-positioned place;
extracting the grid semantic information of a specific marker in the environment image information;
acquiring a grid semantic map;
and matching the grid semantic information with the grid semantic map, and determining a positioning result of the to-be-positioned location according to a matching result.
2. The method according to claim 1, wherein the acquiring the environmental image information of the location to be positioned comprises preprocessing the acquired environmental image information, wherein the preprocessing comprises gaussian denoising and image stitching, wherein the image stitching comprises 3D virtual projection and viewpoint conversion technology, and the stitched image can be arbitrarily converted in view angle.
3. The method of claim 1, wherein extracting the grid semantic information of the specific identifier in the environment image information comprises:
obtaining semantic information of all pixel points of one or more specific markers in the image based on a deep learning target detection algorithm or a semantic segmentation algorithm;
and extracting pixel point semantic information of the specific marker as the grid semantic information of the specific marker.
4. The method of claim 1, wherein the specific marker comprises a pavement marker and a non-pavement space marker, wherein:
when the specific marker is a road marker, converting the grid semantic information of the specific marker into a road plane for matching in a overlooking manner;
and when the specific marker is a non-road surface marker, the non-road surface space marker in the local grid semantic map of the object to be positioned is subjected to perspective transformation to an image plane for matching.
5. The method according to claim 1, wherein the grid semantic map may be a global grid semantic map or a local grid semantic map, wherein the local grid semantic map is a grid semantic map of a specific range determined based on a location of an object to be predicted to be located in the global grid semantic map.
6. The method of claim 5, wherein the global grid semantic map building process comprises:
establishing a point cloud map based on the perception point cloud data and the positioning data;
screening specific marker point clouds based on the point cloud map;
extracting semantic information of all pixel points of the specific marker based on the specific marker point cloud;
extracting pixel point semantic information of the specific marker as grid semantic information of the specific marker;
and forming the global grid semantic map based on the grid semantic information of each specific identifier.
7. The method of claim 5, wherein the local grid semantic mapping process comprises:
acquiring the global grid semantic map;
determining an initial positioning estimation value of an object to be positioned based on an external positioning source or a motion trend;
and acquiring a grid semantic map within a specific range of the initial positioning estimation value based on the initial positioning estimation value.
8. The method according to claim 1, wherein matching the grid semantic information with the grid semantic map and determining a positioning result of the location to be positioned according to the matching result comprises: based on the prediction of the position of the object to be positioned, the current grid semantic information is superposed on the grid semantic map, and the optimal result of the current position of the object to be positioned is obtained by utilizing a probability model algorithm, wherein the probability model algorithm comprises a Kalman filtering algorithm, a particle filtering algorithm and a self-adaptive particle filtering algorithm.
9. The method according to claim 8, wherein the grid semantic information is matched with the grid semantic map based on a particle filtering algorithm, and the positioning result of the location to be positioned is determined according to the matching result, comprising the following steps:
step 1), initializing a particle set, and when an initial time k is 0, determining an approximate position x of a location to be located0Approximate distribution p (x)0) Generating N particle poses
Wherein i is 1,2,. N;
step 2), predicting the particle pose at the next moment k equal to 1 according to the motion trend of the object to be positioned
Step 3), according to the predicted pose of each particle, matching the grid semantic information with a global grid semantic map or a local grid semantic map, and taking the distance value of the position information of the two as the weight of the particle
Step 4), according to the weight of the particles
Resampling the particles according to the size to obtain the position and posture of the resampled particles
Step 5), calculating all particles
The optimal position of the to-be-positioned location at the moment k is obtained according to the weight and the pose of the target location, wherein,
10. a positioning device, comprising:
the image information acquisition module is used for acquiring the environmental image information of the to-be-positioned location;
the semantic information extraction module is used for extracting the grid semantic information of the specific marker in the environment image information;
the semantic map acquisition module is used for acquiring a grid semantic map;
and the map matching and positioning module is used for matching the grid semantic information with the grid semantic map and determining a positioning result of the to-be-positioned place according to a matching result.
11. A computer-readable storage medium, in which a computer program is stored which, when being loaded and executed by a processor, carries out a positioning method according to any one of claims 1 to 9.
12. An electronic device, comprising: a processor and a memory; wherein the content of the first and second substances,
the memory is used for storing a computer program;
the processor is configured to load and execute the computer program to cause the electronic device to perform the positioning method according to any one of claims 1 to 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011633826.3A CN112836698A (en) | 2020-12-31 | 2020-12-31 | Positioning method, positioning device, storage medium and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011633826.3A CN112836698A (en) | 2020-12-31 | 2020-12-31 | Positioning method, positioning device, storage medium and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112836698A true CN112836698A (en) | 2021-05-25 |
Family
ID=75926091
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011633826.3A Pending CN112836698A (en) | 2020-12-31 | 2020-12-31 | Positioning method, positioning device, storage medium and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112836698A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113256712A (en) * | 2021-06-01 | 2021-08-13 | 北京有竹居网络技术有限公司 | Positioning method, positioning device, electronic equipment and storage medium |
| WO2023060592A1 (en) * | 2021-10-15 | 2023-04-20 | 华为技术有限公司 | Positioning method and device |
| WO2024036984A1 (en) * | 2022-08-16 | 2024-02-22 | 华为云计算技术有限公司 | Target localization method and related system, and storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104236551A (en) * | 2014-09-28 | 2014-12-24 | 北京信息科技大学 | Laser range finder-based map establishing method of snake-like robot |
| CN111486855A (en) * | 2020-04-28 | 2020-08-04 | 武汉科技大学 | Indoor two-dimensional semantic grid map construction method with object navigation points |
| CN111539994A (en) * | 2020-04-28 | 2020-08-14 | 武汉科技大学 | Particle filter repositioning method based on semantic likelihood estimation |
| CN111780771A (en) * | 2020-05-12 | 2020-10-16 | 驭势科技（北京）有限公司 | Positioning method, positioning device, electronic equipment and computer readable storage medium |
-
2020
- 2020-12-31 CN CN202011633826.3A patent/CN112836698A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104236551A (en) * | 2014-09-28 | 2014-12-24 | 北京信息科技大学 | Laser range finder-based map establishing method of snake-like robot |
| CN111486855A (en) * | 2020-04-28 | 2020-08-04 | 武汉科技大学 | Indoor two-dimensional semantic grid map construction method with object navigation points |
| CN111539994A (en) * | 2020-04-28 | 2020-08-14 | 武汉科技大学 | Particle filter repositioning method based on semantic likelihood estimation |
| CN111780771A (en) * | 2020-05-12 | 2020-10-16 | 驭势科技（北京）有限公司 | Positioning method, positioning device, electronic equipment and computer readable storage medium |
Non-Patent Citations (1)
| Title |
|---|
| 谭建豪: "数字图像处理与移动机器人路径规划", vol. 1, 30 April 2013, 华中科技大学出版社, pages: 345 - 346 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113256712A (en) * | 2021-06-01 | 2021-08-13 | 北京有竹居网络技术有限公司 | Positioning method, positioning device, electronic equipment and storage medium |
| WO2022253113A1 (en) * | 2021-06-01 | 2022-12-08 | 北京有竹居网络技术有限公司 | Localization method and apparatus, electronic device, and storage medium |
| WO2023060592A1 (en) * | 2021-10-15 | 2023-04-20 | 华为技术有限公司 | Positioning method and device |
| WO2024036984A1 (en) * | 2022-08-16 | 2024-02-22 | 华为云计算技术有限公司 | Target localization method and related system, and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11145073B2 (en) | Computer vision systems and methods for detecting and modeling features of structures in images | |
| CN111325796B (en) | Method and apparatus for determining pose of vision equipment | |
| CN109683699B (en) | Method and device for realizing augmented reality based on deep learning and mobile terminal | |
| CN112836698A (en) | Positioning method, positioning device, storage medium and electronic equipment | |
| JP2021089724A (en) | 3d auto-labeling with structural and physical constraints | |
| You et al. | An accurate and practical calibration method for roadside camera using two vanishing points | |
| CN113989450A (en) | Image processing method, image processing apparatus, electronic device, and medium | |
| US10235800B2 (en) | Smoothing 3D models of objects to mitigate artifacts | |
| Tao et al. | Stereo priori RCNN based car detection on point level for autonomous driving | |
| CN114764778A (en) | Target detection method, target detection model training method and related equipment | |
| CN112348887A (en) | Terminal pose determining method and related device | |
| CN114217665A (en) | Camera and laser radar time synchronization method, device and storage medium | |
| CN115375836A (en) | Point cloud fusion three-dimensional reconstruction method and system based on multivariate confidence filtering | |
| CN110298320B (en) | Visual positioning method, device and storage medium | |
| CN115578432B (en) | Image processing method, device, electronic equipment and storage medium | |
| CN112767477A (en) | Positioning method, positioning device, storage medium and electronic equipment | |
| CN115239899B (en) | Pose map generation method, high-precision map generation method and device | |
| CN115565072A (en) | Road garbage recognition and positioning method and device, electronic equipment and medium | |
| Ariff et al. | Exploratory study of 3D point cloud triangulation for smart city modelling and visualization | |
| CN113409473B (en) | Method, device, electronic equipment and storage medium for realizing virtual-real fusion | |
| Šuľaj et al. | Examples of real-time UAV data processing with cloud computing | |
| Ayadi et al. | A skyline-based approach for mobile augmented reality | |
| Rajabi et al. | Optimization of DTM interpolation using SFS with single satellite imagery | |
| CN117197361B (en) | Live three-dimensional database construction method, electronic device and computer readable medium | |
| Ping et al. | A Rapid Method of Monocular Image Measurement Based on Rectangle Information |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |