CN112284394A

CN112284394A - Map construction and visual positioning method and device

Info

Publication number: CN112284394A
Application number: CN202011148320.3A
Authority: CN
Inventors: 马际洲; 聂琼; 申浩
Original assignee: Beijing Sankuai Online Technology Co Ltd
Current assignee: Beijing Sankuai Online Technology Co Ltd
Priority date: 2020-10-23
Filing date: 2020-10-23
Publication date: 2021-01-29

Abstract

The map construction method comprises the steps of storing image information corresponding to an acquired positioning image into a positioning map according to the corresponding relation between the image information and each sub-region in a positioning region of the map, carrying out visual positioning according to the positioning map, comparing image information contained in a scene image set corresponding to a to-be-positioned sub-region with a current environment image, and selecting a target sub-region and a corresponding scene image set according to a comparison result so as to determine the pose of positioning equipment. In any of the above methods, the features included in each sub-region are described by all the image information corresponding to the sub-region, and for the sub-region having a visual repetitive scene, the representation of the visual repetitive scene in the image is diluted by the image information corresponding to the plurality of images, and the number of contrast samples in positioning is increased, so that a more accurate contrast result can be obtained and positioning can be performed.

Description

Map construction and visual positioning method and device

Technical Field

The present disclosure relates to the field of positioning technologies, and in particular, to a method and an apparatus for map construction and visual positioning.

Background

Currently, with the development of LBS (Location Based Services), Positioning technologies such as GPS (Global Positioning System), base station Positioning, and visual Positioning are widely used, wherein the visual Positioning is often applied to scenes with high requirements for Positioning accuracy.

In the prior art, a method for visual positioning includes retrieving a current environment image acquired during positioning from positioning images included in a pre-stored positioning map, performing feature point matching on the retrieved positioning image and the current environment image, and resolving a pose (including a three-dimensional coordinate, a pitch angle, a yaw angle, and a roll angle of the positioning device) of the positioning device during acquisition of the current environment image according to a real pose of the corresponding positioning image acquisition device during acquisition for a feature point pair successfully matched with the current environment image.

However, when the method is used for visual positioning, since image retrieval is to compare a single positioning image with a current environment image, when a plurality of visually similar areas exist in a positioning area, the occupation ratio of the visually similar areas reflected in the single positioning image may be large, and a phenomenon that the single positioning image has a high degree of similarity with the current environment image but fails in matching when feature point matching is performed occurs.

Disclosure of Invention

The present specification provides a map construction method to partially solve the above-mentioned problems of the prior art.

The technical scheme adopted by the specification is as follows:

the present specification provides a map construction method, including:

determining a positioning area, acquiring positioning images in the positioning area, and determining image information corresponding to each positioning image, wherein the image information at least comprises pose information of acquisition equipment when the positioning images are acquired;

determining a plurality of sub-regions divided in a positioning region in advance;

for each sub-region, determining a scene image set corresponding to the sub-region according to pose information corresponding to each positioning image, wherein the scene image set is formed by image information corresponding to at least one positioning image;

and storing the image information included in each scene image set as a positioning map of the positioning area, wherein the positioning map is used for carrying out visual positioning in the positioning area.

Optionally, determining the scene image set corresponding to the sub-region specifically includes:

and determining a scene image set corresponding to the sub-region, which is formed by image information corresponding to the positioning image with the position in the sub-region, according to the position of the acquisition equipment when each positioning image is acquired.

Optionally, after determining the scene image set corresponding to the sub-region, the method further includes:

and determining at least one view-angle image set according to the pose included by each image information in the scene image set, wherein each view-angle image set is formed by at least one image information with similar pose included in the scene image set.

Optionally, before storing the image information included in each scene image set, the method further includes:

extracting feature points contained in each positioning image, and determining position information of each feature point contained in each positioning image according to pose information of acquisition equipment when the positioning image is acquired; for each positioning image, the image information corresponding to the positioning image further comprises: each feature point included in the positioning image, and position information of each feature point included in the positioning image.

Optionally, the storing image information included in each scene image set specifically includes:

and storing the image information of which the distance between the included position and the position included by the image information is smaller than a distance threshold value as a positioning subgraph corresponding to the image information for each image information included in each scene image set.

The technical scheme of at least one map construction adopted by the specification can achieve the following beneficial effects:

the method comprises the steps of storing image information corresponding to an acquired positioning image into a positioning map according to the corresponding relation between the image information and each sub-region in a positioning region of the map, wherein the characteristics of each sub-region can be described by all image information corresponding to the sub-region together, for the sub-region with a visual repetitive scene, the representation of the visual repetitive scene in the image is diluted by the image information corresponding to a plurality of images, and a comparison sample containing more information is provided for the image acquired by equipment needing positioning, so that a more accurate comparison result can be obtained and positioning can be carried out.

The present specification also provides a method and an apparatus for performing visual positioning based on a positioning map previously constructed using the map construction method, so as to partially solve the above problems in the prior art.

The technical scheme adopted by the specification is as follows:

the present specification provides a visual localization method, comprising:

determining a current environment image acquired by positioning equipment in a positioning area; selecting a to-be-determined subarea from all subareas in the positioning area;

for each undetermined sub-area, determining a scene image set corresponding to the undetermined sub-area according to a positioning map of the positioning area, taking at least partial image information in the scene image set as target image information of the scene image set, and comparing the target image information with the current environment image to obtain a comparison result corresponding to the undetermined sub-area;

and selecting a target sub-region from each sub-region to be determined according to the comparison result corresponding to each sub-region to be determined, and determining the pose information of the positioning equipment according to the scene image set corresponding to the target sub-region and the current environment image.

Optionally, before selecting the pending sub-region, the method further includes:

acquiring the position and attitude information of the positioning equipment which is determined for the last time; estimating the current estimated position of the positioning equipment according to the position and attitude information of the positioning equipment determined at the last time; the selecting of the to-be-determined sub-region specifically includes: and selecting a pending sub-area according to the current pre-estimated position.

Optionally, selecting an undetermined sub-region according to the current estimated position specifically includes:

determining a range which takes the current estimated position as a circle center and a distance threshold as a radius as a current estimated range; and selecting a sub-region which has intersection with the current prediction range as a to-be-determined sub-region according to each sub-region in the positioning region.

Optionally, taking at least part of image information in the scene image set as target image information of the scene image set, and comparing the target image information with the current environment image, specifically including:

and taking the image information of which the distance between the included position and the current estimated position is smaller than the distance threshold value as the target image information of the scene image set, and comparing the target image information with the current environment image.

Optionally, predicting the current predicted position of the positioning device specifically includes:

estimating the acquisition position of the positioning equipment when acquiring the current environment image; determining image information with the closest distance between the included position and the acquisition position in the image information contained in the positioning map; and determining the position included in the determined image information as the estimated current position of the positioning equipment.

Optionally, determining pose information of the positioning device according to the scene image set corresponding to the target sub-region and the current environment image, specifically including:

extracting feature points contained in the current environment image, and determining the positions of the feature points in the current positioning image; taking at least part of image information in a scene image set corresponding to the target sub-region as target image information of the scene image set, taking feature points included in the target image information as feature points included in the target sub-region, and matching the feature points included in the target sub-region with the feature points included in the current environment image to obtain feature point pairs which are successfully matched; and aiming at the feature point pairs successfully matched, determining the pose information of the positioning equipment according to the position of the feature points contained in the current environment image and the position information of the feature points successfully matched with the current environment image.

Optionally, taking at least part of image information in the scene image set as target image information of the scene image set, and comparing the target image information with the current environment image to obtain a comparison result corresponding to the to-be-determined sub-region, specifically including:

taking at least part of image information in the scene image set as target image information of the scene image set, comparing feature information included in the target image information with feature information included in the current environment image, and determining the similarity between the undetermined sub-region and the current environment image as a comparison result corresponding to the undetermined sub-region;

selecting a target sub-region from the sub-regions to be selected, which specifically comprises:

and selecting the undetermined sub-region with the highest similarity with the current environment image as a target sub-region.

Optionally, determining the similarity between the to-be-determined sub-region and the current environment image specifically includes:

determining the similarity between the undetermined sub-region and the current environment image according to the image feature similarity between the undetermined sub-region and the current environment image and/or the feature point goodness of fit between the undetermined sub-region and the current environment image;

determining the image feature similarity between the to-be-determined sub-region and the current environment image, specifically comprising:

taking at least part of image information in the scene image set as target image information of the scene image set, comparing image feature information included in the target image information with image feature information included in the current environment image, and determining image feature similarity of the region to be determined and the current environment image according to a comparison result of the image feature information;

determining the goodness of fit of the characteristic points of the to-be-determined sub-region and the current environment image, specifically comprising:

and taking at least part of image information in the scene image set as target image information of the scene image set, matching the feature points included in the target image information with the feature points included in the current environment image, and determining the feature point goodness of fit of the to-be-determined sub-region and the current environment image according to the matching result of the feature points.

Optionally, taking at least part of image information in the scene image set as target image information of the scene image set, and comparing feature information included in the target image information with feature information included in the current environment image, specifically including:

determining each view angle image set included in the scene image set, regarding each view angle image set, taking at least partial image information in the view angle image set as target image information of the view angle image set, comparing feature information included in the target image information with feature information included in the current environment image, and determining the similarity between the view angle image set and the current environment image; determining the similarity between the to-be-determined sub-region and the current environment image according to the similarity between each view angle image set in the scene image set and the current environment image;

determining pose information of the positioning device according to the scene image set corresponding to the target sub-region and the current environment image, specifically comprising:

and selecting a view angle image set with the highest feature similarity with the current environment image from a scene image set corresponding to the target sub-region, and determining the pose information of the positioning equipment according to the target image information of the view angle image set and the current environment image.

This specification provides a visual positioning device, includes:

the acquisition and undetermined module comprises: determining a current environment image acquired by positioning equipment in a positioning area; selecting a to-be-determined subarea from all subareas in the positioning area;

a pending comparison module: for each undetermined sub-area, determining a scene image set corresponding to the undetermined sub-area according to a positioning map of the positioning area, taking at least partial image information in the scene image set as target image information of the scene image set, and comparing the target image information with the current environment image to obtain a comparison result corresponding to the undetermined sub-area; the positioning map corresponding to the positioning area is constructed according to the map construction method which is used in advance;

a pose determination module: and selecting a target sub-region from each sub-region to be determined according to the comparison result corresponding to each sub-region to be determined, and determining the pose information of the positioning equipment according to the scene image set corresponding to the target sub-region and the current environment image.

The present specification provides a computer-readable storage medium storing a computer program which, when executed by a processor, implements the above-described map construction or visual positioning method.

The present specification provides an electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the above mapping or visual positioning method when executing the program.

The technical scheme of at least one visual positioning adopted by the specification can achieve the following beneficial effects:

selecting undetermined sub-regions from the sub-regions included in the positioning region, respectively comparing image information included in a scene image set corresponding to the undetermined sub-regions with a current environment image, selecting a target sub-region from the undetermined sub-regions according to a comparison result, determining the pose of the positioning equipment according to the selected scene image set corresponding to the target sub-region and the current environment image, changing a sample compared with the current environment image into a plurality of image information with each sub-region as a unit from the image information corresponding to a single positioning image, increasing the comparison sample so as to dilute the proportion of a visual repetitive scene in all samples, and improving the positioning success rate of the positioning equipment of which the acquired current environment image includes the visual repetitive scene.

Drawings

The accompanying drawings, which are included to provide a further understanding of the specification and are incorporated in and constitute a part of this specification, illustrate embodiments of the specification and together with the description serve to explain the specification and not to limit the specification in a non-limiting sense. In the drawings:

FIG. 1 is a schematic flow chart of a map construction method in the present specification;

fig. 2A and 2B are schematic diagrams of two methods for dividing sub-regions shown in this specification, respectively;

FIG. 3 is a schematic diagram of a positioning map for organizing image information by positioning subgraphs in the present specification;

FIG. 4 is a flow chart of a visual positioning method according to the present disclosure;

FIG. 5 is a schematic diagram of a method for determining a set of perspective images from a pose according to the present disclosure;

FIG. 6 is a schematic view of a visual positioning apparatus provided herein;

fig. 7 is a schematic diagram of an electronic device corresponding to fig. 4 provided in the present specification.

Detailed Description

In order to make the objects, technical solutions and advantages of the present disclosure more clear, the technical solutions of the present disclosure will be clearly and completely described below with reference to the specific embodiments of the present disclosure and the accompanying drawings. It is to be understood that the embodiments described are only a few embodiments of the present disclosure, and not all embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments in the present specification without any creative effort belong to the protection scope of the present specification.

The visual positioning technology, namely positioning in a map based on image information, is widely applied as an economical and practical positioning technology due to the low cost of a camera and the rich information which can be obtained by the camera.

The existing visual positioning technology is that in positioning images contained in a positioning map which is acquired and stored in advance, a current environment image acquired during positioning is directly retrieved, characteristic point matching is carried out on the retrieved positioning image and the current environment image, and for a characteristic point pair which is successfully matched, the position and the attitude of the positioning equipment during acquisition of the current environment image are resolved according to the real position and the attitude of the corresponding positioning image acquisition equipment during acquisition.

However, in practical applications, since image retrieval is to compare a single positioning image with a current environment image, when there are many visually similar areas in a positioning area, the proportion of the visually similar areas reflected in the single positioning image may be large, and a phenomenon that the degree of similarity between the single positioning image and the current environment image is high, but matching fails when feature point matching is performed occurs may occur. In order to solve the problem of positioning failure caused by the visual repeatability of a scene, an auxiliary device is often additionally arranged on positioning equipment to assist visual positioning.

Before visual positioning, an auxiliary device additionally arranged in the positioning equipment is used for determining the estimated pose of the positioning equipment in advance, for example, GPS auxiliary positioning is adopted, namely, a GPS device is firstly utilized to determine the estimated pose of the positioning equipment, the estimated pose is used for narrowing the range of a positioning image compared with a current environment image, then the current environment image is retrieved in the narrowed range, and the retrieved positioning image and the current environment image are subjected to feature point matching and pose settlement.

The method for assisting positioning by using the auxiliary device can eliminate positioning images which are similar to visual features contained in a current environment image and appear in an area far away from the position (three-dimensional coordinates) of positioning equipment, so that the problem of visual repeatability is partially solved, but due to the limitation of the positioning accuracy of the GPS technology, the positioning accuracy of the current civil GPS technology usually exceeds ten meters, only a larger range can be determined for the positioning equipment as an image retrieval range, so that the visual similar scene with a close distance cannot be limited and reflected in the positioning images, and the problem of positioning failure is caused.

The technical solutions provided by the embodiments of the present description are described in detail below with reference to the accompanying drawings.

Fig. 1 is a schematic diagram of a map building process provided in an embodiment of the present specification, including:

s100: determining a positioning area, acquiring positioning images in the positioning area, and determining image information corresponding to each positioning image, wherein the image information at least comprises pose information of acquisition equipment when the positioning images are acquired.

In an embodiment of the present specification, the acquisition device includes at least an image input device, such as a camera capable of image capturing, and the acquisition device performs acquisition of the positioning image. By using the map construction method provided by the present specification, the positioning map corresponding to the positioning area is constructed according to the positioning image acquired by the acquisition device, and the map construction method can be executed by the acquisition device or a server controlling the acquisition device. The positioning map constructed by the map construction method provided by the present specification may be stored in a device that needs to perform visual positioning using the map, may be stored in a storage medium that can be read by a device that needs to perform visual positioning using the map, or may be stored in a server that can perform information transmission with a device that needs to perform visual positioning using the map. The present specification does not limit the execution subject of the map construction by using the method provided in the present specification, and does not limit how to store the positioning map constructed by using the method provided in the present specification, and for convenience of description, the following description will take an example of an apparatus storage map in which an acquisition apparatus constructs a map and needs to perform visual positioning.

The positioning area described in the embodiments of the present specification is an area where visual positioning can be performed for a device that needs positioning using a corresponding positioning map. Specifically, the positioning can be performed visually, the designated positioning map can provide positioning service for the equipment in the corresponding positioning area, and the positioning can be successfully performed according to the positioning map instead of acquiring images at any pose in the positioning area.

The acquisition equipment operates in the positioning area and acquires images with the position and posture which can be realized by the acquisition equipment. Specifically, when the acquisition device is a device capable of moving, such as an unmanned vehicle, the acquisition device executes self-movement and image acquisition behaviors according to a preset mode during operation; when the acquisition equipment is the equipment which can not move, the acquisition equipment executes the acquisition action of the image when running. The acquisition device may include one device or a plurality of devices, and when the acquisition device is a plurality of devices, the positioning image is determined according to the positioning image acquired by each acquisition device.

When the mode of the acquisition behavior executed by the acquisition equipment is video shooting, the positioning image can be a key frame in a video determined according to the video shot by the acquisition equipment or a section of video determined according to the shot video; when the mode of the acquisition device executing the acquisition behavior is picture shooting, the positioning image may be all pictures shot by the acquisition device, or at least a part of pictures selected from all pictures shot by the acquisition device.

The positioning image is an image acquired by the acquisition device in the positioning area, but whether all features contained in the positioning image are in the positioning area is not limited, and when the acquisition device acquires the image containing the features outside the positioning area in the positioning area, the image can still be used as the positioning image for map construction.

Specifically, the image information may include the positioning image, and in order to save a space for storing the positioning map, the image information may include only information related to the positioning image, such as features extracted from the positioning image, and does not include the positioning image itself. When the image information includes the corresponding positioning image, the features included in the positioning image may be extracted in advance and stored in the image information as information related to the positioning image, or when the device requiring visual positioning needs to perform positioning using the features included in the positioning image, the required features may be extracted according to the stored positioning image.

In an embodiment of the present specification, features of an image are extracted from a collected positioning image, a clustering algorithm is used to cluster a set of the extracted features, a feature dictionary composed of aggregated feature words is obtained, for each positioning image, features included in the positioning image are converted into representations of the feature words according to the feature dictionary, so that a feature word vector (or a feature word histogram) corresponding to the positioning image is constructed, and the feature word vector is stored in image information as features of the positioning image.

The acquisition equipment can acquire self pose information, when the acquisition equipment acquires images, the corresponding relation between the acquired images and the pose information when the images are acquired is recorded, and for each positioning image, the image information corresponding to the positioning image at least comprises the pose information of the acquisition equipment when the positioning image is acquired, which is recorded by the acquisition equipment.

S102: a plurality of sub-regions divided in advance in the positioning region are determined.

According to the requirement, a plurality of sub-areas are divided in the positioning area in advance, and for each positioning area, the collection of all the sub-areas corresponding to the positioning area is the positioning area.

The sub-regions may be of geometric shape, e.g. rectangular, circular; it can also be divided into irregular shapes according to the geographic information of the location area, for example, according to the block, river, etc.; the areas of the sub-regions can be equal or different; may overlap or may be mutually exclusive.

Fig. 2A illustrates a dividing method of a non-overlapping positioning area between sub-areas, where sub-areas 1 to 4 are sub-areas divided by the dividing method as an example.

FIG. 2B shows another dividing manner in which the sub-regions can overlap with each other, in FIG. 2B, the sub-regions 1-4 are respectively sub-regions divided by the method, wherein the sub-regions include two overlapping regions shown in a shaded manner, and the overlapping region shown in the shaded portion of the upper portion is a portion in which the sub-region 1 overlaps with the sub-region 4; the overlap region shown with lower hatching is a portion where the sub-region 3 overlaps with the sub-region 4.

In an embodiment of the present specification, when the scenes that are not mutually transparent are not different in practice, the scenes are not part of the same sub-area, and the corresponding sub-areas may not overlap each other, or at least may not overlap each other at the parts that are mutually opaque.

S104: and for each sub-region, determining a scene image set corresponding to the sub-region according to the pose information corresponding to each positioning image, wherein the scene image set is formed by image information corresponding to at least one positioning image.

For each sub-region, according to the pose information corresponding to each positioning image, determining a scene image set corresponding to the sub-region for storing image information related to the sub-region, and specifically, according to the range of the sub-region, adding the included image information whose position is within the range of the sub-region to the scene image set corresponding to the sub-region.

Accordingly, each scene image set corresponds to a sub-region, that is, when there are N sub-regions, there are N scene image sets, and the sub-regions correspond to the scene image sets one by one.

In an embodiment of the present specification, when the sub-regions overlap, image information corresponding to the same positioning image may belong to a scene image set corresponding to a plurality of sub-regions at the same time.

S106: and storing the image information included in each scene image set as a positioning map of the positioning area, wherein the positioning map is used for carrying out visual positioning in the positioning area.

When the same image information belongs to the scene image sets corresponding to the multiple sub-regions, the image information may be stored once and occupy a storage space as the image information included in each scene image set, or only occupy a storage space of one image information, and identify a correspondence between the image information and each scene image set to which the image information belongs, for example, point to the image information in a pointer manner.

It can be understood that the image information included in each scene image set is stored, so that each image information is stored at least once as the image information included in one scene image set, and the positioning map stores the image information corresponding to all positioning images included in the corresponding positioning area, the corresponding relationship between the image information and the scene image set, and the corresponding relationship between the image information and the sub-area, so as to provide a positioning service for the device to be positioned.

As can be seen from the method described in fig. 1, the present specification provides a map construction method, in which image information corresponding to an acquired positioning image is stored according to a correspondence between the image information and each sub-region in a positioning region of a map, so that, for a feature included in each sub-region, the feature can be described by all image information corresponding to the sub-region, and for a sub-region having a visual repetitive scene, the representation of the visual repetitive scene in the image is diluted by the image information corresponding to a plurality of images, so that a contrast sample including more information is provided for an image acquired by a device requiring positioning, and thus, a more accurate contrast result can be obtained and positioning can be performed.

Further, for each scene image set, at least one perspective image set composed of at least one image information with similar posture (pitch angle, yaw angle, and roll angle) included in the scene image set may be determined, and specifically, according to the posture included in each image information, one or more of the pitch angle, the yaw angle, and the roll angle are selected as an index for aggregation according to a requirement to aggregate each image information, and at least one perspective image set composed of at least one image information with similar posture included after aggregation is obtained. In practical application, a plurality of acquisition devices with fixed postures are often used for acquiring positioning images, so that each view angle image set can be determined by determining the acquisition devices for acquiring the positioning images. In one embodiment of the present specification, for each scene image set, the same image information in the scene image set exists in only one perspective image set.

When the acquisition equipment acquires the positioning image, the acquisition equipment can simultaneously acquire the relative position between the shot object contained in the positioning image and the positioning equipment, specifically, when the acquisition equipment acquires the positioning image by using a depth camera, the depth camera can acquire the distance between the shot object in the positioning image and the depth camera while acquiring the positioning image, and the relative position between each shot object in the positioning image and the positioning equipment can be acquired by combining the two-dimensional coordinates of the shot object in the two-dimensional plane of the positioning image; when the positioning device further comprises a device capable of acquiring point cloud data such as a radar, the positioning device can acquire the point cloud data around the positioning device when acquiring the positioning image, and the relative position of each shot object in the positioning image and the positioning device is determined according to the position information of the object around the positioning device, which is included in the point cloud data.

Further, for each image information, the image information may further include feature points extracted from the positioning image corresponding to the image information, and position information of each feature point. Specifically, for each positioning image, feature points may be extracted from the positioning image, and the relative position between each feature point extracted from the positioning image and the positioning device may be determined according to the relative position between the subject and the positioning device, and the position (three-dimensional coordinates) of each feature point extracted from the positioning image in the positioning region may be determined according to the real pose of the acquisition device included in the image information when the positioning image is acquired.

According to the positioning mode selected by the equipment needing positioning, the characteristics needed to be used in positioning are different, if the positioning map is stored, the equipment needing positioning can be known to be positioned according to which characteristics, and different characteristics can be stored according to the positioning mode. For example, when a positioning map is required to be compared with image features of an image acquired by equipment to be positioned, and then feature points matched with the image acquired by the equipment to be positioned are selected according to an image feature comparison result, each piece of image information may include two parts, namely basic information and specific information, the image features in the image information are stored as the basic information in the image information, when positioning is performed, the equipment to be positioned loads all the basic information included in the image information, performs image feature comparison with the acquired image according to the basic information, determines which image information includes the feature points to be matched with the feature points included in which the image information is included according to the image feature comparison result, and then loads specific information including the feature points and the feature point positions according to requirements, thereby avoiding pre-loading all the image information included in the positioning map when positioning is performed each time, the occupation of the memory is saved.

In one embodiment of the present specification, the image information may be organized in the form of a positioning subgraph, specifically, the positioning map is stored as a set of positioning subgraphs corresponding to all the image information included in the positioning map, where, for each image information, the positioning subgraph corresponding to the image information is a set of image information whose included position and the position included in the image information are less than a distance threshold, and it is understood that the image information itself may also be stored as image information whose included position and the position included in the image information are less than the distance threshold in the positioning subgraph corresponding to the image information.

In an embodiment of this specification, each image information corresponds to a unique identifier, a positioning sub-image corresponding to the image information may also be identified as the identifier of the positioning sub-image, when a positioning map is stored as a set of positioning sub-images corresponding to all image information included in the positioning map, each positioning sub-image and each image information included in the positioning sub-image may be stored in a linear structure manner such as a linked list, and specifically, the identifier of the positioning sub-image may be used as an index of each positioning sub-image in the linked list.

In one embodiment of the present specification, for each stored locator sub-graph, the locator sub-graph may be stored with a key value as an identifier of the locator sub-graph and the content as a hash table of pointers to all image information stored by the locator sub-graph.

When the image information is organized as a positioning map in the form of a positioning subgraph, for each image information in the positioning map, the correspondence of the image information to the sub-region may be stored in the image information.

As shown in table 1 corresponding to fig. 3, the above-described positioning map storage method for organizing image information in a positioning sub-map manner may be:

TABLE 1

As shown in table 1 above, in the positioning sub-graph represented in fig. 3 and centered on the image information C, the image information a in the sub-region 1, the image information B in the sub-region 2, the image information C and the image information E in the sub-region C, the image information D in the sub-region 4, and the image information F in the sub-region 5 are included, the image information a to F are respectively stored in the scene image sets 1 to 5 corresponding to the sub-regions 1 to 5 intersecting with the positioning sub-graph, and the image information H and the image information G not belonging to the positioning sub-graph are not stored in the positioning sub-graph.

Based on the same idea, the map construction method provided by the embodiment of the present specification further provides a computer-readable storage medium storing a computer program, where the computer program is operable to execute the map construction method provided in fig. 1.

Technical solutions provided by embodiments for performing visual positioning according to a positioning map constructed by using the above-described map construction method are described below with reference to the accompanying drawings.

Fig. 4 is a schematic view of a visual positioning process provided in an embodiment of the present disclosure, including:

s400: determining a current environment image acquired by positioning equipment in a positioning area; and selecting a to-be-determined sub-region from the sub-regions in the positioning region.

When the positioning equipment runs in the positioning area and is positioned by using the corresponding positioning map, the positioning equipment acquires images at the position and posture which can be realized by the positioning equipment, and the images acquired by the positioning equipment are used for positioning, namely the current environment images. The positioning device can move, and for the current environment image acquired when the positioning device moves, the positioning method provided by the specification is used for acquiring a positioning result, namely the pose when the current environment image is acquired.

The method includes that the current environment image is acquired by a positioning device, the acquisition mode of the positioning device is video shooting or picture shooting, and when the acquisition mode of the positioning device is video shooting, the current environment image can be a key frame in a video determined according to the video shot by the positioning device or a section of video determined according to the shot video; when the mode of the positioning device executing the acquisition behavior is picture shooting, the current environment image is a limited number of pictures determined according to all the pictures shot by the positioning device.

When the feature which is collected by the positioning equipment in the positioning area and does not belong to the positioning area, the feature which does not belong to the positioning area can still be used as the current environment image.

In the visual positioning method provided in this specification, when the position of the positioning device cannot be determined, for example, a pose output historically when the positioning device is successfully positioned cannot be obtained (for example, when the positioning device performs initial positioning), or the position of the positioning device cannot be estimated according to pose information output historically when the positioning device is successfully positioned, or the positioning device cannot be positioned according to the estimated position of the positioning device, the sub-region to be determined may be all sub-regions in the positioning region; before the current environment image is compared with the positioning map, if the position of the positioning equipment is considered to be in some sub-areas in the positioning area by a higher confidence degree, the sub-areas can be selected from the sub-areas included in the positioning area to be used as the to-be-determined sub-areas, and the positioning equipment is visually positioned only according to the image information related to the to-be-determined sub-areas.

S402: and for each undetermined sub-area, determining a scene image set corresponding to the undetermined sub-area according to a positioning map of the positioning area, taking at least part of image information in the scene image set as target image information of the scene image set, and comparing the target image information with the current environment image to obtain a comparison result corresponding to the undetermined sub-area.

The positioning map corresponding to the positioning area described in fig. 4 is constructed in advance according to the map construction method shown in fig. 1 described above.

And determining a scene image set corresponding to each selected undetermined sub-region according to the positioning map of the positioning region, wherein the scene image set comprises image information related to the undetermined sub-region.

For each scene image set, comparing image information included in the scene image set with the current environment image as target image information, namely retrieving the current environment image in the scene image set corresponding to each to-be-determined sub-region; the search range may also be narrowed, and at least part of the image information is selected as the target image information for each scene image set, that is, the current environment image is searched in the set of target image information corresponding to each region to be determined.

Further, the target image information is compared with the current environment image, that is, the feature information included in the target image information is compared with the feature information included in the current environment image, and the comparison result is the similarity degree of the current environment image and the target image information corresponding to each to-be-determined subregion, that is, the similarity degree of the current environment image and each to-be-determined subregion. Specifically, the feature information is a feature capable of describing information included in the positioning image or the current environment image, which feature is selected and which comparison method is used for comparison is not limited in this specification, and any feature which can describe image information can be extracted by using any feature extraction method as a feature for comparing the similarity degree, for example, a feature point which is commonly used at present and is extracted in a Scale-invariant feature transform (SIFT) manner; image features represented by color sets and color moments; expressing and matching texture features by using a gray level co-occurrence matrix; shape features extracted by the edge feature method, and the like.

When the feature points included in the target image information are used as the feature information, and the target image information corresponding to the undetermined sub-region is compared with the current environment image for each undetermined sub-region, the essence is to extract the feature points from the current environment image and compare the feature points with the feature points included in the target image information, so that the similarity degree between the feature points is used as the similarity degree between the undetermined sub-region corresponding to the feature points and the current environment image, but because the feature points can exist not only as the feature information, but also for each feature point extracted from the positioning image, because the image information can also include the position (three-dimensional coordinates) of each feature point in the positioning region determined by the method shown in the map construction method shown in the above-mentioned fig. 1, optionally, the comparison process of comparing the feature points as the feature information is described above, according to the pose calculation result of the feature point corresponding to each to-be-determined sub-region to the positioning equipment, the coincidence degree of the feature point contained in the current environment image and the feature point corresponding to each to-be-determined sub-region can be determined, and therefore the similarity degree of each to-be-determined sub-region and the current environment image is obtained.

According to the comparison of the characteristic information, obtaining the similarity degree of the target image information corresponding to each undetermined subregion and the current environment image, and using the similarity degree as the similarity degree of the corresponding undetermined subregion and the current environment image, namely the comparison result; when the feature information is compared in a feature point matching mode, the obtained comparison result is the matching degree of the feature points contained in the current environment image and the feature points corresponding to the sub-regions to be determined, and the similarity degree of the sub-regions to be determined and the current environment image is determined according to the matching degree, namely the comparison result.

S404: and selecting a target sub-region from each sub-region to be determined according to the comparison result corresponding to each sub-region to be determined, and determining the pose information of the positioning equipment according to the scene image set corresponding to the target sub-region and the current environment image.

And selecting at least one undetermined sub-region from the undetermined sub-regions as a target sub-region according to the sequence of the similarity degree of the undetermined sub-regions and the current environment image from high to low.

When two or more than two target sub-regions exist, the pose information of the positioning equipment can be determined according to the scene image set corresponding to each target sub-region, specifically, the feature points and the positions of the feature points included in the target image information corresponding to each target sub-region can be respectively matched with the feature points included in the current environment image, and the pose of the positioning equipment is solved according to the feature points successfully matched; or one target sub-region can be selected, and the pose information of the positioning equipment is determined according to the scene image set corresponding to the target sub-region and the current environment image. The specification does not limit how to select one target sub-region from a plurality of target sub-regions.

When only one target sub-region exists, the pose of the positioning device is determined according to the image information contained in the scene image set corresponding to the target sub-region and the current environment image, and specifically, the pose of the positioning device is determined according to the target image information corresponding to the target sub-region and the current environment image. It can be understood that the image information includes image features and feature point features, the pose of the positioning device can be resolved according to the feature points and the positions of the feature points in the image information, and the pose of the positioning device can also be determined according to the image features in the image information.

As can be seen from the method shown in fig. 4, the present specification provides a method for performing visual positioning according to the map construction method shown in fig. 1, selecting pending sub-regions from the sub-regions included in the positioning region, comparing image information included in the scene image sets corresponding to the pending sub-regions with the current environment image, selecting a target sub-region from the pending sub-regions according to the comparison result, determining the pose of the positioning device according to the selected scene image set corresponding to the target sub-region and the current environment image, so as to change the image information corresponding to the current environment image into a plurality of image information using each sub-region as a unit, increase the comparison sample to dilute the occupation ratio of the visual repetitive scene in all samples, and improve the positioning success rate of the positioning device whose acquired current environment image includes the visual repetitive scene, and when the position of the positioning equipment is judged with higher confidence, the comparison range can be selected from the sub-regions to be determined before comparison, so as to narrow the comparison range of the current environment image, and the influence of other visual similar scenes far away from the judged position of the positioning equipment on positioning can be eliminated.

Furthermore, the visual positioning method provided in the present specification does not include using an auxiliary positioning device, such as a GPS device, to perform estimation positioning, but if the positioning device can move within the positioning area and has been historically positioned successfully, the current estimated position of the positioning device can be estimated according to pose information obtained when the positioning device has been successfully positioned last time.

The current estimated location of the positioning device may be estimated based on at least one of, or a combination of: the time of the positioning device from the latest successful positioning moment to the current positioning moment, the motion state of the positioning device within the time of the latest successful positioning moment to the current positioning moment, the pose output when the latest successful positioning is carried out, and the geographic information around the output position when the latest successful positioning is carried out.

Specifically, the motion tendency of the positioning equipment when the positioning is successful at the last time can be judged according to the pose output when the positioning is successful at the last time; each feasible path of the positioning equipment after the last positioning is successful can be determined according to the position output when the last positioning is successful and the surrounding geographic information; the moving mileage of the positioning equipment within the time from the latest positioning success moment to the current positioning moment can be estimated according to the time from the latest positioning success moment to the current positioning moment of the positioning equipment and the moving state of the positioning equipment within the time from the latest positioning success moment to the current positioning moment; the position of the positioning device can be estimated according to the moving mileage, the moving tendency and each feasible path of the positioning device.

The estimated position of the positioning device in the above manner can be used as the current estimated position of the positioning device; the position near the position of the positioning device estimated in the above manner can also be determined as the current estimated position according to the position of the positioning device estimated in the above manner.

After the current estimated position of the positioning device is obtained, the sub-region around the current estimated position can be selected as the to-be-determined sub-region according to the current estimated position, so as to narrow the comparison range of the current environment image.

Specifically, a range in which the current estimated position is used as a circle center and the distance threshold is used as a radius can be determined as the current estimated range of the current estimated position, and a sub-region intersecting with the current estimated range is selected as a sub-region to be determined, so as to narrow the comparison range to the image information in the scene image set corresponding to the sub-region to be determined.

For the scene image set corresponding to each to-be-determined sub-region, when the current estimated position of the positioning device is estimated, all image information in the scene image set corresponding to each to-be-determined sub-region does not need to be compared with the current environment image, the comparison and range restriction can be performed again in each scene image set according to the current estimated position of the positioning device, specifically, the image information of which the distance between the included position and the current estimated position is smaller than the distance threshold value can be used as the target image information of the scene image set, the target image information is compared with the current environment image, that is, the image information of which the included position is within the current estimated range in the scene image set is used as the target image information for each scene image set, and the target image information in each scene image set is used as the target image information of the corresponding to-be-determined sub-region, and comparing the current environment image with the target image information of each to-be-determined subarea.

When the image information is organized in the form of a positioning subgraph, the current environment image is compared with the target image information of each to-be-determined subregion, conversely, according to the estimated current position, a positioning subgraph to be compared with the current environment image is determined, according to the corresponding relation between the image information and the subregions, each image information in the positioning subgraph is used as the target image information of at least one to-be-determined subregion, each target image information in the positioning subgraph is obtained, the target image information of each to-be-determined subregion is used as a unit and is compared with the current environment image, and specifically, at least one feature contained in each target image information can be selected according to requirements and is used as the feature of the to-be-determined subregion corresponding to the target image information to be compared with the current environment image.

It can be understood that the current estimated position of the positioning device may also be the position of the positioning device estimated in the above manner, that is, the acquisition position when the positioning device acquires the current environment image, and the position near the acquisition position, for example, according to the position included in each image information, the image information whose included position is closest to the estimated acquisition position is selected, the position included in the image information is determined as the estimated current estimated position, when the current estimated position is determined by the method, the positioning map obtained by organizing the image information in the positioning subgraph manner can be used for positioning, and the positioning subgraph can be determined by determining the current estimated position, that is, the positioning subgraph corresponding to the image information including the current estimated position is the comparison range with the current environment image.

It can be understood that, when the feature point matching is adopted to compare the target image information included in each to-be-determined sub-region, the more the feature point included in the corresponding target image information is matched with the feature point included in the current environment image, the higher the similarity between the corresponding to-be-determined sub-region and the current environment image is, for example, the to-be-determined sub-region where the pose of the current positioning device can be successfully calculated by the feature point information included in the corresponding target image information is the target sub-region; however, when the similarity degrees of the features are compared, the positioning device can be positioned according to the corresponding target image information of the target sub-region for the target sub-region selected according to the similarity degrees.

Specifically, according to the corresponding target image information of the target sub-region, determining feature point information included in the target image information, and taking feature points included in each target image information as feature points included in the target sub-region, performing feature point matching on the feature points included in the target sub-region and feature points extracted from the current environment image to obtain successfully matched feature point pairs, wherein the feature points belonging to the target sub-region in each successfully matched feature point pair are determined according to the above method, or the positions (three-dimensional coordinates) of the feature points are determined in advance according to the above method, and the feature points belonging to the current environment image in each successfully matched feature point pair are determined, the positions of the feature points relative to the positioning device are determined according to the above method, and according to the positions of the feature points included in the current environment image, and position information of the feature points successfully matched with the current environment image is determined, the pose information of the positioning equipment is determined, specifically, any algorithm can be selected for pose resolving, such as a quaternion algorithm, and the output pose information of the positioning equipment is obtained.

In an embodiment of the present specification, when the pose information of the positioning apparatus cannot be successfully output according to the feature points included in the target sub-region, the to-be-determined sub-region may be re-determined for the positioning apparatus that collects the current environment image, and optionally, a range that uses the current estimated position as a center of a circle and a second distance threshold as a radius is determined as the current estimated range, and a sub-region that intersects with the current estimated range is selected as the re-determined to-be-determined sub-region according to each sub-region in the positioning region, where the second distance threshold is greater than the distance threshold, it can be understood that, when the determined current estimated range is a range that uses the current position as a center of a circle and the second distance threshold as a radius, the area of the current estimated range is increased, and the sub-region that intersects with the current estimated range is also increased, so as to achieve the selection of the to-be, and the selection range of the region to be determined is enlarged. When a positioning map for organizing image information in the form of positioning subgraphs is adopted, the selection range of the to-be-determined sub-regions is expanded, namely the second distance threshold is used as the radius of the current estimated range, so that a larger current estimated range is obtained, the expanded current estimated range is used as a new positioning subgraph, and the target image information corresponding to each to-be-determined sub-region is compared in the new positioning subgraph.

In an embodiment of the present specification, when the similarity degree of the comparison features is selected as the comparison method of the feature information of each to-be-determined sub-region, the matching degree between the feature points included in the target image information obtained by using the feature point matching method and the feature points included in the current environment image may also be used as a consideration label of the similarity degree of the feature information, for example, the feature points included in each target image information are respectively matched with the feature points included in the current environment image, the pose calculation is performed on the positioning device according to the feature point pairs obtained by matching the current environment image with each target image information, and the similarity degree of the feature comparison between the to-be-determined sub-region and the current environment image is determined according to the result of the pose calculation.

When the adopted positioning map further comprises at least one perspective image set aiming at each scene image set, the image information containing similar postures is aggregated and stored respectively, and the similarity degree of the features can be compared in a mode of a finer-grained perspective image set. Specifically, at least part of image information in each view image set may be used as target image information of the view image set, the target image information of each view image set is respectively compared with the current environment image to obtain the similarity between each view image set and the current environment image, for each pending sub-region, the similarity between the pending sub-region and the current environment image is determined according to the similarity between each view image set included in the scene image set corresponding to the pending sub-region and the current environment image, the target sub-region is selected according to the determined similarity between each pending sub-region and the current environment image, the view image set with the highest similarity between the scene image set corresponding to the target sub-region and the current environment image is determined, and the target image information of the view image set is determined, and determining the pose information of the positioning equipment by adopting any method for determining the pose of the positioning equipment according to the target image information and the environment image.

When a positioning map for organizing image information in the form of positioning subgraphs is adopted, for each positioning subgraph, each view angle image set can be stored as a mutually exclusive image information set in a scene image set contained in the positioning subgraph.

The following describes a storage method of a localization map in which image information is organized as a localization subgraph when localization images a to G included in the subregion 1 shown in fig. 5 are stored in correspondence with table 2 shown in fig. 5:

TABLE 2

As shown in table 2 above, as shown in fig. 5, when each of the positioning images a to G in the sub-region 1 is located in the positioning sub-image corresponding to the image information B, according to the different postures included in each of the image information (i.e., the capturing angles of the capturing devices corresponding to the captured positioning images), the image information a to G are respectively stored in the perspective image sets 1 to 3 included in the scene image set in the positioning sub-image corresponding to the image information B, that is, the perspective image sets 1 to 3 respectively aggregate image information including similar postures.

Based on the same idea, the visual positioning method provided by the embodiments of the present specification further provides a corresponding apparatus, a storage medium, and an electronic device.

Fig. 6 is a schematic structural diagram of a visual positioning apparatus provided in an embodiment of the present specification, the apparatus including:

the acquisition and undetermined module 600 determines a current environment image acquired by positioning equipment located in a positioning area; selecting a to-be-determined subarea from all subareas in the positioning area;

an undetermined comparison module 602, configured to determine, for each undetermined sub-region, a scene image set corresponding to the undetermined sub-region according to a positioning map of the positioning region, use at least part of image information in the scene image set as target image information of the scene image set, and compare the target image information with the current environment image to obtain a comparison result corresponding to the undetermined sub-region; the positioning map corresponding to the positioning area is constructed in advance according to the map construction method shown in fig. 1;

the pose determining module 604 selects a target sub-region from the sub-regions to be determined according to the comparison result corresponding to each sub-region to be determined, and determines pose information of the positioning device according to the scene image set corresponding to the target sub-region and the current environment image.

Optionally, the acquiring and pending module 600 is specifically configured to acquire the latest determined pose information of the positioning device; estimating the current estimated position of the positioning equipment according to the position and attitude information of the positioning equipment determined at the last time; and selecting a pending sub-area according to the current pre-estimated position.

Optionally, the acquiring and undetermined module 600 is specifically configured to determine a range with the current estimated position as a circle center and a distance threshold as a radius as a current estimated range; and selecting a sub-region which has intersection with the current prediction range as a to-be-determined sub-region according to each sub-region in the positioning region.

Optionally, the to-be-determined comparison module 602 is specifically configured to, in the scene image set, use image information that a distance between the included position and the current estimated position is smaller than a distance threshold as target image information of the scene image set, and compare the target image information with the current environment image.

Optionally, the acquiring and pending module 600 is specifically configured to predict an acquisition position of the positioning device when acquiring the current environment image; determining image information with the closest distance between the included position and the acquisition position in the image information contained in the positioning map; and determining the position included in the determined image information as the estimated current position of the positioning equipment.

Optionally, the pose determining module 604 is specifically configured to extract feature points included in the current environment image, and determine positions of the feature points in the current positioning image; taking at least part of image information in a scene image set corresponding to the target sub-region as target image information of the scene image set, taking feature points included in the target image information as feature points included in the target sub-region, and matching the feature points included in the target sub-region with the feature points included in the current environment image to obtain feature point pairs which are successfully matched; and aiming at the feature point pairs successfully matched, determining the pose information of the positioning equipment according to the position of the feature points contained in the current environment image and the position information of the feature points successfully matched with the current environment image.

Optionally, the to-be-determined comparison module 602 is specifically configured to use at least part of image information in the scene image set as target image information of the scene image set, compare feature information included in the target image information with feature information included in the current environment image, and determine a similarity between the to-be-determined sub-region and the current environment image, where the similarity is used as a comparison result corresponding to the to-be-determined sub-region; and selecting the undetermined sub-region with the highest similarity with the current environment image as a target sub-region.

Optionally, the to-be-determined module 602 is specifically configured to determine the similarity between the to-be-determined sub-region and the current environment image according to the image feature similarity between the to-be-determined sub-region and the current environment image and/or the feature point goodness of fit between the to-be-determined sub-region and the current environment image.

Optionally, determining the image feature similarity between the to-be-determined sub-region and the current environment image specifically includes: and taking at least part of image information in the scene image set as target image information of the scene image set, comparing image characteristic information included in the target image information with image characteristic information included in the current environment image, and determining the image characteristic similarity of the region to be determined and the current environment image according to the comparison result of the image characteristic information.

Optionally, determining the goodness of fit of the feature point of the to-be-determined sub-region and the current environment image specifically includes: and taking at least part of image information in the scene image set as target image information of the scene image set, matching the feature points included in the target image information with the feature points included in the current environment image, and determining the feature point goodness of fit of the to-be-determined sub-region and the current environment image according to the matching result of the feature points.

Optionally, the to-be-determined module 602 is specifically configured to determine each view-angle image set included in the scene image set, regarding each view-angle image set, use at least part of image information in the view-angle image set as target image information of the view-angle image set, compare feature information included in the target image information with feature information included in the current environment image, and determine a similarity between the view-angle image set and the current environment image; and determining the similarity between the undetermined sub-region and the current environment image according to the similarity between the image set of each view angle in the scene image set and the current environment image.

Optionally, the pose determining module 604 is specifically configured to select, in a scene image set corresponding to the target sub-region, an angle-of-view image set with the highest feature similarity to the current environment image, and determine the pose information of the positioning device according to target image information of the angle-of-view image set and the current environment image.

The present specification also provides a computer readable storage medium storing a computer program, which can be used to execute the visual positioning method provided in fig. 4.

This specification also provides a schematic block diagram of the electronic device shown in fig. 7. As shown in fig. 7, at the hardware level, the electronic device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory, but may also include hardware required for other services. The processor reads a corresponding computer program from the non-volatile memory into the memory and then runs the computer program to implement the map building method shown in fig. 1 or the visual positioning method shown in fig. 4. Of course, besides the software implementation, the present specification does not exclude other implementations, such as logic devices or a combination of software and hardware, and the like, that is, the execution subject of the following processing flow is not limited to each logic unit, and may be hardware or logic devices.

In the 90 s of the 20 th century, improvements in a technology could clearly distinguish between improvements in hardware (e.g., improvements in circuit structures such as diodes, transistors, switches, etc.) and improvements in software (improvements in process flow). However, as technology advances, many of today's process flow improvements have been seen as direct improvements in hardware circuit architecture. Designers almost always obtain the corresponding hardware circuit structure by programming an improved method flow into the hardware circuit. Thus, it cannot be said that an improvement in the process flow cannot be realized by hardware physical modules. For example, a Programmable Logic Device (PLD), such as a Field Programmable Gate Array (FPGA), is an integrated circuit whose Logic functions are determined by programming the Device by a user. A digital system is "integrated" on a PLD by the designer's own programming without requiring the chip manufacturer to design and fabricate application-specific integrated circuit chips. Furthermore, nowadays, instead of manually making an Integrated Circuit chip, such Programming is often implemented by "logic compiler" software, which is similar to a software compiler used in program development and writing, but the original code before compiling is also written by a specific Programming Language, which is called Hardware Description Language (HDL), and HDL is not only one but many, such as abel (advanced Boolean Expression Language), ahdl (alternate Hardware Description Language), traffic, pl (core universal Programming Language), HDCal (jhdware Description Language), lang, Lola, HDL, laspam, hardward Description Language (vhr Description Language), vhal (Hardware Description Language), and vhigh-Language, which are currently used in most common. It will also be apparent to those skilled in the art that hardware circuitry that implements the logical method flows can be readily obtained by merely slightly programming the method flows into an integrated circuit using the hardware description languages described above.

The controller may be implemented in any suitable manner, for example, the controller may take the form of, for example, a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro) processor, logic gates, switches, an Application Specific Integrated Circuit (ASIC), a programmable logic controller, and an embedded microcontroller, examples of which include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320, the memory controller may also be implemented as part of the control logic for the memory. Those skilled in the art will also appreciate that, in addition to implementing the controller as pure computer readable program code, the same functionality can be implemented by logically programming method steps such that the controller is in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers, embedded microcontrollers and the like. Such a controller may thus be considered a hardware component, and the means included therein for performing the various functions may also be considered as a structure within the hardware component. Or even means for performing the functions may be regarded as being both a software module for performing the method and a structure within a hardware component.

The systems, devices, modules or units illustrated in the above embodiments may be implemented by a computer chip or an entity, or by a product with certain functions. One typical implementation device is a computer. In particular, the computer may be, for example, a personal computer, a laptop computer, a cellular telephone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

For convenience of description, the above devices are described as being divided into various units by function, and are described separately. Of course, the functions of the various elements may be implemented in the same one or more software and/or hardware implementations of the present description.

As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.

The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.

Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.

It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.

As will be appreciated by one skilled in the art, embodiments of the present description may be provided as a method, system, or computer program product. Accordingly, the description may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the description may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

This description may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.

The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the system embodiment, since it is substantially similar to the method embodiment, the description is simple, and for the relevant points, reference may be made to the partial description of the method embodiment.

The above description is only an example of the present specification, and is not intended to limit the present specification. Various modifications and alterations to this description will become apparent to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present specification should be included in the scope of the claims of the present specification.

Claims

1. A map construction method, comprising:

2. The method of claim 1, wherein determining the set of images of the scene corresponding to the sub-region comprises:

3. The method of claim 1 or 2, wherein after determining the set of images of the scene corresponding to the sub-region, the method further comprises:

4. The method of claim 1, wherein prior to storing the image information included in each scene image set, the method further comprises:

extracting feature points contained in each positioning image, and determining position information of each feature point contained in each positioning image according to pose information of acquisition equipment when the positioning image is acquired;

for each positioning image, the image information corresponding to the positioning image further comprises: each feature point included in the positioning image, and position information of each feature point included in the positioning image.

5. The method of claim 1, wherein storing the image information included in each scene image set comprises:

6. A visual positioning method, comprising:

for each undetermined sub-area, determining a scene image set corresponding to the undetermined sub-area according to a positioning map of the positioning area, taking at least partial image information in the scene image set as target image information of the scene image set, and comparing the target image information with the current environment image to obtain a comparison result corresponding to the undetermined sub-area; the positioning map corresponding to the positioning area is constructed in advance according to the map construction method of any one of claims 1 to 5;

7. The method of claim 6, wherein prior to selecting a pending sub-region, the method further comprises:

acquiring the position and attitude information of the positioning equipment which is determined for the last time;

estimating the current estimated position of the positioning equipment according to the position and attitude information of the positioning equipment determined at the last time;

the selecting of the to-be-determined sub-region specifically includes:

and selecting a pending sub-area according to the current pre-estimated position.

8. The method of claim 7, wherein selecting a region to be determined based on the current estimated position comprises:

determining a range which takes the current estimated position as a circle center and a distance threshold as a radius as a current estimated range;

and selecting a sub-region which has intersection with the current prediction range as a to-be-determined sub-region according to each sub-region in the positioning region.

9. The method according to claim 8, wherein taking at least a portion of image information in the scene image set as target image information of the scene image set, and comparing the target image information with the current environment image specifically includes:

10. The method of claim 7, wherein estimating a current estimated location of the positioning device comprises:

estimating the acquisition position of the positioning equipment when acquiring the current environment image;

determining image information with the closest distance between the included position and the acquisition position in the image information contained in the positioning map;

and determining the position included in the determined image information as the estimated current position of the positioning equipment.

11. The method according to any one of claims 6 to 10, wherein determining pose information of the positioning device according to the set of scene images corresponding to the target sub-region and the current environment image specifically includes:

extracting feature points contained in the current environment image, and determining the positions of the feature points in the current positioning image;

taking at least part of image information in a scene image set corresponding to the target sub-region as target image information of the scene image set, taking feature points included in the target image information as feature points included in the target sub-region, and matching the feature points included in the target sub-region with the feature points included in the current environment image to obtain feature point pairs which are successfully matched;

and aiming at the feature point pairs successfully matched, determining the pose information of the positioning equipment according to the position of the feature points contained in the current environment image and the position information of the feature points successfully matched with the current environment image.

12. The method according to claim 11, wherein at least a part of image information in the scene image set is used as target image information of the scene image set, and the target image information is compared with the current environment image to obtain a comparison result corresponding to the to-be-determined sub-region, specifically including:

13. The method of claim 12, wherein determining the similarity between the pending subregion and the current environmental image comprises:

14. The method according to claim 12, wherein the step of taking at least part of the image information in the scene image set as target image information of the scene image set, and comparing the feature information included in the target image information with the feature information included in the current environment image specifically includes:

determining each view angle image set included in the scene image set, regarding each view angle image set, taking at least partial image information in the view angle image set as target image information of the view angle image set, comparing feature information included in the target image information with feature information included in the current environment image, and determining the similarity between the view angle image set and the current environment image;

determining the similarity between the to-be-determined sub-region and the current environment image according to the similarity between each view angle image set in the scene image set and the current environment image;

15. A visual positioning device, comprising:

a pending comparison module: for each undetermined sub-area, determining a scene image set corresponding to the undetermined sub-area according to a positioning map of the positioning area, taking at least partial image information in the scene image set as target image information of the scene image set, and comparing the target image information with the current environment image to obtain a comparison result corresponding to the undetermined sub-area; the positioning map corresponding to the positioning area is constructed in advance according to the map construction method of any one of claims 1 to 5;

16. A computer-readable storage medium, characterized in that the storage medium stores a computer program which, when executed by a processor, implements the method of any of the preceding claims 1-5 or 6-14.

17. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor when executing the program implements the method of any of claims 1 to 5 or 6 to 14.