WO2021129142A1

WO2021129142A1 - Building based outdoor positioning method, device and mobile equipment

Info

Publication number: WO2021129142A1
Application number: PCT/CN2020/125210
Authority: WO
Inventors: 崔倚瑞
Original assignee: 华为技术有限公司
Priority date: 2019-12-24
Filing date: 2020-10-30
Publication date: 2021-07-01
Also published as: CN111194015A

Abstract

The present application belongs to the field of artificial intelligence, and realizes auxiliary outdoor positioning using a computer vision method. The application proposes the method for positioning the mobile equipment based on the visual characteristics of the building contour line regarding the problem that the mobile equipment has positioning errors in a building dense area. Specifically, a building structure line extracting and repairing method is firstly proposed; then a descriptor for representing the characteristics of the building structural line is proposed to solve the problem that the traditional descriptor is not robust to illumination and shielding; and then, by taking the descriptor as a visual fingerprint, a fingerprint map construction algorithm based on a rendering three-dimensional model is proposed, the problem that the construction of a fingerprint map is time-consuming and labor-consuming is solved; finally a multiple sensors cooperative positioning method is proposed to improve the positioning efficiency. When a user uses the method, the user only needs to photograph any building, and the mobile equipment extracts the structural line of the building and calculates the visual characteristics of the structural line, and solves the accurate position of the user by matching with a fingerprint map and combining with multiple sensors.

Description

Building-based outdoor positioning method, device and mobile equipment

This application claims the priority of a Chinese patent application filed with the State Intellectual Property Office on December 24, 2019, the application number is 201911348363.3, and the application name is "Building-based outdoor positioning methods, devices and mobile equipment". The entire content of the application is approved The reference is incorporated in this application.

Technical field

This application belongs to the field of Artificial Intelligence (AI), which is specifically related to the field of outdoor positioning, and particularly relates to building-based outdoor positioning methods, devices, mobile devices, and computer-readable storage medium information.

Background technique

In a densely built area of a city, such as the CBD of a city, multiple bounces of GPS signals through buildings will cause a multipath effect (Multipath Effect), which causes large positioning errors in mobile devices such as mobile phones. Related research Shows that the error is between 60 meters and 300 meters. For mobile phone users, mobile phone positioning in the city center is a frequently used operation, and large positioning errors will seriously affect the user experience and cause inconvenience to users.

Summary of the invention

The embodiments of the present application provide building-based outdoor positioning methods, devices, mobile equipment, and computer-readable storage medium information, which can improve the accuracy of outdoor positioning of mobile equipment.

In the first aspect, an embodiment of the present application provides a building-based outdoor positioning method, including:

Acquiring a first image containing a first building; the first image is an image obtained by a user using a mobile device to photograph any surrounding buildings;

Extracting a first contour of the first building based on the first image, where the first contour is composed of at least two first structural lines;

Performing fusion processing on at least two first structure lines meeting a preset fusion condition in the first contour, to obtain a second contour composed of at least two second structure lines;

Obtaining a first feature vector corresponding to the second contour based on the second structure line;

Determine the current location of the user according to the matching result of the first feature vector and the second feature vector.

Exemplarily, the first image is an image obtained by a user taking pictures of any surrounding buildings through a mobile device.

The first contour is composed of at least two first structure lines.

It should be understood that the corresponding feature vector is obtained by extracting the structural lines in the building image, and matching it with the second feature vector, so as to determine the current location of the user according to the matched second feature vector, so that the user can be in a certain area or The densely-built area of the city can accurately locate the user's current location.

In a first possible implementation manner of the first aspect, determining the current location of the user according to a matching result of the first feature vector and the second feature vector includes:

Acquiring the positioning information of the user through a positioning sensor;

Using the positioning information as a center to determine a candidate set of buildings within a predetermined range from the center;

Acquiring a second feature vector corresponding to the candidate set from a feature vector database;

From the second feature vector corresponding to the candidate set, search for a second feature vector matching the first feature vector, and determine the current location of the user according to the found second feature vector.

It should be understood that by narrowing the range of buildings to be matched, that is, determining the candidate set of buildings in the area where the user is located according to the user's positioning information such as GPS positioning information, the amount of data of the second feature vector to be matched can be reduced, and the amount of The efficiency of feature vector matching is improved, and the positioning efficiency is correspondingly improved.

In a second possible implementation manner of the first aspect, the second feature vector corresponding to the candidate set is searched for a second feature vector matching the first feature vector, and according to the found The second feature vector determining the current location of the user includes:

Obtain the first direction angle α;

Using the first direction angle α as a reference, search for a second eigenvector within a range of the second direction angle α±β;

From the found second eigenvectors within the range of the second direction angle α±β, search for a second eigenvector that matches the first eigenvector, and determine the current location of the user by using a linear interpolation method.

It should be understood that by further reducing the data amount of the second feature vector to be matched by the first direction angle α, the matching efficiency of the feature vector is again improved, and accordingly, the positioning efficiency is also improved.

In a third possible implementation manner of the first aspect, in the second feature vector within the range of the second direction angle found from the search, a second feature vector that matches the first feature vector is searched for , Before determining the current position of the user by using a linear interpolation method, the method further includes:

The invalid second eigenvectors within the range of the second direction angle α±β are filtered out.

It should be understood that by filtering out the invalid second feature vector, the accuracy of the positioning information can be further improved.

Exemplarily, the preset fusion condition is that in the at least two first structure lines, the distance between two adjacent end points is less than a first value, and the distance between two adjacent first structure lines The vertical distance is smaller than the second value, and the slope difference is smaller than the third value.

Exemplarily, the extracting the first contour of the first building based on the first image includes:

A straight line detection algorithm is used to extract a first structural line of the first building from the first image, and the first structural line composes a first outline of the first building.

Exemplarily, the performing fusion processing on at least two first structure lines satisfying a preset fusion condition in the first contour to obtain a second contour composed of second structure lines includes:

At least two first structure lines that meet the preset fusion condition are fused and connected to form a second structure line.

It should be understood that the fusion processing of structural lines with obvious discontinuities enables them to accurately reflect the structure of the building, correspondingly improve the accuracy of feature vector extraction, and further reduce the accumulation rate of positioning errors, so as to achieve accurate positioning The purpose of the information.

In a fourth possible implementation manner of the first aspect, before the acquiring the first feature vector corresponding to the second contour based on the second structure line, the method further includes:

Normalization processing is performed on the second structure line, so that the line width of the second structure line after the normalization processing is a fixed number of pixels.

It should be understood that by setting the line width of the second structure line to a fixed number of pixels, the accumulation rate of positioning errors can be further reduced.

Exemplarily, the acquiring a first feature vector corresponding to the second contour based on the second structure line includes: acquiring a first response image made based on a second structure line in a different direction;

Use the HOG descriptor of the directional gradient histogram to perform feature description on the first response graph, and obtain the first feature vector corresponding to the second contour.

It should be understood that the use of the HOG descriptor to characterize the response image can reduce the influence of illumination and obstructions, improve the accuracy of the feature vector, thereby reduce the accumulation rate of positioning errors, and improve the accuracy of positioning.

In the second aspect, an embodiment of the present application provides a method for constructing a feature vector database, including:

Obtain a three-dimensional model of the target building;

Rendering the three-dimensional model to obtain a set of second images containing the second building;

Extracting a third contour of the building from each of the second images, where the third contour is composed of a third structural line;

Performing fusion processing on at least two third structure lines meeting preset fusion conditions in the third contour, to obtain a fourth contour composed of fourth structure lines;

Obtaining a second feature vector corresponding to the fourth contour based on the fourth structure line;

Based on the second feature vector and position information corresponding to each of the second images, a feature vector database is constructed.

It should be understood that each set of second images includes at least two images, and different three-dimensional models are rendered to obtain different sets of second images. The more second images rendered by each three-dimensional model, the more corresponding second feature vectors, so that the matched feature vectors can determine accurate position information and provide users with accurate positioning information.

Exemplarily, the performing fusion processing on at least two third structure lines meeting a preset fusion condition in the third contour to obtain a fourth contour composed of fourth structure lines includes:

At least two third structure lines that meet the preset fusion condition are fused and connected to form a fourth structure line.

In a first possible implementation manner of the second aspect, before the acquiring a second feature vector corresponding to the fourth contour based on the fourth structure line, the method further includes:

Normalization processing is performed on the fourth structure line, so that the line width of the fourth structure line after the normalization processing is a fixed number of pixels.

Exemplarily, the acquiring a second feature vector corresponding to the fourth contour based on the fourth structure line includes:

Acquire a second response image based on the fourth structure line in a different direction.

Use the HOG descriptor to perform feature description on the second response graph to obtain the second feature vector corresponding to the fourth contour.

Exemplarily, the preset fusion condition is that in the at least two third structure lines, the distance between two adjacent end points is less than a first value, and the distance between two adjacent third structure lines The vertical distance is smaller than the second value, and the slope difference is smaller than the third value.

Exemplarily, the constructing a feature vector database based on the second feature vector and position information corresponding to each of the second images includes:

Acquiring the position information and the third angle γ corresponding to each of the second images;

Establishing a mapping relationship between each second image and its corresponding second feature vector, position information, and third-direction angle γ, to obtain a mapping relationship corresponding to the set of second images;

A feature vector database is constructed based on the mapping relationship corresponding to the set of second images.

In the third aspect, an embodiment of the present application provides a building-based outdoor positioning device, including:

The first image acquisition unit is configured to acquire a first image containing a first building, where the first building is any building near the user's current location;

A first contour extraction unit, configured to extract a first contour of the first building based on the first image, where the first contour is composed of at least two first structure lines;

The first fusion unit is configured to perform fusion processing on at least two first structure lines meeting preset fusion conditions in the first contour, to obtain a second contour composed of at least two second structure lines;

A first feature vector obtaining unit, configured to obtain a first feature vector corresponding to the second contour based on the second structure line;

The positioning unit is configured to determine the current location of the user according to the matching result of the first feature vector and the second feature vector.

In a fourth aspect, an embodiment of the present application provides an apparatus for constructing a feature vector database, including:

The 3D model acquisition unit is used to acquire the 3D model of the target building;

The second image acquisition unit is configured to render the three-dimensional model to obtain a set of second images including the second building;

A third contour extraction unit, configured to extract a third contour of a building from each of the second images, where the third contour is composed of a third structure line;

The second fusion unit is configured to perform fusion processing on at least two third structure lines meeting preset fusion conditions in the third contour, to obtain a fourth contour composed of fourth structure lines;

A second feature vector obtaining unit, configured to obtain a second feature vector corresponding to the fourth contour based on the fourth structure line;

The feature vector database construction unit is configured to construct a feature vector database based on the second feature vector and position information corresponding to each of the second images.

In the fifth aspect, an embodiment of the present application provides a mobile device, including:

In a sixth aspect, an embodiment of the present application provides a cloud server, including:

Exemplarily, the feature vector database construction unit is specifically configured to:

Obtain the position information and the third angle γ corresponding to each of the second images, and establish the mapping relationship between each second image and its corresponding second feature vector, the position information and the third angle γ, and obtain the Describe the mapping relationship corresponding to a set of second images;

In a seventh aspect, an embodiment of the present application provides a computer-readable storage medium, including:

The first fusion unit is configured to perform fusion processing on at least two first structure lines meeting preset fusion conditions in the first contour to obtain a second contour composed of at least two second structure lines, and the preset The fusion condition is that the distance between two adjacent end points of the at least two first structure lines is less than a first value, and the vertical distance between two adjacent first structure lines is less than a second value, and The slope difference is less than the third value;

as well as:

A third contour extraction unit, configured to extract a third contour of a building from each of the second images, where the third contour is composed of a third structural line;

In an eighth aspect, embodiments of the present application provide a computer program product, which when the computer program product runs on a mobile device, causes the mobile device to execute the building-based outdoor positioning method described in any one of the above-mentioned first aspects. When the computer program product runs on the cloud server, the cloud server is caused to execute the method for constructing a feature vector database according to any one of the second aspects.

It can be understood that, for the beneficial effects of the third aspect to the eighth aspect described above, reference may be made to the related description in the first aspect and the second aspect described above, and details are not described herein again.

Compared with the prior art, the embodiment of the present application has the beneficial effect that by acquiring an image containing a building, and extracting the structural line corresponding to the building from the image containing the building, the After the structural line is fused, the feature vector corresponding to the outline of the building is extracted from the outline of the building obtained after the fusion process, and the user's location is determined according to the second feature vector that matches the feature vector to improve This improves the positioning accuracy of users in densely-built areas.

Description of the drawings

FIG. 1 is a schematic diagram of a structural line with obvious discontinuity problems provided by an embodiment of the present application;

FIG. 2 is a schematic diagram of two structural lines satisfying preset fusion conditions according to an embodiment of the present application;

FIG. 3 is a response image obtained by responding to structural lines in different directions in a contour provided by an embodiment of the present application;

4 is a specific implementation flowchart of a building-based outdoor positioning method provided by Embodiment 1 of the present application;

FIG. 5 is a schematic diagram of a feature vector database provided by Embodiment 1 of the present application;

FIG. 6 is a specific implementation flowchart of a method for finding a second feature vector matching the first feature vector according to Embodiment 1 of the present application;

FIG. 7 is a schematic diagram of searching for a second eigenvector matching the first eigenvector within the range of the second direction angle α±β according to the first embodiment of the present application;

FIG. 8 is an implementation flowchart of a method for constructing a feature vector database provided by Embodiment 2 of the present application;

9 is a schematic diagram of performing equidistant rendering on a three-dimensional model to obtain a set of second images according to Embodiment 2 of the present application;

FIG. 10 is a schematic diagram of a positioning interaction provided by Embodiment 3 of the present application;

FIG. 11 is a schematic diagram of a three-dimensional model of a building provided in Embodiment 3 of the present application;

FIGS. 12(a) to 12(d) are schematic diagrams of a set of rendered images obtained by equidistantly rendering the three-dimensional model in FIG. 11 from different positions and directions according to Embodiment 3 of the present application;

Figures 13(a) and 13(b) are schematic diagrams of the positioning results of a group of buildings from different angles, different distances, and different illumination provided in the third embodiment of the present application;

FIG. 14 is a schematic diagram of the average errors obtained by using different positioning methods under different lighting conditions for four groups provided in the third embodiment of the present application

15 is a schematic structural diagram of a building-based outdoor positioning device provided by Embodiment 4 of the present application;

FIG. 16 is a schematic structural diagram of an apparatus for constructing a feature vector database provided by Embodiment 5 of the present application;

FIG. 17 is a schematic structural diagram of a mobile device according to Embodiment 6 of the present application;

FIG. 18 is a schematic structural diagram of a cloud server provided in Embodiment 7 of the present application.

Detailed ways

In the following description, for the purpose of illustration rather than limitation, specific details such as a specific system structure and technology are proposed for a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid unnecessary details from obstructing the description of this application.

It should be understood that when used in the specification and appended claims of this application, the term "comprising" indicates the existence of the described features, wholes, steps, operations, elements and/or components, but does not exclude one or more other The existence or addition of features, wholes, steps, operations, elements, components, and/or collections thereof.

It should also be understood that the term "and/or" used in the specification and appended claims of this application refers to any combination of one or more of the associated listed items and all possible combinations, and includes these combinations.

As used in the description of this application and the appended claims, the term "if" can be construed as "when" or "once" or "in response to determination" or "in response to detecting ". Similarly, the phrase "if determined" or "if detected [described condition or event]" can be interpreted as meaning "once determined" or "in response to determination" or "once detected [described condition or event]" depending on the context ]" or "in response to detection of [condition or event described]".

In addition, in the description of the specification of this application and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the description, and cannot be understood as indicating or implying relative importance.

The reference to "one embodiment" or "some embodiments" described in the specification of this application means that one or more embodiments of this application include a specific feature, structure, or characteristic described in combination with the embodiment. Therefore, the sentences "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. appearing in different places in this specification are not necessarily All refer to the same embodiment, but mean "one or more but not all embodiments" unless it is specifically emphasized otherwise. The terms "including", "including", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized.

The building-based outdoor positioning method provided in the embodiments of the present application can be applied to mobile devices. The mobile device can be any device with image collection, sensor data collection and other functions, including but not limited to smart phones, smart home appliances, tablets, in-vehicle devices, wearable devices, and augmented reality (AR)/virtual reality devices (virtual reality, VR), etc. The building-based outdoor positioning method provided in this application can be specifically stored in the mobile device in the form of an application or software, and the mobile device can implement the building-based outdoor positioning method provided in this application by executing the application or software.

In the implementation of this application, the line segments in the building can reflect its structure, and can be detected by line detection algorithms such as Hough line detection algorithm, LSD (a Line Segment Detector) line detection algorithm, FLD (Fast Line Detector) line detection algorithm, etc. Detect the line segment in the building from the image containing the building, and extract the line segment based on the detection result, which is called the structural line of the building. The extracted line segment constitutes the outline of the building, that is, the outline of the building consists of at least It consists of two structural lines.

However, due to the slight gradient changes in the image, the detected and extracted line segments have obvious discontinuities as shown in Figure 1 (the selected part of the dashed frame). For the building, the discontinuous line segment does not reflect its structure. Therefore, it is necessary to merge the structural lines that are confirmed to have obvious discontinuities in the outline of the building to obtain the repaired ones without obvious discontinuities. Contours, such as the fusion and connection of two structural lines that have obvious discontinuities.

In this embodiment of the application, at least two structural lines that meet preset fusion conditions are fused and connected into one structural line to repair structural lines that have obvious discontinuities. The preset fusion condition referred to here is that of the at least two structural lines , The distance between two adjacent end points is less than the first value, the vertical distance between two adjacent structure lines is less than the second value, and the slope difference is less than the third value. For example, between two structural lines with obvious discontinuities, if the distance between two adjacent end points is less than the first value, such as 5mm, the vertical distance between two adjacent structure lines is less than the second value For example, for 1 mm or 5 pixels, if the absolute value of the difference between the slopes of two adjacent structure lines is less than at least one of the third values, the two adjacent structure lines are merged and connected to form a structure line. That is to say, the embodiment of the present application merges and connects at least two structural lines that are close in distance, close in endpoints, and close in slope into one structural line.

It should be noted that the vertical distance referred to here refers to the end point of any one of the two adjacent structural lines with obvious discontinuities as the vertical point, which is the vertical line that intersects with the extension of the other structural line. And take the distance of the vertical line with the largest distance value as the vertical distance. As shown, l _i and l _j 2 close to the endpoint, from two similar structures such as the existence of lines in the outline of the building structure in two adjacent lines, wherein, l _i and l _j adjacent two The distance between the two endpoints is d ₁ , d _{1 is} less than the first value, the vertical distance between _{l i} and l _j _{is d 2} , d _{2 is} less than the second value, and the slopes of the two are equal, that is, l _i and l _j The slope difference of is zero, that is, l _i and l _j meet the preset fusion conditions. At this time, l _i and l _j can be merged and connected to form a structural line.

In the embodiment of the present application, when there are multiple continuous structure lines with obvious discontinuity in the outline, if two adjacent structure lines meet the preset fusion condition, the two adjacent structure lines can be merged into A structure line, and then determine whether the fused structure line and the adjacent structure line meet the preset fusion condition, if it meets, then merge again; otherwise, the two adjacent first structure lines are deemed to have no obvious discontinuity The structural line of the problem does not need to be fused. As an example and not a limitation, the specific fusion operation refers to connecting the start point of the first structure line with the end point of the second structure line to form a new structure line, and delete the original two after fusing into a new structure line. Article structure line.

It should be noted that, due to the shooting angle, the slope of the structural line in the outline of the building can basically be calculated. The slope referred to here refers to the amount of a straight line with respect to the horizontal axis such as the degree of inclination of the ground. If the slope of a certain structure line cannot be calculated, the corresponding preset fusion condition is that the distance between the two adjacent end points of the at least two structure lines is less than the first value, and the two adjacent structures The vertical distance between the lines is less than the second value.

It should also be noted that the extracted structure lines of the building appear as line segments in different directions in the contour image. In the images taken from different angles, the distribution of these line segments in different directions is different. Therefore, the line segments need to be extracted. To obtain the response image in the corresponding direction, use the preset feature descriptor to describe the response image to obtain the corresponding feature vector.

In the embodiment of the present application, a Gabor filter is used to respond to structural lines in different directions in the contour, so as to obtain response images corresponding to different directions. For example, using Gabor filters in four directions (0°, 45°, 90°, 135°) to extract the directional characteristics of the structure line, four response images as shown in Figure 3 are obtained, and different directions correspond to different response images .

As shown in Figure 3, the first row of images is the direction used, and the second row of images is the corresponding response image. In this response image, the pixels in the corresponding direction have the highest brightness, as shown in Figure 3, which is highlighted in white. Part of the characteristic line.

Since the response image is a straight line in a specific direction, its directionality can be described by the distribution feature. The embodiment of this application uses the Histogram of Oriented Gradients (HOG) descriptor for the four response images shown in FIG. 3 Feature description, get the corresponding feature vector. The feature vector represents the distribution feature of line segments with the same direction feature in an image.

In the embodiments of the present application, the neural network model can also be used to respond to structural lines in different directions in the contour, so as to obtain response images corresponding to different directions. The neural network model referred to here includes but is not limited to convolutional neural network.

The application will be described in detail below in conjunction with specific embodiments.

Example one

Please refer to FIG. 4, which shows an implementation process of a building-based outdoor positioning method provided by an embodiment of the present application. The details are as follows:

In step S401, a first image containing a first building is acquired.

In the embodiment of the present application, the first building is any building near the user's current location (within a predetermined range from the user's current location). The first image is an image obtained by taking pictures of any surrounding buildings by the camera of the mobile device used by the user at the current location. In a specific embodiment of the present application, when the user uses a mobile device to locate in the CBD of the user, the camera of the mobile device is used to capture the surrounding environment to obtain one or several images, and the one or several images contain At least one building taken from any angle.

It should be noted that the shooting of the first image needs to be aimed at the center of the building. In addition to the direction angle of the shooting, there is no restriction on the shooting direction, and it is not required to capture the features of the specific building such as the skyline and the appearance due to the shooting angle. Intersecting lines such as the three numerical edge lines of a building, etc.

It should also be noted that there are no restrictions on the buildings captured in the first image. The buildings can be common tetrahedral structures (as shown in Figure 12) or unusual irregular structures (as shown in Figure 13).

In step S402, a first contour of the first building is extracted based on the first image.

In the embodiment of the present application, the first contour is composed of at least two first structure lines, and the first structure lines are line segments with the same or different directions.

As an example and not a limitation, step S402 specifically includes:

In the embodiment of the present application, the straight line detection algorithm includes, but is not limited to, the Hough straight line detection algorithm, the LSD straight line detection algorithm, and the FLD straight line detection algorithm.

In step S403, fusion processing is performed on at least two first structure lines satisfying a preset fusion condition among the first contours to obtain a second contour composed of second structure lines.

In the embodiment of the present application, the preset fusion condition is that the distance between two adjacent end points of the at least two first structure lines is less than a first value, and the two adjacent first structure lines The vertical distance between them is smaller than the second value, and the slope difference is smaller than the third value.

By fusing the at least two first structure lines in the first contour that meet the preset fusion conditions, that is, the structure lines that have obvious discontinuities, the purpose of repairing the first contour is achieved, and the second structure line composed of the second structure lines is obtained. The contour is the contour of the building after repairing the obvious discontinuity problem.

It should be noted that the second structure line includes but is not limited to the structure line obtained after the fusion process, that is, the second structure line includes the first structure line. Since in the first contour, not all the first structure lines meet the preset fusion condition, that is, there are some structure lines that do not need to be fused. Therefore, the second contour is the structure line after the fusion processing and the first structure line. An outline is composed of structural lines that do not need to be fused. For ease of description, the embodiment of the present application collectively refers to all the structure lines in the second contour as the second structure line.

As an example and not a limitation, step S403 specifically includes:

In the embodiment of the present application, the second structure line is the structure line obtained after the fusion process. Before this, no matter how many times the second structure line has been fused, the final structure line obtained by the fusion process is set to the second structure After the fusion process is completed, the first structural line that does not need to be fused and connected in the first contour is also set as the second structural line, and all the second structural lines constitute the second contour of the building.

In step S404, a first feature vector corresponding to the second contour is obtained based on the second structure line.

In the embodiment of the present application, the Gabor filter is used to respond to the structure lines in different directions in the second contour to obtain response images corresponding to different directions, for example, using four directions (0°, 45°, 90°, 135°). °) Gabor filter extracts the directional features of the second structure line. Alternatively, the convolutional neural network model is used to respond to structural lines in different directions in the second contour, so as to obtain response images corresponding to different directions.

It should be noted that the different directions here are not limited to the four directions of 0°, 45°, 90°, and 135°, but can also be directions from other angles, that is, the number of response images corresponding to different directions is not determined. Specific restrictions.

In a specific embodiment of the present application, before step S404, the method further includes:

In the embodiment of the present application, the width of the structure line obtained by the straight line detection algorithm is different, and the first feature vector obtained directly based on the structure line of different width may cause unnecessary errors and fail to pass the The first feature vector accurately describes the structure line of the building. Therefore, before the first feature vector corresponding to the second contour is obtained, the line pixels of the second structure line are normalized to make the normalized The line width of the second structure line is a fixed number of pixels.

In another specific embodiment of the present application, step S404 specifically includes:

Step S4041: Acquire first response images based on second structure lines in different directions.

In the embodiment of the present application, through the Gabor operator or the convolutional neural network model, it is possible to respond to the structural lines in different directions in the contour, so as to obtain the response image corresponding to the direction.

Step S4042: Use the HOG descriptor to perform feature description on the first response graph, and obtain the first feature vector corresponding to the second contour.

In the embodiment of the present application, the number of first feature vectors is the same as the number of response images, how many response images in different directions exist, and how many first feature vectors are corresponding, that is, the final first feature vector is multiple The result of splicing the first feature vector corresponding to the response image.

In step S405, the current location of the user is determined according to the matching result of the first feature vector and the second feature vector.

In the embodiment of the present application, the second feature vector is data stored in a feature vector database. Please refer to FIG. 5, which shows related data of a feature vector database.

It can be seen from FIG. 5 that the second feature vector and the location information corresponding to the second feature vector are stored in the feature vector database, and the location information is composed of latitude and longitude information. After obtaining the first feature vector corresponding to the second contour, the second feature vector that matches the feature vector can be found from the feature vector database, so that the location information of the mobile device can be determined, that is, according to the matched second feature vector Get the positioning result of the mobile device.

It should be noted that the feature vector database can be stored on the mobile device side, or on the cloud server side. Since the creation or update of the feature vector database requires a large amount of positioning data support and a large number of operations, the feature vector database is generally created on the cloud server side, or the feature vector database is updated. The cloud server synchronously sends the created or updated feature vector database to the mobile device side for storage, so that the mobile device side can call the second feature vector in the feature vector database to determine the positioning result of the mobile device.

It should also be noted that since there are a large number of second feature vectors in the feature vector database, if the global search method is used to find the second feature vector that matches the first feature vector, it will lead to inefficiency. In order to improve the search and the first feature vector For the efficiency of the second feature vector that matches the feature vector, the embodiment of the present application provides specific implementation steps of a method for finding a second feature vector that matches the first feature vector as shown in FIG. 6, which are detailed as follows:

In step S601, the positioning information of the user is acquired through a positioning sensor.

In the embodiment of the present application, when the user uses the mobile device to shoot the first building to obtain the first image including the first building, at the same time, the GPS positioning information obtained is the user's positioning through the positioning sensor of the mobile device, such as a GPS positioning sensor. information.

It should be noted that in densely-built areas, GPS signals will cause multipath effects after multiple reflections from buildings, which will lead to large positioning errors in GPS positioning information obtained by mobile devices, and the positioning errors are generally Between 60 meters and 100 meters, this will bring great changes to the user's travel. Therefore, the acquired GPS positioning information can only be used as rough position information, which needs to be corrected to obtain more accurate positioning results.

In step S602, a candidate set of buildings within a predetermined range from the center is determined with the positioning information as the center.

In the embodiment of the present application, after obtaining the GPS positioning information, taking the GPS positioning information as the center, obtain buildings within a predetermined range from the center as a candidate set of buildings in the area where the mobile device is located.

For example, taking GPS positioning information as the center of the circle, and taking buildings within a predetermined radius as a candidate set of buildings in the area where the mobile device is located. For another example, take GPS positioning information as the center to acquire buildings within a predetermined shape such as a square.

In step S603, a second feature vector corresponding to the candidate set is obtained from the feature vector database.

In the embodiment of the present application, by determining the candidate set of buildings in the area where the mobile device is located, the second feature vector of the area in the feature vector database is determined accordingly, thereby reducing the amount of search for the second feature vector and greatly improving The efficiency of finding the second feature vector that matches the first feature vector, thereby correspondingly improving the positioning efficiency.

In step S604, search for a second feature vector matching the first feature vector from the second feature vector corresponding to the candidate set, and determine the current location of the user according to the found second feature vector .

In the embodiment of the present application, the first feature vector is matched with any second feature vector of each building in the candidate set, such as the first second feature vector, so as to determine the building corresponding to the first image. That is, the building photographed by the mobile device is determined, and the positioning result of the mobile device is further determined according to the building.

Please refer to FIG. 7, which is a schematic diagram of searching for a second eigenvector matching the first eigenvector within the range of the second direction angle α±β according to an embodiment of the application, in order to further reduce the search for the second eigenvector To accurately obtain the positioning result of the mobile device, step S604 will be described in detail with reference to Fig. 7, which is specifically as follows:

Step S6041: Obtain the first direction angle α.

In the embodiment of the present application, the user obtains the first direction angle α through the built-in sensor of the mobile device, such as a posture sensor or a gyroscope, while capturing the first building to obtain the first image including the first building.

Step S6042, using the first direction angle α as a reference, search for a second eigenvector within the range of the second direction angle α±β.

In the embodiment of the present application, the second direction angle α±β is the sum/difference of the first direction angle α and the preset angle β. By further reducing the corresponding second feature vector in the range of the candidate set to the second feature vector in the range of the second direction angle α±β, the search amount of the second feature vector is reduced, thereby improving the search to match the first feature vector The efficiency of the second feature vector improves the efficiency of positioning.

Step S6043, from the found second eigenvectors within the range of the second direction angle α±β, search for a second eigenvector matching the first eigenvector, and use a linear interpolation method to determine the user’s current position.

In the embodiment of this application, it can be seen from Figure 6 that the building model shown in Figure 6 is the center of GPS positioning information, and the quadrilateral is a candidate set of buildings in the area where the mobile device is located, and the black dots on the quadrilateral The five-pointed star is the second eigenvector corresponding to the position, and the five-pointed star is the second eigenvector within the range of the second direction angle α±β. Obviously, from the second feature vector within the range of the second direction angle α±β Finding the second feature vector that matches the first feature vector within the vector, the search volume is greatly reduced.

In order to further obtain a more accurate second feature vector, thereby obtaining accurate positioning of the mobile device, before step S6043, the method further includes:

In the embodiment of the present application, the invalid second feature vector is specifically an area that cannot be reached by people or vehicles, such as roads that are prohibited from pedestrians or vehicles, such as lakes, pools, etc., which can be filtered through the map data of the mobile device. For the invalid second feature vector within the range of the second direction angle α±β, the map data can be obtained through the built-in map of the mobile device.

In the embodiment of the present application, by integrating GPS positioning information, the first direction angle α, and the built-in map of the mobile device, efficient and accurate positioning of the mobile device is realized, and the positioning accuracy of the user in densely-built areas is improved. It has high practicability and ease of use.

In a specific embodiment of the present application, after step S405, the method further includes:

Push the determined current location of the user to the user.

In the embodiment of the present application, by acquiring an image containing a building, and extracting the structural line corresponding to the building from the image containing the building, after performing the fusion processing on the structural line that meets the preset fusion condition, proceed from From the contour of the building obtained after the fusion process, the first feature vector corresponding to the building is extracted, and the user's location is determined according to the second feature vector matching the first feature vector. The first feature vector and the second feature vector are used to determine the location of the user. The feature vector realizes the accurate positioning of the user's current location in a certain area or city with dense buildings, and improves the positioning accuracy of the user in a certain area or city with dense buildings.

Example two

Please refer to FIG. 8. FIG. 8 shows an implementation process of a method for constructing a feature vector database provided by an embodiment of the present application. The details are as follows:

In step S801, a three-dimensional model of the target building is acquired.

In the embodiment of the present application, the three-dimensional model can quickly render a large number of images, so as to obtain enough second feature vectors to construct a feature vector database.

For the three-dimensional model, the three-dimensional model in Google Earth (GE) is selected in the embodiments of this application, which is mainly based on the following considerations:

Google Earth’s 3D model is completely open source;

2) The areas with large positioning errors of mobile devices are limited to high-rise buildings in a city, such as the urban CBD. The three-dimensional model in Google Earth basically covers the three-dimensional model of the buildings in the city center of major cities in the world.

3) Google Earth's 3D model is associated with Sketchup, the world's largest free 3D model library, and the number of 3D models is increasing and updating quickly.

Therefore, the three-dimensional model based on Google Earth can ensure the accuracy of the second feature vector obtained, thereby creating a feature vector database with more complete and accurate data.

In step S802, the three-dimensional model is rendered to obtain a set of second images containing second buildings.

In the embodiment of the present application, by rendering different three-dimensional models, a different set of second images including the second building is correspondingly obtained, and each set of second images includes at least two images.

It should be noted that different three-dimensional models correspond to different buildings, and the second buildings included in the second image obtained correspondingly are also different. The second building referred to here is an image of a different direction of the target building.

In a specific embodiment of the present application, after the three-dimensional model is obtained, the three-dimensional model is rendered at equal intervals from different angles and positions according to the preset rendering path and image sampling interval, and the camera's information is recorded while rendering. The position and attitude angle is the third angle γ, and a set of rendered images containing accurate position information is obtained.

Please refer to FIG. 9, which is a schematic diagram of rendering a three-dimensional model to obtain a set of second images according to an embodiment of the application.

As shown in Figure 9, when using a mobile device to shoot from different angles and different positions, the shape of the building in the photo is different. For example, the image obtained from the three positions A, B, and C in Figure 9 can use the building The structure line describes the shape. However, it is not realistic to manually collect images from different angles and different positions. Therefore, the method of rendering the three-dimensional model in computer graphics is used to obtain images rendered at different positions and different angles at equal distances, and build based on these rendered images. Feature vector database.

As shown in FIG. 9, the rendering path is actually the path of the camera movement, the black dots indicate the position of the camera (expressed as latitude and longitude) when the 3D model is rendered, and the interval of the dots is the interval at which the camera collects images. The point interval can be customized and is related to the required positioning accuracy. The smaller the point interval, the higher the positioning accuracy. After the rendering process is completed, each resulting rendered image contains accurate position information.

In step S803, the third contour of the building is extracted from each of the second images.

In the embodiment of the present application, the third contour is composed of a third structural line, and the structural line of the building is extracted from each second image through a straight line detection algorithm, and the extracted structural line forms the contour of the building, that is, the first Three contours. The line detection algorithm includes, but is not limited to, the Hough line detection algorithm, the LSD line detection algorithm, and the FLD line detection algorithm.

In step S804, fusion processing is performed on at least two third structure lines satisfying preset fusion conditions among the third contours to obtain a fourth contour composed of fourth structure lines.

In the embodiment of the present application, the preset fusion condition is that among the at least two third structure lines, the distance between the two adjacent end points is less than the first value, and the two adjacent third structure lines The vertical distance between them is smaller than the second value, and the slope difference is smaller than the third value.

By fusing at least two third structures in the third contour that meet the preset fusion conditions, that is, structural lines that have obvious discontinuities, the purpose of repairing the third contour is achieved, and the fourth contour composed of the fourth structural lines is obtained. That is, the outline of the building after the obvious discontinuity problem is repaired.

It should be noted that the fourth structure line includes but is not limited to the structure line obtained after the fusion process, that is, the fourth structure line includes the third structure line. Since in the third contour, not all the third structure lines meet the preset fusion condition, that is, there are some structure lines that do not need to be fused. Therefore, the fourth contour is the structure line after fusion processing and the first The three contours are composed of structural lines that do not need to be fused. For ease of description, the embodiment of the present application collectively refers to all the structure lines in the fourth contour as the fourth structure line.

As an example and not a limitation, step S804 specifically includes:

In the embodiment of the present application, the third structure line is the structure line obtained after the fusion process. Before this, no matter how many times the fourth structure line has been fused, the final structure line obtained by the fusion process is set to the fourth structure After the fusion processing is completed, the third structural line that does not need to be fused and connected in the third contour is also set as the fourth structural line, and all the fourth structural lines constitute the fourth contour of the building.

In step S805, a second feature vector corresponding to the fourth contour is acquired based on the fourth structure line.

In the embodiment of the present application, the Gabor filter or the convolutional neural network model is also used to respond to the structural lines in different directions in the fourth contour to obtain response images corresponding to different directions.

In a specific embodiment of the present application, before step S805, the method further includes:

In another specific embodiment of the present application, step S805 specifically includes:

Step S8051: Acquire a second response image based on the fourth structure line in a different direction.

Step S8052: Use the HOG descriptor to perform feature description on the second response graph, and obtain a second feature vector corresponding to the fourth contour.

In step S806, a feature vector database is constructed based on the second feature vector and location information corresponding to each of the second images.

In the embodiment of the present application, different second images correspond to different second feature vectors, and a set of second feature vectors corresponding to a set of second images is used as the second feature vector, which is related to the position information of the set of second images. The combination of data such as the third direction angle γ forms a feature vector database, so that the corresponding position information can be found according to the second feature vector.

It should be noted that after the second feature vector corresponding to each second image is set as the second feature vector, the position information and the third angle γ corresponding to each second image are acquired at the same time, and each second image is created The mapping relationship between the corresponding second feature vector, the position information and the third direction angle γ, and the feature vector database shown in FIG. 5 is constructed based on the mapping relationship corresponding to each second image.

It should also be noted that the primary key of the feature vector database is the second feature vector, and the matching location information can be found through the second feature vector.

Specifically, step S806 includes:

Step S8061: Obtain the position information and the third direction angle γ corresponding to each of the second images. In the embodiment of the present application, during the rendering process, the position information of the camera that collects each second image and the attitude angle of the camera, that is, the third direction angle γ, are recorded at the same time, so as to obtain the second feature vector corresponding to the second image. Then establish a mapping relationship with the position information and the third angle γ.

Step S8062: Establish a mapping relationship between each second image and its corresponding second feature vector, position information, and third-direction angle γ, to obtain a mapping relationship corresponding to the set of second images.

In the embodiment of the present application, by increasing the degree of matching between the third direction angle γ and the first direction angle, the positioning accuracy of the user's current position is further improved.

Step S8063: Construct a feature vector database based on the mapping relationship corresponding to the set of second images.

In the embodiment of the present application, the feature vector database constructed based on the mapping relationship corresponding to a set of second images contains the second feature vector, the position information corresponding to the second feature vector, and the associated direction angle information, That is, step S806 is specifically:

Based on the second feature vector, position information, and direction angle information corresponding to each of the second images, a feature vector database is constructed.

In the embodiment of the present application, several sets of second images are correspondingly obtained through a number of three-dimensional models, and the second feature vector is constructed based on the mapping relationship corresponding to the several sets of second images. By continuously acquiring new three-dimensional models, the feature vector can be The second feature vector in the database is further improved, so as to ensure the data perfection of the feature vector database.

In the embodiment of the present application, by obtaining a three-dimensional model of the target building, rendering the three-dimensional model, obtaining a set of second images containing the second building, and extracting the building’s image from each of the second images The third contour, the third contour is composed of third structure lines; the fusion processing is performed on at least two third structure lines that meet the preset fusion conditions in the third contour, to obtain the fourth structure line composed of the fourth structure lines. Contour; based on the fourth structure line, obtain the second feature vector corresponding to the fourth contour; based on the second feature vector and position information corresponding to each of the second images, construct a feature vector database, by comparing the three-dimensional model A large number of rendered images and feature vectors and location information corresponding to the rendered images are obtained by rendering, which improves the construction efficiency and accuracy of the feature vector database.

Example three

Corresponding to the building-based outdoor positioning method and feature vector database construction method described in the above embodiments, combined with the positioning interaction schematic diagram shown in FIG. 10, the positioning effect in Guangzhou Zhujiang New City is taken as an example for related description.

In the example of this application, the tested area of Zhujiang New City is 2,497,400m ² , including 12 buildings, and the test time is a time period when the light changes drastically: 16:00-18:00, and the mobile equipment used is the current market Smart phone, the smart phone can automatically obtain GPS positioning information when shooting, and the real location is manually marked according to the map.

Take the precise positioning of the Guangdong Museum in Zhujiang New Town as an example, follow the steps below:

a) The cloud server obtains the three-dimensional model of the Guangdong Museum as shown in Figure 11 from Goolge Earth;

b) The cloud server sets the rendering path and the interval of image collection, and renders the 3D model in Fig. 11 equidistantly from different positions and directions, and obtains a set as shown in Fig. 12(a) to Fig. 12(d) Render the image. During the rendering process, record the position and direction angle of the camera that collected the image at the same time;

c) The cloud server uses the method in the second embodiment to extract and repair the structural lines of the building in the rendered image;

d) The cloud server uses the method in the second embodiment to obtain the feature vector corresponding to each rendered image, that is, the second feature vector, and establish the mapping relationship between the position information of each rendered image, the second feature vector, and the attitude angle, that is, the direction angle, and The mapping relationship is stored in the feature vector database, and the feature vector database as shown in FIG. 5 is constructed.

e) The user takes photos of the Guangdong Museum through the camera of the smartphone;

f) The smart phone uses the method in the first embodiment to extract and repair the structural lines of the building in the captured image;

g) The smart phone uses the method in the first embodiment to extract the feature vector corresponding to the captured image;

h) The smart phone searches the feature vector database stored in the smart phone for a second feature vector that matches the feature vector based on the extracted feature vector, the smart phone’s GPS positioning information, the reading of the attitude sensor, and the map data. And through linear interpolation method to determine the user's current location.

or:

i) The smart phone sends the extracted feature vector, the smart phone’s GPS positioning information, the reading of the attitude sensor and the map data to the cloud service,

j) According to the information sent by the smart phone, the cloud server searches the feature vector database for a second feature vector that matches the feature vector in the smart phone, and determines the current location of the user through linear interpolation and returns the current location of the user to smart phone.

It can be seen that in a certain area or a densely built area of a city, the location information obtained by the GPS positioning sensor of the mobile device is corrected by combining the relevant data of the cloud server and the mobile device, so as to return accurate location information for the user.

The positioning accuracy provided by the embodiments of the present application is higher than the positioning accuracy of other existing positioning methods.

As shown in Figure 13 (a) and Figure 13 (b), Figure 13 (a) and Figure 13 (b) show the positioning results of a group of buildings from different angles, different distances and different illumination, where the first The row image represents the captured image, that is, the first image, and the second row image is the structure line extracted and repaired according to the first image corresponding to the first row after the obvious discontinuity problem, that is, the contour image composed of the second structure line. The three rows of images correspond to the structure lines extracted and repaired by the second image that matches the first image, that is, the contour image composed of the fourth structure line, and the last row is the positioning result provided by the embodiment of the application. Compared with the actual location of the user, "+" represents the positioning result provided by the embodiment of the present application, and "○" represents the actual location of the user.

It can be seen from FIG. 13 that the positioning result provided by the embodiment of the present application is very close to the actual position of the user. It can be seen that the embodiment of the present application has very high positioning accuracy.

As shown in FIG. 14, FIG. 14 shows a schematic diagram of the average positioning errors obtained by using different positioning methods in four groups under different lighting conditions.

It can be seen from Figure 14 that Figure 14 contains four sets of cumulative positioning errors obtained by using different positioning methods under different lighting conditions. The first positioning error in each group is the average positioning error of all test cases. , The second positioning error is the average error of the positioning error of the test case under sufficient light such as daytime, and the third positioning error is the average error of the positioning error of the test case under insufficient light such as the night.

In FIG. 14, the first set of average positioning errors is the average value of the positioning errors calculated based on the positioning information obtained by the GPS sensor of the mobile device and its real position; the second set of positioning errors is the positioning result and the real position provided by this embodiment of the application. The average value of the calculated positioning error; the third group is the average value of the positioning error provided by the existing positioning method 1 and the real position; the fourth group is the positioning result and the real position provided by the existing positioning method 2 The average value of the calculated positioning error. It can be seen from FIG. 14 that the average positioning error of the embodiment of the present application is far lower than the average positioning error of the other three groups, and the average positioning error under different lighting conditions is also much lower than the average positioning error of the remaining three groups. It can also be seen that the positioning result of this embodiment is not affected by the lighting conditions.

Among them, the first existing positioning method is specifically as follows:

This method proposes a positioning method based on point features. First, suppose that the photo taken by the user is located at a corner of the building. The photo contains three vertical edge lines of the building. Since the user always has an oblique angle when taking pictures, the three edge lines in the photo can intersect at one point. The location of the intersection of different photos is different. This feature is called the TICEP feature. According to the distribution of the intersection, several potential photo shooting points can be found on the 2-dimensional map where the building is located. Finally, the potential shooting point is determined by the self-defined TICEP feature, and the deviation function (deviation function) constructed based on this is used to find the matching feature to obtain the accurate position.

The second existing positioning method is specifically as follows:

This method combines SLAM and positioning, and proposes a point-based positioning method. First, the user takes a picture and obtains the angle and position information measured by the built-in sensor of the mobile phone, and uses this information to retrieve a 2.5-dimensional map (position information + height) to obtain the user's rough position in the 2.5-dimensional map. Then correct this rough position and use the following formula to estimate the precise position C of the camera:

Among them, l ₁ and l ₂ are the two vertical straight lines in the image, and x ₁ and x ₂ are the two points where the building and the ground intersect.

It can be seen that the area located in the existing positioning method 1 is limited to the area around the corners of the building, and the image of the building taken must include three vertical line segments of the building; in the existing positioning method 2, 2.5 A three-dimensional map assists positioning, and the captured image of the building must include the intersection of the building and the ground. In the embodiment of this application, there is no restriction on the captured image of the building, so that the user can be at any location. Accurate positioning information is obtained through the embodiments of this application.

It should be understood that the size of the sequence number of each step in the foregoing embodiment does not mean the order of execution. The execution sequence of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application.

Example four

Corresponding to the building-based outdoor positioning method described in the above embodiment, FIG. 15 shows a structural block diagram of the building-based outdoor positioning device provided by an embodiment of the present application. The relevant part of the embodiment.

Referring to Figure 15, the device includes:

The first image acquisition unit 151 is configured to acquire a first image containing a first building, where the first building is any building near the user's current location;

The first contour extraction unit 152 is configured to extract a first contour of the first building based on the first image, where the first contour is composed of at least two first structural lines;

The first fusion unit 153 is configured to perform fusion processing on at least two first structure lines meeting preset fusion conditions in the first outline, to obtain a second outline composed of at least two second structure lines;

The first feature vector obtaining unit 154 is configured to obtain a first feature vector corresponding to the second contour based on the second structure line;

The positioning unit 155 is configured to determine the current location of the user according to the matching result of the first feature vector and the second feature vector.

In a possible implementation manner, the positioning unit 155 includes:

The location information acquisition subunit is used to acquire the location information of the user through a location sensor;

A candidate set determining subunit, used to determine a candidate set of buildings within a predetermined range from the center with the positioning information as the center;

A second feature vector obtaining subunit, configured to obtain a second feature vector corresponding to the candidate set from a feature vector database;

The positioning subunit is used to search for a second feature vector matching the first feature vector from the second feature vector corresponding to the candidate set, and determine the current user’s current feature vector based on the found second feature vector position.

In a possible implementation manner, the positioning subunit is specifically configured to:

Obtain the first direction angle α;

In another possible implementation manner, the positioning subunit is specifically further used for:

Specifically, the preset fusion condition is that in the at least two first structure lines, the distance between two adjacent end points is less than a first value, and the vertical distance between two adjacent first structure lines is The distance is smaller than the second value, and the slope difference is smaller than the third value.

As an example and not a limitation, the first contour extraction unit 152 is specifically configured to:

As an example and not a limitation, the first fusion unit 153 is specifically configured to:

In a possible implementation manner, the device further includes:

The second structure line normalization processing unit is configured to perform a normalization process on the second structure line, so that the line width of the second structure line after the normalization process is a fixed number of pixels.

As an example and not a limitation, the first feature vector acquiring unit 154 is specifically configured to:

Acquiring first response images based on second structure lines in different directions;

In this embodiment of the application, by acquiring an image containing a building, and extracting the first structural line corresponding to the building from the image containing the building, the first structural line that meets the preset fusion condition is fused. Then, from the contour of the building obtained after the fusion process, the first feature vector corresponding to the building is extracted, and the user's location is determined according to the second feature vector matching the first feature vector, and the first feature vector is used to determine the location of the user. The vector and the second feature vector realize accurate positioning of the user's current location in the densely-built area, and improve the positioning accuracy of the user in the densely-built area.

Example five

Corresponding to the method for constructing a feature vector database described in the above embodiment, FIG. 16 shows a structural block diagram of the device for constructing a feature vector database provided by an embodiment of the present application. The relevant part.

Referring to Figure 16, the device includes:

The three-dimensional model obtaining unit 161 is configured to obtain a three-dimensional model of the target building;

The second image obtaining unit 162 is configured to render the three-dimensional model to obtain a set of second images including the second building;

The third contour extraction unit 163 is configured to extract a third contour of the building from each of the second images, where the third contour is composed of third structural lines;

The second fusion unit 162 is configured to perform fusion processing on at least two third structure lines meeting preset fusion conditions in the third contour, to obtain a fourth contour composed of fourth structure lines;

The second feature vector obtaining unit 165 is configured to obtain a second feature vector corresponding to the fourth contour based on the fourth structure line;

The feature vector database construction unit 166 is configured to construct a feature vector database based on the second feature vector and position information corresponding to each of the second images.

The second fusion unit 162 is specifically configured to:

In a possible implementation manner, the device further includes:

The fourth structure line normalization processing unit is configured to perform normalization processing on the fourth structure line, so that the line width of the fourth structure line after the normalization processing is a fixed number of pixels.

The second feature vector obtaining unit 165 is specifically configured to:

Exemplarily, the feature vector database construction unit 166 is specifically configured to:

It should be noted that the information interaction and execution process between the above-mentioned devices/units are based on the same concept as the method embodiment of this application, and its specific functions and technical effects can be found in the method embodiment section. I won't repeat it here.

Those skilled in the art can clearly understand that for the convenience and conciseness of description, only the division of the above functional units and modules is used as an example. In practical applications, the above functions can be allocated to different functional units and modules as required. Module completion, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist alone physically, or two or more units can be integrated into one unit. The above-mentioned integrated units can be hardware-based Formal realization can also be realized in the form of a software functional unit. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the protection scope of the present application. For the specific working process of the units and modules in the foregoing system, reference may be made to the corresponding process in the foregoing method embodiment, which will not be repeated here.

Example Six

FIG. 17 is a schematic structural diagram of a mobile device provided by an embodiment of this application. As shown in FIG. 17, the mobile device 17 of this embodiment includes: at least one processor 170 (only one is shown in FIG. 17), a processor, a memory 171, and a processor that is stored in the memory 171 and can be processed in the at least one processor. A computer program 172 running on the processor 170, when the processor 170 executes the computer program 172, the steps in any of the foregoing building-based outdoor positioning method embodiments are implemented. Alternatively, when the processor 170 executes the computer program 172, the functions of the units in the foregoing device embodiments, for example, the functions of the units 151 to 155 shown in FIG. 15 are realized.

The mobile device 17 may be a computing device such as a mobile phone, a desktop computer, a notebook, and a palmtop computer. The mobile device 17 may include, but is not limited to, a processor 170 and a memory 171. Those skilled in the art can understand that FIG. 17 is only an example of the mobile device 17 and does not constitute a limitation on the mobile device 17. It may include more or less components than those shown in the figure, or a combination of certain components, or different components. , For example, can also include input and output devices, network access devices, and so on.

The so-called processor 170 may be a central processing unit (Central Processing Unit, CPU), and the processor 170 may also be other general-purpose processors, digital signal processors (Digital Signal Processors, DSPs), and application specific integrated circuits (Application Specific Integrated Circuits). , ASIC), ready-made programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor or the like.

The memory 171 may be an internal storage unit of the mobile device 17 in some embodiments, such as a hard disk or a memory of the electronic mobile sub-device 17. In other embodiments, the memory 171 may also be an external storage device of the mobile device 17, such as a plug-in hard disk equipped on the mobile device 17, a smart media card (SMC), and a secure digital (Secure Digital, SD) card, Flash Card, etc. Further, the memory 171 may also include both an internal storage unit of the mobile device 17 and an external storage device. The memory 171 is used to store an operating system, an application program, a boot loader (BootLoader), data, and other programs, such as the program code of the computer program. The memory 171 can also be used to temporarily store data that has been output or will be output. The embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program. When the processor is executed, the steps in the foregoing method embodiments can be realized.

Example Seven

FIG. 18 is a schematic structural diagram of a cloud server provided by an embodiment of the application. As shown in FIG. 18 and FIG. 18, the cloud server 18 of this embodiment includes: at least one processor 180 (only one is shown in FIG. 18), a processor, a memory 181, and a memory 181 stored in the memory 181 and available in the at least one processor. A computer program 182 running on the processor 180, when the processor 180 executes the computer program 182, the steps in any of the above-mentioned embodiments of the building-based outdoor positioning method are implemented. Alternatively, when the processor 180 executes the computer program 182, the functions of the units in the foregoing device embodiments, for example, the functions of the units 161 to 166 shown in FIG. 16 are realized.

The cloud server 18 may be a computing device such as a mobile phone, a desktop computer, a notebook, a palmtop computer, and a cloud server. The cloud server 18 may include, but is not limited to, a processor 180 and a memory 181. Those skilled in the art can understand that FIG. 18 is only an example of the cloud server 18, and does not constitute a limitation on the cloud server 18. It may include more or less components than shown in the figure, or a combination of certain components, or different components. , For example, can also include input and output devices, network access devices, and so on.

The so-called processor 180 may be a central processing unit (Central Processing Unit, CPU), and the processor 180 may also be other general-purpose processors, digital signal processors (Digital Signal Processors, DSPs), and application specific integrated circuits (Application Specific Integrated Circuits). , ASIC), ready-made programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor or the like.

The memory 181 may be an internal storage unit of the cloud server 18 in some embodiments, such as a hard disk or memory of the cloud server 18. In other embodiments, the memory 181 may also be an external storage device of the cloud server 18, such as a plug-in hard disk equipped on the cloud server 18, a smart media card (SMC), and a secure digital (Secure Digital, SD) card, Flash Card, etc. Further, the storage 181 may also include both an internal storage unit of the cloud server 18 and an external storage device. The memory 181 is used to store an operating system, an application program, a boot loader (BootLoader), data, and other programs, such as the program code of the computer program. The memory 181 can also be used to temporarily store data that has been output or will be output. The embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program. When the processor is executed, the steps in the foregoing method embodiments can be realized.

The embodiments of the present application provide a computer program product. When the computer program product runs on a mobile device, the steps in the foregoing method embodiments can be realized when the mobile device is executed.

If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the implementation of all or part of the processes in the above-mentioned embodiment methods in the present application can be accomplished by instructing relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. The computer program can be stored in a computer-readable storage medium. When executed by the processor, the steps of the foregoing method embodiments can be implemented. Wherein, the computer program includes computer program code, and the computer program code may be in the form of source code, object code, executable file, or some intermediate forms. The computer-readable medium may include at least: any entity or device capable of carrying computer program code to a photographing device/mobile device, recording medium, computer memory, read-only memory (ROM, Read-ONly MeMory), random access memory ( RAM, RaNdoM Access MeMory), electric carrier signal, telecommunications signal and software distribution medium. For example, U disk, mobile hard disk, floppy disk or CD-ROM, etc. In some jurisdictions, according to legislation and patent practices, computer-readable media cannot be electrical carrier signals and telecommunication signals.

In the above-mentioned embodiments, the description of each embodiment has its own focus. For parts that are not described in detail or recorded in an embodiment, reference may be made to related descriptions of other embodiments.

A person of ordinary skill in the art may realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be implemented by electronic hardware or a combination of computer software and electronic hardware. Whether these functions are executed by hardware or software depends on the specific application and design constraint conditions of the technical solution. Professionals and technicians can use different methods for each specific application to implement the described functions, but such implementation should not be considered beyond the scope of this application.

In the embodiments provided in this application, it should be understood that the disclosed apparatus/network equipment and method may be implemented in other ways. For example, the device/network device embodiments described above are only illustrative. For example, the division of the modules or units is only a logical function division, and there may be other divisions in actual implementation, such as multiple units. Or components can be combined or integrated into another system, or some features can be omitted or not implemented. In addition, the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments.

The above-mentioned embodiments are only used to illustrate the technical solutions of the present application, not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, a person of ordinary skill in the art should understand that it can still implement the foregoing The technical solutions recorded in the examples are modified, or some of the technical features are equivalently replaced; these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the application, and should be included in Within the scope of protection of this application.

Claims

A building-based outdoor positioning method, which is characterized in that it includes:

Acquiring a first image containing a first building, where the first image is an image obtained by a user using a mobile device to photograph any surrounding buildings;

Extracting a first contour of the first building based on the first image, where the first contour is composed of at least two first structural lines;

Performing fusion processing on at least two first structure lines meeting a preset fusion condition in the first contour, to obtain a second contour composed of at least two second structure lines;

Obtaining a first feature vector corresponding to the second contour based on the second structure line;

Determine the current location of the user according to the matching result of the first feature vector and the second feature vector.
The method of claim 1, wherein determining the current location of the user according to a matching result of the first feature vector and the second feature vector comprises:

Acquiring the positioning information of the user through a positioning sensor;

Using the positioning information as a center to determine a candidate set of buildings within a predetermined range from the center;

Acquiring a second feature vector corresponding to the candidate set from a feature vector database;

From the second feature vector corresponding to the candidate set, search for a second feature vector matching the first feature vector, and determine the current location of the user according to the found second feature vector.
The method according to claim 2, wherein the second feature vector corresponding to the candidate set is searched for a second feature vector matching the first feature vector, and the second feature vector is searched according to the found first feature vector. The two feature vectors determining the current location of the user include:

Obtain the first direction angle α;

Using the first direction angle α as a reference, search for a second eigenvector within a range of the second direction angle α±β, where β is the preset angle;

From the found second eigenvectors in the range of the second direction angle, search for a second eigenvector matching the first eigenvector, and determine the current position of the user by using a linear interpolation method.
The method according to claim 3, wherein, in the second eigenvectors within the range of the second direction angle found from the search, searching for a second eigenvector that matches the first eigenvector, Before using the linear interpolation method to determine the current location of the user, the method further includes:

The invalid second eigenvectors within the range of the second direction angle α±β are filtered out.
The method according to any one of claims 1 to 4, wherein the preset fusion condition is that the distance between two adjacent end points of the at least two first structure lines is less than a first value , The vertical distance between two adjacent first structure lines is smaller than the second value, and the slope difference is smaller than the third value.
The method according to claim 1, wherein said extracting a first contour of said first building based on said first image comprises:

A straight line detection algorithm is used to extract a first structural line of the first building from the first image, and the first structural line composes a first outline of the first building.
The method according to claim 1, wherein the fusion processing is performed on at least two first structure lines satisfying preset fusion conditions in the first contour to obtain a second contour composed of second structure lines include:

At least two first structure lines that meet the preset fusion condition are fused and connected to form a second structure line.
The method according to claim 1, characterized in that, before said obtaining the first feature vector corresponding to the second contour based on the second structure line, the method further comprises:

Normalization processing is performed on the second structure line, so that the line width of the second structure line after the normalization processing is a fixed number of pixels.
The method according to claim 1, wherein the obtaining a first feature vector corresponding to the second contour based on the second structure line comprises: obtaining a second structure line based on a different direction. First response image;

Use the HOG descriptor of the directional gradient histogram to perform feature description on the first response graph, and obtain the first feature vector corresponding to the second contour.
A method for constructing a feature vector database, which is characterized in that it includes:

Obtain a three-dimensional model of the target building;

Rendering the three-dimensional model to obtain a set of second images containing the second building;

Extracting a third contour of the building from each of the second images, where the third contour is composed of a third structural line;

Performing fusion processing on at least two third structure lines meeting preset fusion conditions in the third contour, to obtain a fourth contour composed of fourth structure lines;

Obtaining a second feature vector corresponding to the fourth contour based on the fourth structure line;

Based on the second feature vector and position information corresponding to each of the second images, a feature vector database is constructed.
The method according to claim 10, wherein the fusion processing is performed on at least two third structure lines satisfying preset fusion conditions among the third contours to obtain a fourth contour composed of fourth structure lines include:

At least two third structure lines that meet the preset fusion condition are fused and connected to form a fourth structure line.
The method according to claim 10, wherein before said acquiring a second feature vector corresponding to said fourth contour based on said fourth structure line, the method further comprises:

Normalization processing is performed on the fourth structure line, so that the line width of the fourth structure line after the normalization processing is a fixed number of pixels.
The method according to claim 10, wherein said acquiring a second feature vector corresponding to said fourth contour based on said fourth structure line comprises:

Acquiring a second response image based on a fourth structure line in a different direction;

Use the HOG descriptor to perform feature description on the second response graph to obtain the second feature vector corresponding to the fourth contour.
The method according to claim 10, wherein the preset fusion condition is that in the at least two third structural lines, the distance between two adjacent end points is less than a first value, and the adjacent The vertical distance between the two third structure lines is smaller than the second value, and the slope difference is smaller than the third value.
The method according to any one of claims 10 to 14, wherein the constructing a feature vector database based on the second feature vector and position information corresponding to each of the second images comprises:

Acquiring the position information and the third angle γ corresponding to each of the second images;

Establishing a mapping relationship between each second image and its corresponding second feature vector, position information, and third-direction angle γ, to obtain a mapping relationship corresponding to the set of second images;

A feature vector database is constructed based on the mapping relationship corresponding to the set of second images.
A building-based outdoor positioning device, which is characterized in that it comprises:

The first image acquisition unit is configured to acquire a first image containing a first building, where the first building is any building near the user's current location;

A first contour extraction unit, configured to extract a first contour of the first building based on the first image, where the first contour is composed of at least two first structure lines;

The first fusion unit is configured to perform fusion processing on at least two first structure lines meeting preset fusion conditions in the first contour, to obtain a second contour composed of at least two second structure lines;

A first feature vector obtaining unit, configured to obtain a first feature vector corresponding to the second contour based on the second structure line;

The positioning unit is configured to determine the current location of the user according to the matching result of the first feature vector and the second feature vector.
A device for constructing a feature vector database, which is characterized in that it comprises:

The 3D model acquisition unit is used to acquire the 3D model of the target building;

The second image acquisition unit is configured to render the three-dimensional model to obtain a set of second images including the second building;

A third contour extraction unit, configured to extract a third contour of a building from each of the second images, where the third contour is composed of a third structural line;

The second fusion unit is configured to perform fusion processing on at least two third structure lines meeting preset fusion conditions in the third contour, to obtain a fourth contour composed of fourth structure lines;

A second feature vector obtaining unit, configured to obtain a second feature vector corresponding to the fourth contour based on the fourth structure line;

The feature vector database construction unit is configured to construct a feature vector database based on the second feature vector and position information corresponding to each of the second images.
A mobile device comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, wherein the processor executes the computer program as claimed in claims 1 to 9. The method of any one of.
A cloud server, comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, wherein the processor executes the computer program as claimed in claims 10 to 15. The method of any one of.
A computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, wherein the computer program implements any one of claims 1 to 9 and/or 10 to 15 when the computer program is executed by a processor. The method described.