CN106793086B - Indoor positioning method - Google Patents

Abstract

The present invention is an indoor positioning method, which relates to the wireless communication network technology for the purpose of network management. It adopts a method combining WiFi fingerprint-based positioning and sign-based visual positioning. First, the WiFi location fingerprint positioning algorithm is used to obtain the WiFi positioning range and WiFi Positioning coordinates, and then obtain visual positioning coordinates according to the feature matching and visual positioning of the tested image, and finally combine WiFi location fingerprint positioning with visual positioning to achieve high-precision indoor positioning. The method overcomes the defects of low precision existing in the existing WiFi fingerprint positioning technology and the existing single visual positioning method technology is not suitable for indoor positioning.

Description

An indoor positioning method

technical field

The technical scheme of the present invention relates to a wireless communication network technology for the purpose of network management, in particular to an indoor positioning method.

Background technique

Indoor positioning refers to the realization of position positioning in the indoor environment. It mainly uses wireless communication, base station positioning, inertial navigation positioning and other technologies to integrate to form an indoor position positioning system, so as to realize the position monitoring of people and objects in indoor space. Due to the complex and changeable indoor environment and the inability to receive GPS (Global Positioning System) signals, it is difficult to locate indoors at present. When satellite positioning cannot be used in the indoor environment, indoor positioning technology is used as an auxiliary positioning for satellite positioning to solve the problem that the satellite signal is weak and cannot penetrate buildings when it reaches the ground, and finally locates the current position of the object.

Judging from the currently published literature and technical means, the indoor positioning technologies that have been developed more and more include: Wi-Fi technology, which is based on WLAN (Wireless Local Area Network) indoor positioning, and requires wireless access points to be arranged in advance. AP (Access Point), which will cause waste of resources when there is no positioning requirement; UWB technology is based on UWB (Ultra Wide Band ultra-wideband technology) indoor positioning. At present, at least three signal receivers are required, and the receiver and transmitter are There can be no obstacles between them; the inertial navigation positioning technology is based on the indoor positioning of inertial sensors. Because the inertial sensors are affected by the noise of the micro-electromechanical system, the positioning accuracy is inevitably affected.

CN103402256B discloses an indoor positioning method based on Wi-Fi fingerprints; CN106304331A discloses a WiFi fingerprint indoor positioning method; CN103582119B announces a fingerprint database construction method of Wi-Fi indoor positioning system. According to the published literature on WiFi positioning technology, the positioning technology based on WiFi fingerprints all use MAC address and signal strength RSSI value to construct fingerprints. The fingerprint database constructed by this method is complex, and the positioning accuracy is easily affected by indoor environment changes. .

CN106295512A discloses an identification-based multi-correction line indoor visual database construction method and an indoor positioning method. The method is based on a camera and realizes positioning and navigation by retrieving images with specific identifications. This is difficult to achieve in a real indoor environment and needs to be A series of image sets with the same logo are arranged indoors, and the indoor environment needs to be changed; CN106228538A discloses a logo-based binocular vision indoor positioning method. The positioning process requires two cameras to obtain images, and the cameras are used to locate the target point, which involves The calibration of the internal and external parameters of the camera and the conversion between the camera coordinate system, the image coordinate system and the world coordinate system are generally used in the field of 3D positioning of mobile robots, and it is difficult to popularize and apply to the general public. In conclusion, the existing single visual localization method technology is not suitable for indoor localization.

To sum up, there is currently no economical and mature indoor positioning technology. With the rapid development of key technologies such as the Internet, wireless communication, computer technology, surveying and mapping technology and equipment manufacturing, indoor positioning will develop towards the complementary combination of different indoor positioning technologies. The disadvantage of a certain indoor positioning method is compensated by the complementary combination of different indoor positioning technologies. How to organically combine various indoor positioning technologies will be a research hotspot in this technical field.

SUMMARY OF THE INVENTION

The technical problem to be solved by the present invention is: to provide an indoor positioning method, which adopts a method combining WiFi fingerprint-based positioning and sign-based visual positioning to achieve high-precision indoor positioning, and the method overcomes the existing WiFi fingerprint positioning. The technology has low precision and the existing single visual positioning method technology is not suitable for indoor positioning defects.

The technical solution adopted by the present invention to solve the technical problem is as follows: an indoor positioning method, which adopts a method combining WiFi fingerprint-based positioning and sign-based visual positioning. First, the WiFi location fingerprint positioning algorithm is used to obtain the WiFi positioning range and WiFi Positioning coordinates, and then obtain the visual positioning coordinates according to the feature matching and visual positioning of the tested image, and finally combine the WiFi location fingerprint positioning with the visual positioning. The specific steps are as follows:

The first step is to generate a WiFi fingerprint:

Develop an Android App based on Java language, obtain the MAC address of the WiFi signal, and generate a txt file and save it in the smartphone, thereby generating WiFi fingerprints;

The second step is to build a WiFi location fingerprint database:

The corridor area in the indoor environment is selected as the positioning area. In the corridor area, 30 to 60 WiFi sampling points are selected. The coordinates of each sampling point are known, and the installed App is used to detect the detectable points at each sampling point. WiFi signal, obtain its MAC address sequence and save it as a fingerprint in the WiFi location fingerprint database. , set 60 WiFi sampling points, use the MAC address collected at each sampling point as the fingerprint of the sampling point, traverse all sampling points to get 60 fingerprints, and save them into the WiFi location fingerprint database, thus completing the construction of WiFi location fingerprint database;

The third step, WiFi fingerprint positioning:

利用指纹匹配算法依照上述第二步所述的每一个指纹对应唯一的位置信息的规则，将这个实测指纹xf与上述第二步中的WiFi位置指纹数据库中的指纹进行匹配，得到匹配度最高的三个指纹，从而得到WiFi定位范围(x₀～x₁,y₀～y₁)及WiFi定位坐标(x_w,y_w)，实现WiFi指纹定位，其中的x₀、x₁为待定位点的横坐标，x₀～x₁为待定位的横坐标范围，单位为m；其中的y₀、y₁为待定位点的纵坐标，y₀～y₁为待定位的纵坐标范围，单位为m；In the positioning stage, in the positioning area selected in the second step above, set the position coordinates of the to-be-located point x as (x, y), and N _x WiFi signals can be received. The App detection in the first step is used to obtain this The measured fingerprint of the anchor point x

The fingerprint matching algorithm is used to match the measured fingerprint xf with the fingerprints in the WiFi location fingerprint database in the second step above according to the rule that each fingerprint corresponds to unique location information described in the second step above, and the highest matching degree is obtained. Three fingerprints are obtained to obtain the WiFi positioning range (x ₀ ~ x ₁ , y ₀ ~ y ₁ ) and WiFi positioning coordinates (x _w , y _w ) to realize WiFi fingerprint positioning, where x ₀ and x ₁ are the points to be located. x ₀ ~ x ₁ is the abscissa range to be positioned, and the unit is m; y ₀ and y ₁ are the vertical coordinates of the point to be positioned, and y ₀ ~ y ₁ are the vertical coordinate range to be positioned, in units of is m;

The fourth step is to generate a training image set:

In the positioning area of the second step above, the coordinates of all the house numbers are known, and these house numbers are the sign sampling points, that is, the points to be located. Use the smartphone to shoot the house number, traverse all the sign sampling points, and generate a training image set. Since the marker sampling point is a part of the WiFi sampling point, the coordinates of the marker sampling point are known, and the coordinates of the training image are the coordinates of the marker sampling point;

The fifth step is to calculate the SURF global feature descriptor:

First, the training images obtained in the fourth step above are preprocessed, including normalization and grayscale processing, and then the SURF global feature descriptor is calculated, including two parts, the first part is the feature point location, and the second part is the feature description Symbol calculation, the center point of the normalized image is used as the feature point, the whole image is regarded as a single neighborhood of the feature point, and the feature descriptor calculated from this is used as the SURF global feature description of the whole image. symbol;

The sixth step is to calculate the ORB global feature descriptor:

(6.1) Determine the main direction of the feature point:

其中I(x,y)是点(x,y)处的灰度值，该特征点的邻域图像的质心

邻域图像的质心与特征点的夹角为θ＝arctan2(m₀₁,m₁₀)，即为特征点的主方向；In the same way as in the fifth step above, the center point of the normalized image is used as the feature point, and the main direction of the feature point is calculated by using the image moment. For any feature point, its image moment is

where I(x,y) is the gray value at point (x,y), the centroid of the neighborhood image of this feature point

The angle between the centroid of the neighborhood image and the feature point is θ=arctan2(m ₀₁ , m ₁₀ ), which is the main direction of the feature point;

(6.2) Generate BRIEF feature descriptor:

其中，p1(x)表示图像邻域p1上一点x处的强度，p1(y)表示图像邻域p1上一点y处的强度,经过n次二进制测试，得到一个n维向量，即BRIEF特征描述子

这里选取n＝256，得到的是256位的二进制字符串；The generation process of the BRIEF feature descriptor is as follows: p1 represents a smoothed image neighborhood, and the binary test of x and y at any position is the logical result of the intensity test of these two points:

Among them, p1(x) represents the intensity of a point x on the image neighborhood p1, and p1(y) represents the intensity of a point y on the image neighborhood p1. After n binary tests, an n-dimensional vector is obtained, that is, the Brief feature description son

Here n=256 is selected, and a 256-bit binary string is obtained;

(6.3) Calculate the ORB global feature descriptor:

由上述(6.1)步确定的特征点的主方向计算出仿射变换矩阵

由此得到S_θ＝R_θS，S_θ即为具有旋转不变性的BRIEF特征描述子，最后计算出具有旋转不变性的ORB全局特征描述符：g_n(P,θ):＝f_n(P)|(x_i,y_i)∈S_θ，这里选取n＝256；In order to make the Brief feature descriptor have rotation invariance, the orientation of the Brief feature descriptor is set according to the direction of the feature points determined in the above step (6.1), and the n-bit binary test set for any image pixel point (x _i , y _i ) is obtained. The feature set of , defines a 2×n matrix

Calculate the affine transformation matrix from the main direction of the feature points determined in the above (6.1) step

From this, S _θ = R _θ S, S _θ is the Brief feature descriptor with rotation invariance, and finally the ORB global feature descriptor with rotation invariance is calculated: g _n (P, θ):=f _n ( P)|(x _i ,y _i )∈S _θ , where n=256 is selected;

The seventh step is to collect the image to be tested:

In the positioning area described in the second step above, use a smartphone to photograph the nearest house number at the to-be-located point, and collect the image to be tested;

The eighth step, the feature matching and visual positioning of the tested image:

The feature matching method of the tested image is: ① Calculate the Euclidean distance between two SURF global feature descriptors:

其中i为64维特征向量的第i维；②计算两个ORB全局特征描述符之间的汉明距离：两个ORB全局特征描述符R¹，R²之间的汉明距离通过对两个二进制字符串T¹，T²进行位异或运算得到，

其中i为256位字符串的第i位；上述两个特征描述符的距离越小，图像匹配度越高；Euclidean distance between ^two SURF global feature descriptors ^L1 , L2

where i is the ith dimension of the 64-dimensional feature vector; ② Calculate the Hamming distance between the two ORB global feature descriptors: The Hamming distance between the two ORB global feature descriptors R ¹ , R ² is calculated by comparing the two Binary strings T ¹ , T ² are obtained by bitwise XOR operation,

where i is the ith bit of a 256-bit string; the smaller the distance between the two feature descriptors above, the higher the image matching degree;

The method of visual positioning is to first calculate the SURF global feature descriptor and the ORB global feature descriptor of the tested image collected in the above seventh step by using the methods of the fifth step and the sixth step above, and then use the above test image. The feature matching method calculates the distance between the SURF global feature descriptor and ORB global feature descriptor of the tested image and the SURF global feature descriptor and ORB global feature descriptor of the training image obtained in the fourth step above, and then uses KNN The algorithm calculates three nearest neighbors in the SURF matching space, that is, calculates the three training images obtained in the fourth step above with the smallest Euclidean distance from the SURF global feature descriptor of the tested image, and calculates two nearest neighbors in the ORB matching space, namely Calculate the two training images obtained in the fourth step above with the smallest Hamming distance to the ORB global feature descriptor of the image to be tested, and finally take the intersection of the three neighbors and the two neighbors as the closest neighbor to the image to be tested. The training images in the training image set in the fourth step above are called matching images, and the position coordinates corresponding to the matching images are the visual positioning coordinates (x _v , y _v ), thereby completing the visual positioning;

The ninth step, the combination of WiFi fingerprint positioning and visual positioning:

After obtaining the WiFi positioning range in the third step above, the training images in the training image set generated in the fourth step above with the coordinates located within the WiFi positioning range form a matching image set, and when the matching image set in the eighth step above is located in the matching image set, Then take the visual positioning coordinates (x _v , y _v ) obtained in the above eighth step as the final indoor positioning position coordinates, otherwise take the WiFi positioning coordinates (x _w , y _w ) obtained in the third step above as the final position coordinates, Thus, the positioning of the combination of WiFi fingerprint positioning and visual positioning is completed.

其中，xf(MAC)是指实测指纹的地址序列，lf(MAC)是指上述WiFi位置指纹数据库中第l个指纹的MAC地址序列，l＝(1,2,...,m)，Num[xf(MAC)＝lf(MAC)]表示MAC地址匹配成功的数目，最后将匹配度由高到低的三个指纹对应的位置作为粗略定位范围(x₀～x₁,y₀～y₁)，在此基础上，得到WiFi定位坐标(x_w,y_w)，

In the above-mentioned indoor positioning method, the fingerprint matching algorithm in the third step is to compare the MAC address sequence of the measured fingerprint with the MAC address sequence of all fingerprints in the above-mentioned WiFi location fingerprint database one by one. When the MAC addresses are the same, then match success

Among them, xf(MAC) refers to the address sequence of the measured fingerprint, lf(MAC) refers to the MAC address sequence of the lth fingerprint in the above WiFi location fingerprint database, l=(1,2,...,m), Num [xf(MAC)=lf(MAC)] indicates the number of successful MAC address matching, and finally the positions corresponding to the three fingerprints with matching degrees from high to low are used as the rough positioning range (x ₀ ~ x ₁ , y ₀ ~ y ₁ ), on this basis, the WiFi positioning coordinates (x _w , y _w ) are obtained,

In the above-mentioned indoor positioning method, the method for calculating the SURF global feature descriptor in the fifth step is as follows:

向量模长最大值对应的角度即为特征点主方向；(1) Calculate the main direction of the feature point: take the feature point as the center, in a circular neighborhood with a radius of 6s, calculate the sum m _w of the Haar wavelet responses of all points in the 60-degree sector in the x and y directions. Gaussian weighting these response values, d _x and _dy are the information of the Haar wavelet response in the x and y directions, the 60-degree fan-shaped sliding window rotates in 5-degree steps, and the resultant vector angle θ _w is calculated, Then find the maximum value of the composite vector modulus length of the fan in each direction:

The angle corresponding to the maximum vector modulus length is the main direction of the feature point;

(2) Calculate the SURF global feature descriptor: After obtaining the main direction of the feature point through the above step (1), take a square frame around the feature point, the side length of the frame is 20s, and the square is divided into 4 × 4 sub-regions, For each subregion, calculate the Haar wavelet response with 5×5 feature points at regular intervals, and the feature descriptor v of each subregion is calculated as follows: v=(Σd _x ,Σd _y ,Σ|d _x |,Σ| d _y |), the descriptions of all 16 sub-regions are matched to form the SURF feature descriptor of this feature point, and the final SURF global feature descriptor is a 64-dimensional vector: V={v ₁ ,v ₂ ,... ,v ₁₆ }, where v _i (i=1,2,...,16) is the feature descriptor of the ith subregion;

The above s is a scale factor.

The beneficial effects of the present invention are: compared with the prior art, the outstanding substantive features of the present invention are as follows:

(1) An indoor positioning method of the present invention utilizes the principle of WiFi fingerprint positioning and the principle of visual positioning, adopts the method of combining WiFi fingerprint-based positioning and sign-based visual positioning, and first uses WiFi location fingerprint positioning algorithm to obtain WiFi positioning The range and WiFi positioning coordinates are obtained, and then the visual positioning coordinates are obtained according to the feature matching and visual positioning of the tested image, and a high-precision indoor positioning method based on the fusion of WiFi fingerprint and vision is created. Both in principle and in practice, it is difficult to perform WiFi fingerprint positioning and visual positioning at the same time on the same platform. WiFi fingerprints need to be collected by smartphones, and the fingerprint matching algorithm needs to be implemented in the C++ environment; The training image and the tested image need to be taken with a mobile phone, and the calculation of the SURF global feature descriptor and the ORB global feature descriptor and the feature matching algorithm need to be implemented in the C++ environment. After long-term and arduous research and development work, the inventor of the present invention has obtained the results of WiFi fingerprint positioning and the results of visual positioning respectively and combined them, and proposed a method for correcting the positioning results. The experimental results show that the positioning method combining the two positioning technologies proposed by the present invention has high accuracy and small error.

(2) In the method of the present invention, WiFi fingerprint positioning and visual positioning are complementary in that: the results obtained only by WiFi positioning have large errors and low accuracy, and can be corrected by visual positioning; When the error is large, the exact coordinates of the WiFi fingerprint positioning can be used to replace the visual positioning coordinates, thereby reducing the positioning error and realizing high-precision indoor positioning.

Compared with the prior art, the significant progress of the present invention is as follows:

(1) The present invention only uses the MAC address when constructing the WiFi fingerprint, which simplifies the fingerprint database, and as long as the wireless access point AP is not removed, the fingerprint remains unchanged, and the positioning accuracy rate has nothing to do with changes in the indoor environment.

(2) The present invention innovatively combines WiFi fingerprint positioning with visual positioning, and can easily realize high-precision indoor positioning.

(3) The method of the present invention can effectively improve the accuracy of indoor positioning. The accuracy of the traditional indoor positioning method based on WiFi fingerprint positioning is generally about 10m. The method of the present invention combines WiFi fingerprint positioning and visual positioning, and the positioning accuracy can reach 6m. The positioning accuracy can reach 80%.

(4) The method of the present invention first uses WiFi fingerprint positioning to obtain a rough positioning range and exact position coordinates, and then focuses on using visual positioning to correct the exact position coordinates obtained by WiFi fingerprint positioning, so it can also be used in an environment with fewer WiFi signals. .

(5) The method of the present invention does not require the deployment of fixed WiFi access points, is simple in operation, low in cost, does not require additional devices, and does not require modification of the indoor environment.

(6) The method of the present invention utilizes indoor house numbers for visual positioning, and is suitable for all indoor places with house numbers, such as conference rooms, indoor activity centers and office buildings.

Description of drawings

The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

FIG. 1 is a schematic block diagram of the steps of the method of the present invention.

FIG. 2 is a schematic diagram of the use of an Android App developed by the method of the present invention.

FIG. 3 is a schematic diagram of the positioning area and the distribution of sampling points in the method of the present invention.

In the figure, 1 to 60 are WiFi sampling points in the positioning area, 1, 4, 7, 11, 14, 17, 20, 23, 38, 41, 45, 47, 51, 54, 57, 59 is the mark sampling point of the training image, that is, the point to be located.

Detailed ways

The embodiment shown in FIG. 1 shows that the steps of the method of the present invention are: generating WiFi fingerprints → building WiFi location fingerprint database → WiFi fingerprint positioning → generating training image sets → calculating SURF global feature descriptors → calculating ORB global feature descriptors → acquisition Tested image → Feature matching and visual positioning of the tested image → Positioning combined with WiFi fingerprint positioning and visual positioning.

The embodiment shown in FIG. 2 shows that the Android App developed using the method of the present invention obtains the MAC address (MAC ADDRESS) of the WiFi signal, and generates a txt file and saves it in the smart phone, thereby generating WiFi fingerprints; the WiFi location is shown in the figure The fingerprint at a WiFi sampling point in the fingerprint database is composed of a series of MAC addresses. The App can receive 17 WiFi signals at this sampling point, and there are 17 MAC addresses correspondingly. Press "Save" in the smartphone. " button to save the fingerprint as a txt file.

The embodiment shown in Figure 3 shows the distribution of the positioning area and sampling points. The positioning area selects a section of the corridor, which is 60m long and 3m wide. The WiFi sampling points in the positioning area are distributed in a mesh, with a total of 60 , the interval between each sampling point is 2m, and there are 16 marked sampling points in total, namely 1, 4, 7, 11, 14, 17, 20, 23, 38, 41, 45, 47, 51, which are indicated by the triangular arrows in the figure. 54, 57, and 59 are the mark sampling points of the training image, which are also the points to be located.

Example

The first step is to generate a WiFi fingerprint:

Develop an Android App based on Java language, obtain the MAC address of the WiFi signal, and generate a txt file and save it in the smartphone, thereby generating a WiFi fingerprint; from the embodiment shown in Figure 2, it can be seen that the App is used in a selected selected 17 WiFi signals can be received at the sampling point, correspondingly 17 MAC addresses, press the "Save" button in the smartphone to save the fingerprint as a txt file;

The second step is to build a WiFi location fingerprint database:

The corridor area in the indoor environment is selected as the positioning area. In this corridor area, 60 WiFi sampling points are selected. The coordinates of each sampling point are known. Use the installed App to detect the detectable WiFi signal at each sampling point. , obtain its MAC address sequence and save it as a fingerprint in the WiFi location fingerprint database. The WiFi location fingerprint database is composed of a series of stored MAC address sequences. Each fingerprint corresponds to unique location information. In the selected positioning area, set Set 60 WiFi sampling points, use the MAC address collected at each sampling point as the fingerprint of the sampling point, traverse all sampling points to get 60 fingerprints, and save them into the WiFi location fingerprint database, thus completing the construction of the WiFi location fingerprint database. ; As shown in the embodiment shown in Figure 3, the positioning area in this embodiment 1 selects a section of the corridor, its length is 60m, the width is 3m, the WiFi sampling points in the positioning area are distributed in a mesh, a total of 60, each The sampling point interval is 2m;

The third step, WiFi fingerprint positioning:

利用指纹匹配算法依照上述第二步所述的每一个指纹对应唯一的位置信息的规则，将这个实测指纹xf与上述第二步中的WiFi位置指纹数据库中的指纹进行匹配，得到匹配度最高的三个指纹，从而得到WiFi定位范围(x₀～x₁,y₀～y₁)及WiFi定位坐标(x_w,y_w)，实现WiFi指纹定位，其中的x₀、x₁为待定位点的横坐标，x₀～x₁为待定位的横坐标范围，单位为m；其中的y₀、y₁为待定位点的纵坐标，y₀～y₁为待定位的纵坐标范围，单位为m；In the positioning stage, in the positioning area selected in the second step above, set the position coordinates of the to-be-located point x as (x, y), and N _x WiFi signals can be received. The App detection in the first step is used to obtain this The measured fingerprint of the anchor point x

The fingerprint matching algorithm is used to match the measured fingerprint xf with the fingerprints in the WiFi location fingerprint database in the second step above according to the rule that each fingerprint corresponds to unique location information described in the second step above, and the highest matching degree is obtained. Three fingerprints are obtained to obtain the WiFi positioning range (x ₀ ~ x ₁ , y ₀ ~ y ₁ ) and WiFi positioning coordinates (x _w , y _w ) to realize WiFi fingerprint positioning, where x ₀ and x ₁ are the points to be located. x ₀ ~ x ₁ is the abscissa range to be positioned, and the unit is m; y ₀ and y ₁ are the vertical coordinates of the point to be positioned, and y ₀ ~ y ₁ are the vertical coordinate range to be positioned, in units of is m;

其中，xf(MAC)是指实测指纹的地址序列，lf(MAC)是指上述WiFi位置指纹数据库中第l个指纹的MAC地址序列，l＝(1,2,...,m)，Num[xf(MAC)＝lf(MAC)]表示MAC地址匹配成功的数目，最后将匹配度由高到低的三个指纹对应的位置作为粗略定位范围(x₀～x₁,y₀～y₁)，在此基础上，得到WiFi定位坐标(x_w,y_w)，

The fingerprint matching algorithm is to compare the MAC address sequence of the measured fingerprint with the MAC address sequence of all the fingerprints in the above WiFi location fingerprint database one by one, when the MAC addresses are the same, the matching is successful, and the matching degree is

Among them, xf(MAC) refers to the address sequence of the measured fingerprint, lf(MAC) refers to the MAC address sequence of the lth fingerprint in the above WiFi location fingerprint database, l=(1,2,...,m), Num [xf(MAC)=lf(MAC)] indicates the number of successful MAC address matching, and finally the positions corresponding to the three fingerprints with matching degrees from high to low are used as the rough positioning range (x ₀ ~ x ₁ , y ₀ ~ y ₁ ), on this basis, the WiFi positioning coordinates (x _w , y _w ) are obtained,

The fourth step is to generate a training image set:

In the positioning area of the second step above, the coordinates of all the house numbers are known, and these house numbers are the sign sampling points, that is, the points to be located. Use the smartphone to shoot the house number, traverse all the sign sampling points, and generate a training image set. Since the marker sampling point is a part of the WiFi sampling point, the coordinates of the marker sampling point are known, and the coordinates of the training image are the coordinates of the marker sampling point; as shown in the embodiment shown in FIG. 16, namely 1, 4, 7, 11, 14, 17, 20, 23, 38, 41, 45, 47, 51, 54, 57, 59 indicated by the triangular arrows in the figure are the sampling points of the training images. , is also the point to be positioned.

The fifth step is to calculate the SURF global feature descriptor:

First, the training images obtained in the fourth step above are preprocessed, including normalization and grayscale processing, and then the SURF global feature descriptor is calculated, including two parts, the first part is the feature point location, and the second part is the feature description Symbol calculation, the center point of the normalized image is used as the feature point, the whole image is regarded as a single neighborhood of the feature point, and the feature descriptor calculated from this is used as the SURF global feature description of the whole image. symbol;

The method for calculating the SURF global feature descriptor is as follows:

d_x和d_y是Haar小波响应在x和y方向的信息，60度扇形滑动窗以5度一步转动，计算合成向量角度θ_w，

再求出各方向扇形的合成向量模长最大值：

向量模长最大值对应的角度即为特征点主方向；(1) Calculate the main direction of the feature point: take the feature point as the center, in a circular neighborhood with a radius of 6s, calculate the sum m _w of the Haar wavelet responses of all points in the 60-degree sector in the x and y directions. Gaussian weighting these response values,

d _x and _dy are the information of the Haar wavelet response in the x and y directions, the 60-degree fan-shaped sliding window rotates in 5-degree steps, and the resultant vector angle θ _w is calculated,

Then find the maximum value of the composite vector modulus length of the fan in each direction:

The angle corresponding to the maximum vector modulus length is the main direction of the feature point;

(2) Calculate the SURF global feature descriptor: After obtaining the main direction of the feature point through the above step (1), take a square frame around the feature point, the side length of the frame is 20s, and the square is divided into 4 × 4 sub-regions, For each subregion, calculate the Haar wavelet response with 5 × 5 feature points at regular intervals, and the feature descriptor v of each subregion is calculated as follows: v=(∑d _x ,∑d _y, ∑|d _x |,∑ |d _y |), the descriptions of all 16 sub-regions are matched to form the SURF feature descriptor of this feature point, and the final SURF global feature descriptor is a 64-dimensional vector: V={v ₁ , v ₂ , .. .,v ₁₆ }, where v _i (i=1,2,...,16) is the feature descriptor of the ith subregion;

The above s is the scale factor;

The sixth step is to calculate the ORB global feature descriptor:

(6.1) Determine the main direction of the feature point:

其中I(x,y)是点(x,y)处的灰度值，该特征点的邻域图像的质心

邻域图像的质心与特征点的夹角为θ＝arctan2(m₀₁,m₁₀)，即为特征点的主方向；In the same way as in the fifth step above, the center point of the normalized image is used as the feature point, and the main direction of the feature point is calculated by using the image moment. For any feature point, its image moment is

where I(x,y) is the gray value at point (x,y), the centroid of the neighborhood image of this feature point

The angle between the centroid of the neighborhood image and the feature point is θ=arctan2(m ₀₁ , m ₁₀ ), which is the main direction of the feature point;

(6.2) Generate BRIEF feature descriptor:

其中，p1(x)表示图像邻域p1上一点x处的强度，p1(y)表示图像邻域p1上一点y处的强度,经过n次二进制测试，得到一个n维向量，即BRIEF特征描述子

这里选取n＝256，得到的是256位的二进制字符串；The generation process of the BRIEF feature descriptor is as follows: p1 represents a smoothed image neighborhood, and the binary test of x and y at any position is the logical result of the intensity test of these two points:

Among them, p1(x) represents the intensity of a point x on the image neighborhood p1, and p1(y) represents the intensity of a point y on the image neighborhood p1. After n binary tests, an n-dimensional vector is obtained, that is, the Brief feature description son

Here n=256 is selected, and a 256-bit binary string is obtained;

(6.3) Calculate the ORB global feature descriptor:

由上述(6.1)步确定的特征点的主方向计算出仿射变换矩阵由此得到S_θ＝R_θS，S_θ即为具有旋转不变性的BRIEF特征描述子，最后计算出具有旋转不变性的ORB全局特征描述符：g_n(P,θ):＝f_n(P)|(x_i,y_i)∈S_θ，这里选取n＝256；In order to make the Brief feature descriptor have rotation invariance, the orientation of the Brief feature descriptor is set according to the direction of the feature points determined in the above step (6.1), and the n-bit binary test set for any image pixel point (x _i , y _i ) is obtained. The feature set of , defines a 2×n matrix

Calculate the affine transformation matrix from the main direction of the feature points determined in the above (6.1) step From this, S _θ = R _θ S, S _θ is the Brief feature descriptor with rotation invariance, and finally the ORB global feature descriptor with rotation invariance is calculated: g _n (P, θ):=f _n ( P)|(x _i ,y _i )∈S _θ , where n=256 is selected;

The seventh step is to collect the image to be tested:

In the positioning area described in the second step above, use a smartphone to photograph the nearest house number at the to-be-located point, and collect the image to be tested;

The eighth step, the feature matching and visual positioning of the tested image:

其中i为256位字符串的第i位；上述两个特征描述符的距离越小，图像匹配度越高；The feature matching method of the tested image is: ① Calculate the Euclidean distance between two SURF global feature descriptors: the Euclidean distance between the two SURF global feature descriptors L ¹ , L ² where i is the ith dimension of the 64-dimensional feature vector; ② Calculate the Hamming distance between the two ORB global feature descriptors: The Hamming distance between the two ORB global feature descriptors R ¹ , R ² is calculated by comparing the two Binary strings T ¹ , T ² are obtained by bitwise XOR operation,

where i is the ith bit of a 256-bit string; the smaller the distance between the two feature descriptors above, the higher the image matching degree;

The method of visual positioning is to first calculate the SURF global feature descriptor and the ORB global feature descriptor of the tested image collected in the above seventh step by using the methods of the fifth step and the sixth step above, and then use the above test image. The feature matching method calculates the distance between the SURF global feature descriptor and ORB global feature descriptor of the tested image and the SURF global feature descriptor and ORB global feature descriptor of the training image obtained in the fourth step above, and then uses KNN The algorithm calculates three nearest neighbors in the SURF matching space, that is, calculates the three training images obtained in the fourth step above with the smallest Euclidean distance from the SURF global feature descriptor of the tested image, and calculates two nearest neighbors in the ORB matching space, namely Calculate the two training images obtained in the fourth step above with the smallest Hamming distance to the ORB global feature descriptor of the image to be tested, and finally take the intersection of the three neighbors and the two neighbors as the closest neighbor to the image to be tested. The training images in the training image set in the fourth step above are called matching images, and the position coordinates corresponding to the matching images are the visual positioning coordinates (x _v , y _v ), thereby completing the visual positioning;

The ninth step, the combination of WiFi fingerprint positioning and visual positioning:

After obtaining the WiFi positioning range in the third step above, the training images in the training image set generated in the fourth step above with the coordinates located within the WiFi positioning range form a matching image set, and when the matching image set in the eighth step above is located in the matching image set, Then take the visual positioning coordinates (x _v , y _v ) obtained in the above eighth step as the final indoor positioning position coordinates, otherwise take the WiFi positioning coordinates (x _w , y _w ) obtained in the third step above as the final position coordinates, Thus, the positioning of the combination of WiFi fingerprint positioning and visual positioning is completed.

In this embodiment, the first floor of the indoor pedestrian street is used as the test site, and all the pictures taken by the mobile phone are 4160*3120 (pixels), and the positioning results are shown in Table 1.

Table 1. Positioning test results on the first floor of the indoor pedestrian street

By comparing the real coordinates of the point to be positioned and the positioning coordinates of this embodiment, it is proved that the indoor positioning method of the present invention adopts the method of combining WiFi fingerprint-based positioning and sign-based visual positioning to achieve high-precision indoor positioning. .

Example 2

In addition to "select the corridor area in the indoor environment as the positioning area, in the corridor area, select 30 WiFi sampling points; in the selected positioning area, set 30 WiFi sampling points, and collect the data collected from each sampling point. The MAC address is used as the fingerprint of the sampling point, and 30 fingerprints can be obtained by traversing all the sampling points, which are saved into the WiFi location fingerprint database, thereby completing the construction of the WiFi location fingerprint database”, the others are the same as in Example 1.

Example 3

In addition to "select the corridor area in the indoor environment as the positioning area, in the corridor area, select 45 WiFi sampling points; in the selected positioning area, set 45 WiFi sampling points, and collect the data collected from each sampling point. The MAC address is used as the fingerprint of the sampling point, and 45 fingerprints can be obtained by traversing all the sampling points, which are stored in the WiFi location fingerprint database, thereby completing the construction of the WiFi location fingerprint database.

Claims (3)

1. An indoor positioning method, characterized in that: the method combines positioning based on WiFi fingerprint and visual positioning based on marks, firstly obtains WiFi positioning range and WiFi positioning coordinates by utilizing a WiFi position fingerprint positioning algorithm, then obtains visual positioning coordinates according to feature matching and visual positioning of a tested image, and finally combines WiFi position fingerprint positioning and visual positioning, and comprises the following specific steps:

first, generating a WiFi fingerprint:

developing an Android App based on Java language, obtaining an MAC address of a WiFi signal, generating a txt file and storing the txt file in a smart phone, and generating a WiFi fingerprint;

secondly, constructing a WiFi position fingerprint database:

selecting a corridor area under an indoor environment as a positioning area, selecting 30-60 WiFi sampling points in the corridor area, wherein the coordinates of each sampling point are known, detecting detectable WiFi signals at each sampling point by using an installed App, obtaining an MAC address sequence of the WiFi sampling points and storing the MAC address sequence as a fingerprint in a WiFi position fingerprint database, forming a WiFi position fingerprint database by a series of stored MAC address sequences, wherein each fingerprint corresponds to unique position information, setting 60 WiFi sampling points in the selected positioning area, taking the MAC address acquired by each sampling point as the fingerprint of the sampling point, traversing all the sampling points to obtain 60 fingerprints, storing the 60 fingerprints in the WiFi position fingerprint database, and thus completing the construction of the WiFi position fingerprint database;

thirdly, WiFi fingerprint positioning:

in the positioning stage, the position coordinate of the position x to be positioned is set to be (x, y) in the positioning area selected in the second step, and N can be received_xThe WiFi signal is detected by the App in the first step to obtain the actual measurement fingerprint of the positioning point x

Matching the actual measurement fingerprint xf with the fingerprints in the WiFi position fingerprint database in the second step by using a fingerprint matching algorithm according to the rule that each fingerprint corresponds to unique position information in the second step to obtain three fingerprints with the highest matching degree, thereby obtaining the WiFi positioning range (x)₀～x₁,y₀～y₁) And WiFi positioning coordinates (x)_w,y_w) Realizing WiFi fingerprint positioning, wherein x₀、x₁As the abscissa, x, of the point to be located₀～x₁The range of the abscissa to be positioned is in m; wherein y is₀、y₁As the ordinate, y, of the point to be located₀～y₁Is a vertical coordinate range to be positioned, and the unit is m;

fourthly, generating a training image set:

in the positioning area of the second step, the coordinates of all doorplates are known, the doorplates are mark sampling points, namely to-be-positioned points, the doorplates are shot by using a smart phone, all mark sampling points are traversed, a training image set is generated, the coordinates of the mark sampling points are known as the mark sampling points are part of the WiFi sampling points, and the coordinates of the training image are the coordinates of the mark sampling points;

fifthly, calculating SURF global feature descriptors:

firstly, preprocessing the training image obtained in the fourth step, wherein the preprocessing comprises normalization processing and graying processing, then calculating a SURF global feature descriptor, and the SURF global feature descriptor comprises two parts, wherein the first part is feature point positioning, the second part is feature descriptor calculation, the central point of the image after the normalization processing is used as a feature point, the whole image is used as a single neighborhood of the feature point, and the calculated feature descriptor is used as the SURF global feature descriptor of the whole image;

sixthly, calculating an ORB global feature descriptor:

(6.1) determining the main direction of the characteristic points:

taking the central point of the normalized image as a feature point, calculating the main direction of the feature point by using the image moment, and calculating the image moment of any feature point

Where I (x, y) is the gray value at point (x, y), the centroid of the neighborhood image of the feature point

The included angle between the centroid of the neighborhood image and the feature point is theta, arctan2 (m)₀₁,m₁₀) I.e. the main direction of the feature point;

(6.2) generating BRIEF feature descriptors:

the BRIEF feature descriptor is generated as follows: p1 represents a smoothed image neighborhood, and the binary test at any location point x and y is the logical result of the two intensity tests:wherein p1(x) represents the intensity of a point x on the image neighborhood p1, p1(y) represents the intensity of a point y on the image neighborhood p1, and an n-dimensional vector, namely a BRIEF feature descriptor, is obtained through n binary testsWhere n is 256, the result is 256 bitsA binary string;

(6.3) computing ORB global feature descriptors:

in order to make the BRIEF feature descriptor have rotation invariance, the direction of the BRIEF feature descriptor is set according to the direction of the feature point determined in the step (6.1), and the pixel point (x) of any image is subjected to_i,y_i) Defining a 2 x n matrix of feature sets obtained from the n-bit binary test set

Calculating affine transformation matrix from principal direction of feature point determined in the above step (6.1)

Thereby obtaining S_θ＝R_θS，S_θNamely, the BRIEF feature descriptor with the rotation invariance, and finally, the ORB global feature descriptor with the rotation invariance is calculated: g_n(P,θ):＝f_n(P)|(x_i,y_i)∈S_θHere, n is 256;

and seventhly, collecting a tested image:

in the positioning area in the second step, a doorplate closest to the positioning area is shot at the positioning point by using a smart phone, and a tested image is acquired;

eighth step, feature matching and visual positioning of the tested image:

the characteristic matching method of the tested image is that ① calculates the Euclidean distance between two SURF global characteristic descriptors L¹，L²Euclidean distance between them

Where i is the ith dimension of the 64-dimensional feature vector, ② calculates the Hamming distance between two ORB global feature descriptors, R¹，R²The Hamming distance between the two binary character strings T¹，T²The result of the bit exclusive-or operation is obtained,

where i is the ith bit of the 256-bit string; the smaller the distance between the two feature descriptors is, the higher the image matching degree is;

the visual localization method comprises the steps of firstly respectively adopting the fifth step and the sixth step to calculate the SURF global feature descriptor and the ORB global feature descriptor of the tested image acquired in the seventh step, then respectively calculating the distance between the SURF global feature descriptor and the ORB global feature descriptor of the tested image and the SURF global feature descriptor and the ORB global feature descriptor of the training image obtained in the fourth step by using the feature matching method of the tested image, then calculating three neighbors in a SURF matching space by using a KNN algorithm, namely calculating three training images obtained in the fourth step with the minimum Euclidean distance with the SURF global feature descriptor of the tested image, calculating two neighbors in the ORB matching space, namely calculating two training images obtained in the fourth step with the minimum Hamming distance with the ORB global feature descriptor of the tested image, finally, the intersection of the three neighbors and the two neighbors is taken as a training image in the fourth step training image set which is closest to the tested image and is called as a matching image, and the position coordinate corresponding to the matching image is the visual positioning coordinate (x)_v,y_v) Thereby completing the visual positioning;

ninth, the location that wiFi fingerprint location and visual positioning combined together:

after the WiFi positioning range is obtained in the third step, the training images in the training image set generated in the fourth step with the coordinates in the WiFi positioning range form a matching image set, and when the matching image in the eighth step is located in the matching image set, the visual positioning coordinates (x) obtained in the eighth step are used_v,y_v) As the final indoor positioning position coordinate, otherwise, the WiFi positioning coordinate (x) obtained in the third step is used_w,y_w) And as final position coordinates, positioning combining WiFi fingerprint positioning and visual positioning is completed.

2. The indoor positioning method according to claim 1, wherein: the fingerprint matching algorithm in the third step is to compare the MAC address sequence of the actual measurement fingerprint with the MAC address sequences of all the fingerprints in the WiFi position fingerprint database one by one, when the MAC addresses are the same, the matching is successful, and the matching degree is high

Wherein xf (MAC) refers to an address sequence of a measured fingerprint, lf (MAC) refers to a MAC address sequence of the ith fingerprint in the WiFi location fingerprint database, l ═ (1, 2.. multidot., m), Num [ xf (MAC) ═ lf (MAC)]The number of successful matching of the MAC address is represented, and finally, the positions corresponding to three fingerprints with high matching degree to low matching degree are used as a rough positioning range (x)₀～x₁,y₀～y₁) On the basis, obtaining WiFi positioning coordinates (x)_w,y_w)，

3. The indoor positioning method according to claim 1, wherein: the method of calculating the SURF global feature descriptor in the fifth step is as follows:

(1) calculating the main direction of the feature points: taking the characteristic point as the center, in the circular neighborhood with the radius of 6s, calculating the sum m of Haar wavelet responses of all points in the sector of 60 degrees in the x and y directions_wThe response values are gaussian weighted when the sum is calculated,

d_xand d_yThe information of Haar wavelet response in x and y directions, a 60-degree fan-shaped sliding window rotates by 5 degrees in one step, and the angle theta of the synthesized vector is calculated_w，

Then, the maximum value of the mode length of the synthetic vector of each direction sector is obtained:

the angle corresponding to the maximum value of the vector mode length is the main direction of the feature point;

(2) compute SURF global feature descriptors: after the main direction of the feature point is obtained through the step (1), a square frame is taken around the feature point, the side length of the frame is 20s, the square is divided into 4 × 4 subregions, for each subregion, Haar wavelet responses of the feature points at 5 × 5 fixed intervals are calculated, and the feature descriptor v of each subregion is calculated as follows: v ═ d (∑ d)_x,∑d_y,∑|d_x|,∑|d_y|), the descriptions of all 16 sub-regions are matched to form the SURF feature descriptor of the feature point, and the final SURF global feature descriptor is a 64-dimensional vector: v ═ V₁,v₂,...,v₁₆In which v is_i(i 1, 2.., 16) is a feature descriptor of the i-th sub-region;

s is a scale factor.

Images