CN113628272A - Indoor positioning method and device, electronic equipment and storage medium - Google Patents
Indoor positioning method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN113628272A CN113628272A CN202110784734.3A CN202110784734A CN113628272A CN 113628272 A CN113628272 A CN 113628272A CN 202110784734 A CN202110784734 A CN 202110784734A CN 113628272 A CN113628272 A CN 113628272A
- Authority
- CN
- China
- Prior art keywords
- sample
- indoor
- positioning
- indoor positioning
- model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 62
- 230000004927 fusion Effects 0.000 claims abstract description 34
- 238000012549 training Methods 0.000 claims description 60
- 238000000605 extraction Methods 0.000 claims description 43
- 230000006870 function Effects 0.000 claims description 18
- 238000013527 convolutional neural network Methods 0.000 claims description 12
- 238000013528 artificial neural network Methods 0.000 claims description 11
- 238000012360 testing method Methods 0.000 claims description 9
- 238000004590 computer program Methods 0.000 claims description 8
- 238000003062 neural network model Methods 0.000 claims description 8
- 125000004122 cyclic group Chemical group 0.000 claims description 7
- 238000012545 processing Methods 0.000 claims description 6
- 230000004807 localization Effects 0.000 claims description 4
- 239000010410 layer Substances 0.000 description 61
- 238000010586 diagram Methods 0.000 description 12
- 239000013598 vector Substances 0.000 description 8
- 230000000007 visual effect Effects 0.000 description 7
- 238000004891 communication Methods 0.000 description 6
- 230000008901 benefit Effects 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 230000000306 recurrent effect Effects 0.000 description 4
- 230000009286 beneficial effect Effects 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000011156 evaluation Methods 0.000 description 2
- 238000005286 illumination Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000007781 pre-processing Methods 0.000 description 2
- 238000004422 calculation algorithm Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000010187 selection method Methods 0.000 description 1
- 239000002356 single layer Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S5/00—Position-fixing by co-ordinating two or more direction or position line determinations; Position-fixing by co-ordinating two or more distance determinations
- G01S5/02—Position-fixing by co-ordinating two or more direction or position line determinations; Position-fixing by co-ordinating two or more distance determinations using radio waves
- G01S5/0257—Hybrid positioning
- G01S5/0258—Hybrid positioning by combining or switching between measurements derived from different systems
- G01S5/02585—Hybrid positioning by combining or switching between measurements derived from different systems at least one of the measurements being a non-radio measurement
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S5/00—Position-fixing by co-ordinating two or more direction or position line determinations; Position-fixing by co-ordinating two or more distance determinations
- G01S5/02—Position-fixing by co-ordinating two or more direction or position line determinations; Position-fixing by co-ordinating two or more distance determinations using radio waves
- G01S5/0294—Trajectory determination or predictive filtering, e.g. target tracking or Kalman filtering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
Abstract
The invention provides an indoor positioning method, an indoor positioning device, electronic equipment and a storage medium, wherein the method comprises the following steps: acquiring WiFi signal data and indoor image data of a current position; inputting the WiFi signal data and the indoor image data into an indoor positioning model to obtain a predicted position of the current position output by the indoor positioning model; the indoor positioning model is used for respectively extracting signal features in the WiFi signal data and image features in the indoor image data, fusing the signal features and the image features and then performing positioning prediction. The method, the device, the electronic equipment and the storage medium provided by the invention directly use the WiFi signal data and the indoor image data for fusion positioning, thereby improving the precision and the efficiency of indoor positioning.
Description
Technical Field
The present invention relates to the field of computer technologies, and in particular, to an indoor positioning method and apparatus, an electronic device, and a storage medium.
Background
In an indoor environment, due to attenuation, shielding and the like of Global Navigation Satellite System (GNSS) signals, position information of a user cannot be acquired, or the acquired position information of the user is not accurate enough. Thus, other signals may be employed for indoor positioning, such as WiFi, bluetooth, magnetic fields, visible light, acoustic signals, images, and the like. Currently, research on indoor positioning techniques can be divided into two main categories depending on the type of sensor used: wireless device-based positioning techniques and visual image-based positioning techniques.
Based on the positioning technology of the wireless device, such as WiFi and bluetooth, the deployment mode directly affects the positioning result. On one hand, the maintenance cost of the equipment is excessive, and on the other hand, the positioning signal is unstable and has more noise due to the attenuation of the signal and the multipath effect, so that the target precision cannot be achieved. The positioning technology based on the visual image has higher precision, but the visual positioning precision has higher requirements on the image, and more challenges are often met: (1) the geometric distance from the camera to the scene is short, even a small change of a viewpoint can cause a large change on an image, and the influence of occlusion on the image is large; (2) objects like white walls are often encountered in indoor environment, and the image features are not obvious due to lack of textures; (3) the high similarity of the indoor environment also causes difficulties in visual positioning; (4) images are susceptible to complex variations in scene illumination.
In the prior art, data of different sensors are usually adopted to carry out positioning respectively, and the results of the respective positioning are selected to be selected, so that the positioning efficiency is low, and the positioning accuracy is poor.
Disclosure of Invention
The invention provides an indoor positioning method and device, electronic equipment and a storage medium, which are used for solving the technical problem of poor indoor positioning precision in the prior art.
The invention provides an indoor positioning method, which comprises the following steps:
acquiring WiFi signal data and indoor image data of a current position;
inputting the WiFi signal data and the indoor image data into an indoor positioning model to obtain a predicted position of the current position output by the indoor positioning model;
the indoor positioning model is used for respectively extracting signal features in the WiFi signal data and image features in the indoor image data, fusing the signal features and the image features and then performing positioning prediction.
According to the indoor positioning method provided by the invention, the indoor positioning model comprises a signal feature extraction layer, an image feature extraction layer and a fusion prediction layer;
the inputting the WiFi signal data and the indoor image data to an indoor positioning model to obtain a predicted position of the current position output by the indoor positioning model includes:
inputting the WiFi signal data to the signal feature extraction layer to obtain signal features output by the signal feature extraction layer;
inputting the indoor image data to the image feature extraction layer to obtain image features output by the image feature extraction layer;
and inputting the signal characteristics and the image characteristics to the fusion prediction layer to obtain the prediction position of the current position output by the fusion prediction layer.
According to the indoor positioning method provided by the invention, the indoor positioning model is obtained by training based on the following steps:
determining a sample set, the sample set comprising a plurality of sample anchor point sequences, each sample anchor point sequence comprising actual positions of a plurality of sample anchor points, sample WiFi signal data, and sample indoor image data;
training an initial model based on the sample set to obtain the indoor positioning model;
the signal feature extraction layer of the initial model adopts a cyclic neural network model, the image feature extraction layer adopts a convolutional neural network model, and the fusion prediction layer adopts a fully-connected neural network.
According to the indoor positioning method provided by the invention, the determining of the sample set comprises the following steps:
determining a candidate training set of the indoor positioning model based on the actual indoor position of each sample positioning point, sample WiFi signal data and sample indoor image data;
and determining the sample set based on the sample positioning points meeting the Gaussian distribution in the candidate training set.
According to the indoor positioning method provided by the invention, the determining the sample set based on the sample positioning points satisfying the gaussian distribution in the candidate training set comprises the following steps:
based on the length of a preset sequence, taking any sample positioning point in the candidate training set as a starting point, and selecting a sample positioning point satisfying Gaussian distribution from the candidate training set as a sample positioning point sequence;
determining the sample set based on a plurality of sequences of sample localization points.
According to the indoor positioning method provided by the invention, the step of determining the candidate training set of the indoor positioning model based on the actual indoor position of each sample positioning point, the sample WiFi signal data and the sample indoor image data comprises the following steps:
determining the actual indoor position of each sample positioning point, sample WiFi signal data and sample indoor image data based on the set acquisition sequence;
and determining a candidate training set and a data test set of the indoor positioning model based on the acquisition sequence number of each sample positioning point.
According to the indoor positioning method provided by the invention, the loss function of the indoor positioning model is determined based on the following steps:
determining a root mean square error based on the actual position and the predicted position of the sample positioning point in the sample set;
classifying each sample positioning point in the sample set based on the sample positioning points and the minimum distance between the sample positioning points, and determining a classification error based on the cross entropy loss of each category;
determining a loss function for the indoor positioning model based on the root mean square error and the classification error.
The present invention also provides an indoor positioning device, including:
the device comprises an acquisition unit, a processing unit and a processing unit, wherein the acquisition unit is used for acquiring WiFi signal data and indoor image data of a current position;
the prediction unit is used for inputting the WiFi signal data and the indoor image data into an indoor positioning model to obtain a predicted position of the current position output by the indoor positioning model;
the indoor positioning model is used for respectively extracting signal features in the WiFi signal data and image features in the indoor image data, fusing the signal features and the image features and then performing positioning prediction.
The invention also provides an electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the steps of the indoor positioning method when executing the program.
The invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when being executed by a processor, carries out the steps of the indoor positioning method.
According to the indoor positioning method, the indoor positioning device, the electronic equipment and the storage medium, the signal characteristics in the WiFi signal data and the image characteristics in the indoor image data are respectively extracted through the indoor positioning model, the signal characteristics and the image characteristics are fused and then are subjected to positioning prediction to obtain the predicted position of the current position, the problem of signal attenuation caused by the fact that the WiFi signal data are independently adopted for positioning is solved, the problem of visual range limitation caused by the fact that the indoor image data are independently adopted for positioning is also solved, data advantage complementation is achieved, meanwhile, the WiFi signal data and the indoor image data are directly used for fusion positioning, and the accuracy and the efficiency of indoor positioning are improved.
Drawings
In order to more clearly illustrate the technical solutions of the present invention or the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
Fig. 1 is a schematic flow chart of an indoor positioning method according to the present invention;
FIG. 2 is a schematic structural diagram of a signal feature extraction layer provided in the present invention;
FIG. 3 is a schematic structural diagram of a fusion prediction layer provided in the present invention;
FIG. 4 is a schematic diagram of an indoor sample location point acquisition provided by the present invention;
fig. 5 is a schematic diagram illustrating an application of the indoor positioning method according to the present invention;
FIG. 6 is a schematic structural diagram of an indoor positioning apparatus according to the present invention;
fig. 7 is a schematic structural diagram of an electronic device provided in the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is obvious that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Fig. 1 is a schematic flow chart of an indoor positioning method provided by the present invention, as shown in fig. 1, the method includes:
Specifically, the current position is a position of indoor position information to be determined. The indoor space can be an airport hall, an exhibition hall, a warehouse, a supermarket, a library, an underground parking lot, a mine and other places. The WiFi signal data is data characterizing an indoor WiFi signal, and may be, for example, a WiFi signal strength value. The indoor image data is data representing characteristics of an indoor environment, and may be, for example, an environmental picture taken at a certain indoor position.
Step 120, inputting the WiFi signal data and the indoor image data into an indoor positioning model to obtain a predicted position of a current position output by the indoor positioning model; the indoor positioning model is used for respectively extracting signal features in WiFi signal data and image features in indoor image data, fusing the signal features and the image features and then performing positioning prediction.
Specifically, the predicted location is a prediction of geographic information of the current location from WiFi signal data and indoor image data of the current location. The geographic information here may be absolute position coordinates or relative position coordinates.
The indoor positioning model can be obtained by pre-training, and can be obtained by the following training modes: firstly, collecting actual positions of a large number of sample positioning points, sample WiFi signal data and sample indoor image data; secondly, training the initial model according to the actual positions of a large number of sample positioning points, the WiFi signal data of the samples and the indoor image data of the samples, enabling the initial model to learn the correlation characteristics among the WiFi signal data, the indoor image data and the actual positions of the positioning points, improving the prediction capability of the initial model on the positions of the positioning points according to the WiFi signal data and the indoor image data, and obtaining the indoor positioning model.
When the correlation characteristics among the WiFi signal data, the indoor image data and the actual positions of the positioning points are learned, the indoor positioning model respectively extracts the signal characteristics in the WiFi signal data and the image characteristics in the indoor image data, fuses the signal characteristics and the image characteristics, and uses the fused characteristics obtained after fusion for positioning prediction. For example, a signal feature with a fixed dimension may be extracted from WiFi signal data, an image feature with the same dimension may be extracted from indoor image data, and then the information feature and the image feature may be stitched in the dimension to obtain a fusion feature.
The initial model may be selected from a Convolutional Neural Network (CNN), a full Convolutional Neural Network (FCN), a cyclic Neural Network (RNN), and the like, and the selection of the initial model is not specifically limited in the embodiment of the present invention.
According to the indoor positioning method provided by the embodiment of the invention, the signal characteristics in the WiFi signal data and the image characteristics in the indoor image data are respectively extracted through the indoor positioning model, the signal characteristics and the image characteristics are fused and then are positioned and predicted to obtain the predicted position of the current position, the problem of signal attenuation caused by positioning by independently adopting the WiFi signal data is solved, the problem of visual range limitation caused by positioning by independently adopting the indoor image data is also solved, the advantage complementation of data is realized, meanwhile, the WiFi signal data and the indoor image data are directly used for fusion positioning, and the precision and the efficiency of indoor positioning are improved.
Based on the embodiment, the indoor positioning model comprises a signal feature extraction layer, an image feature extraction layer and a fusion prediction layer;
inputting WiFi signal data and indoor image data into an indoor positioning model to obtain a predicted position of a current position output by the indoor positioning model, wherein the method comprises the following steps:
inputting WiFi signal data to a signal feature extraction layer to obtain signal features output by the signal feature extraction layer;
inputting indoor image data to an image feature extraction layer to obtain image features output by the image feature extraction layer;
and inputting the signal characteristics and the image characteristics into the fusion prediction layer to obtain the prediction position of the current position output by the fusion prediction layer.
Specifically, from the model structure, the indoor positioning model may include a signal feature extraction layer, an image feature extraction layer, and a fusion prediction layer. The signal feature extraction layer is used for extracting features from WiFi signal data, the image feature extraction layer is used for extracting features from indoor image data, and the fusion prediction layer is used for fusing the features extracted by the signal feature extraction layer and the image feature extraction layer and then positioning and predicting the fused features.
Based on any of the above embodiments, the indoor positioning model is obtained based on the following training steps:
determining a sample set, wherein the sample set comprises a plurality of sample positioning point sequences, and each sample positioning point sequence comprises the actual positions of a plurality of sample positioning points, sample WiFi signal data and sample indoor image data;
training the initial model based on the sample set to obtain an indoor positioning model;
the signal feature extraction layer of the initial model adopts a cyclic neural network model, the image feature extraction layer adopts a convolutional neural network model, and the fusion prediction layer adopts a fully-connected neural network.
In particular, the sample set comprises a plurality of sequences of sample localization points. Each sample positioning point sequence is a track point sequence formed by a plurality of sample positioning points. The length of the sequence of sample anchor points, i.e. the number of sample anchor points, may be set to be the same.
A recurrent neural network model may be employed as the signal feature extraction layer of the initial model. The number of network layers of the recurrent neural network model may be the number of sample anchor points in the sequence of sample anchor points. The dimensionality of the signal features output by the signal feature extraction layer can be determined according to the number of parameters in the sample WiFi signal data. If m parameters are included in the sample WiFi signal data, the dimension of the signal feature may be m.
For example, the length of the sample positioning point sequence is set to be T, WiFi signal data of a sample positioning point in any sample positioning point sequence is formed into a WiFi signal strength vector RSSI (received signal strength indicator), and the WiFi signal strength vector RSSI (received signal strength indicator) is input into the recurrent neural network. The WiFi signal strength vector RSSI of each sample anchor point contains the signal strengths of S WiFi, and the dimension of RSSI is S. The recurrent neural network adopts a long-time memory network (LSTM) structure, and sets the cycle number, namely the network layer number is the same as T. And determining the hidden layer parameter dimension according to the dimension of the WiFi signal strength vector, namely the dimension of the RSSI. Fig. 2 is a schematic structural diagram of a signal feature extraction layer provided by the present invention, and as shown in fig. 2, at each time point in a sequence T corresponding to each layer of the network, a code (embedding) corresponding to an implied layer is used as a new feature code (embedding) of a WiFi signal strength vector corresponding to a time sequence node, that is, a signal feature of each sample anchor point in the sample anchor point sequence.
A convolutional neural network model may be employed as the image feature extraction layer of the initial model. For example, according to the serial number of the sample positioning point in any sample positioning point sequence, the sample indoor image data of the sample positioning point is sequentially input into the convolutional neural network for T pieces, and the corresponding sequential output is used as a new feature description of each sample indoor image data. The convolutional neural network can select a ResNet structure for design, and the output of the last layer of network of the ResNet is averaged and pooled to the dimension same as the signal characteristic of the sample positioning point.
A fully connected neural network may be employed as the fusion prediction layer of the initial model. For example, in a sample positioning point sequence with a length of T, new feature codes and descriptions, i.e., signal features and image features, are generated from sample WiFi signal data and sample indoor image data of each sample positioning point, and the two features can be directly spliced end to end under the condition of the same dimension, so that the two features become new feature descriptions, i.e., fusion features. Fig. 3 is a schematic structural diagram of the fusion prediction layer provided by the present invention, and as shown in fig. 3, the fusion characteristics are input into the fusion prediction layer for training, so as to obtain the indoor positioning model. The fusion prediction layer can adopt a single-layer fully-connected network structure. In training, the learning rate is set to 0.0001, and the dropout ratio is set to 0.2.
Based on any of the above embodiments, determining a sample set comprises:
determining a candidate training set of an indoor positioning model based on the actual indoor position of each sample positioning point, sample WiFi signal data and sample indoor image data;
and determining a sample set based on the sample positioning points meeting the Gaussian distribution in the candidate training set.
Specifically, the positioning point can be sampled indoors, and a plurality of sample positioning points are obtained. The data that each sample location point needs to acquire includes the actual location, sample WiFi signal data, and sample indoor image data.
And after a plurality of sample positioning points are acquired, the sample positioning points are used as a candidate training set. In this case, in order to improve the robustness and generalization capability of the indoor positioning model, a sample positioning point satisfying a gaussian distribution may be selected from the candidate training set as a sample set.
Based on any of the above embodiments, determining a sample set based on a sample anchor point satisfying a gaussian distribution in a candidate training set includes:
based on the length of a preset sequence, taking any sample positioning point in a candidate training set as a starting point, and selecting a sample positioning point meeting Gaussian distribution from the candidate training set as a sample positioning point sequence;
based on the plurality of sequences of sample anchor points, a set of samples is determined.
Specifically, the preset sequence length T may be set as needed, for example, T may be set to 10. And selecting a sample positioning point satisfying Gaussian distribution from the candidate training set as a sample positioning point sequence by taking any sample positioning point in the candidate training set as a starting point.
For example, the sample positioning points are acquired in the walking process of the person, and the distance between the selected sample positioning points can be determined to meet the gaussian distribution according to the walking speed of the person. It is assumed that the selection of the sample anchor point is performed in seconds during the walking of the person. And if the length of one sample positioning point sequence is T, the T sample positioning points in the sample positioning point sequence are acquired within T seconds in the walking process, and one sample positioning point is acquired every second correspondingly.
The indoor walking speed of a normal person is 0.4-2 m/s, and a sample positioning point l acquired at the last moment can be obtainedt-1The position point l of the sample positioning point collected at the next momenttThe relation between the two points obeys Gaussian distribution, and the sample positioning point at the next moment can be selected according to the probability, and is expressed by a formula:
dmax＝vmax×Δt
wherein, P (l)t) Selection probability, x, for a sample anchor point at the next time instanttA vector of position coordinates representing time t, σ represents the standard deviation of the distance from all other points to the point as a variable, Δ t is the time difference between adjacent times, here 1 second, dmaxRepresenting the maximum distance, root, traveled by the pedestrian within the time differenceAccording to the maximum value v of the average speed of the pedestrianmaxThe maximum value is 2 m/s.
In the candidate training set, any sample positioning point is taken as a starting point lt-1Namely, the sample positioning point acquired at the time t-1, and other sample positioning points except the sample positioning point are possible sample positioning points acquired at the time t, the selection probability of each other sample positioning point is respectively calculated according to the formula, and the positioning point with the maximum selection probability is used as the sample positioning point l acquired at the time ttAccording to the selection method, points with the number meeting the length of the preset sequence are selected from the candidate training set to serve as a sample positioning point sequence.
By replacing other points as starting points, a plurality of sample positioning point sequences can be obtained according to the method. These sample localization point sequences are taken as a sample set. The number of sample anchor point sequences can be set as desired.
Based on any of the above embodiments, determining a candidate training set of an indoor positioning model based on an actual indoor position of each sample positioning point, sample WiFi signal data, and sample indoor image data includes:
determining the actual indoor position of each sample positioning point, sample WiFi signal data and sample indoor image data based on the set acquisition sequence;
and determining a candidate training set and a data test set of the indoor positioning model based on the acquisition sequence number of each sample positioning point.
Specifically, a plurality of sample positioning points, the actual position of each sample positioning point in the room, sample WiFi signal data and sample indoor image data can be acquired respectively according to a certain acquisition sequence indoors. The plurality of sample anchor points may be divided into a candidate training set and a data test set according to parity of acquisition sequence numbers of the respective sample anchor points.
For example, fig. 4 is a schematic diagram of acquiring an indoor sample positioning point provided by the present invention, as shown in fig. 4, WiFi signals and images are acquired on a trajectory line according to a set acquisition sequence, specifically, the method includes walking along a trajectory line, stopping at a point, and taking a picture of a sample positioning point at each sample positioning point by using a mobile phone camera, then searching for WiFi signals at the point, and storing all searched WiFi signal addresses, corresponding signal strengths, and real coordinates of the point in a mobile phone file, where the specific storage form of the WiFi file may be a text file:
(x-coordinate, y-coordinate, MAC address 1: intensity value, MAC address 2: intensity value.. once.)
The pictures are stored in the same folder in the mobile phone as jpg picture files. And finally, transmitting the stored WiFi text file and all images to a computer through a mobile phone for processing. The sample location points may be spaced 1 meter apart. And (4) sequentially and alternately selecting all the collected sample positioning points to obtain a candidate training set and a data test set. For example, sample samples with odd acquisition numbers may be sorted into the candidate training set, and sample samples with even acquisition numbers may be sorted into the data test set.
Based on any of the above embodiments, the loss function of the indoor positioning model is determined based on the following steps:
determining a root mean square error based on the actual position and the predicted position of the sample positioning point in the sample set;
classifying each sample positioning point in the sample set based on the sample positioning points and the minimum distance between the sample positioning points, and determining a classification error based on the cross entropy loss of each category;
based on the root mean square error and the classification error, a loss function of the indoor positioning model is determined.
In particular, a combined loss function may be employed when training an indoor positioning model. Combined loss function LnewIncluding the root mean square error LRMSEAnd a classification error LClassificationIs formulated as:
Lnew＝LRMSE+LClassification
wherein L isRMSEThe root mean square error between the predicted position of the sample positioning point output by the indoor positioning model and the actual position of the sample positioning point may be used.
Classification error LClassificationThe determination method comprises the following steps:
setting a circle area with the minimum adjacent node distance (grid size) as a radius as a category Label by taking each sample positioning point as a reference, judging whether the sample positioning points are of the same category according to the Euclidean distance between the sample positioning points, and determining the category Label of each sample positioning point, wherein the specific calculation mode is as follows:
wherein N is equal to the ratio of the area of the indoor scene to the area of a circle with grid size as the radius, N is equal to the number of all sample positioning points, and LabelkNamely the category label of the k points, the cross entropy loss is obtained by taking the category label as a standard, and the category error loss L can be obtainedClassification。
Recording the training loss of each round in the training process, and storing the model with the minimum loss as an indoor positioning model after the training loss is converged.
Based on any of the above embodiments, fig. 5 is a schematic application diagram of the indoor positioning method provided by the present invention, and as shown in fig. 5, the application of the indoor positioning method can be divided into an offline (training) stage and an online (prediction) stage.
And in the off-line stage, after data preprocessing is finished, sequentially taking WiFi data and image data from the training set according to the length of the set T, inputting the WiFi data and the image data into a cyclic neural network structure and a convolutional neural network structure, and respectively extracting high-level feature description.
The WiFi data processing steps are as follows:
(1) traversing all the acquired WiFi signal MAC addresses, recording as an address complete set, and counting 100 MAC address signal sources;
(2) rearranging and combining the signal strength values acquired by each positioning point according to the MAC address sequence of the WiFi signals in the whole set to form a uniform-dimension vector set RSSI, wherein the WiFi signal strength of the address which is not acquired is represented by a minimum value of-100;
(3) all RSSI signal strength vectors are normalized.
The image preprocessing comprises the following steps:
(1) arranging according to the WiFi acquisition point sequence;
(2) the center is cut to a size specification of 224 x 224.
And finally, after the WiFi and the images are processed respectively, sequentially and alternately selecting positioning points as training set and test set data points along the trajectory line, then selecting a fixed sequence length, and generating a continuous point trajectory according to the sequence length.
And in an online stage, an evaluation module of the algorithm inputs the preprocessed test set data, WiFi and images into a stored neural network model (indoor positioning model) and outputs corresponding position coordinates. And calculating the average Error distance between the position coordinate output by the network and the real coordinate to obtain the average precision Error of the final positioning. The specific evaluation equation is:
wherein liThe location result coordinates are predicted for the output of point i,
and N is the number of all positioning points.
Table 1 shows that the indoor positioning method in the embodiment of the present invention compares the positioning accuracy based on the WiFi signal in the data set collected in the embodiment of the present invention and in the public data set ujiindioorloc with other methods. The loss function of the invention has better generalization function, namely, the loss functions with different data and different forms have better performance.
Table 1 WiFi positioning accuracy (mean square error ± standard deviation) comparison (in meters) with other methods on different datasets for the present invention
| MLP | MLNN | P-MIMO | The invention | |
| Invention dataset | 1.20±0.83 | 1.79±1.60 | 0.73±0.75 | 0.69±0.49 |
| UJIIndoorLoc | 9.2±5.8 | 7.6±4.2 | 4.5±2.7 | 3.35±2.29 |
Table 2 shows that the indoor positioning method in the embodiment of the present invention uses different source data and a comparison of the experimental positioning accuracy of the loss function in the experimental data, which shows that the loss function in the embodiment of the present invention is applied to WiFi and image fusion, and the improvement of the positioning accuracy is significant.
TABLE 2 comparison of positioning accuracy of the method of the present invention under different data sources and loss functions
| Using source data | Loss function | Mean square error (unit meter) |
| WiFi | LRMSE | 0.71±0.74 |
| WiFi | Lnew | 0.69±0.49 |
| Image of a person | LRMSE | 3.92±2.29 |
| Image of a person | Lnew | 3.37±1.86 |
| WiFi and image | LRMSE | 0.69±0.45 |
| WiFi and image | Lnew | 0.58±0.36 |
The indoor positioning method provided by the embodiment of the invention is based on a deep learning technology, trains a tightly coupled neural network model, and fuses WiFi and image data. Firstly, extracting WiFi sequence signal data characteristics by using a cyclic neural network, simultaneously extracting image characteristics by using a convolutional neural network, splicing and combining the two characteristic descriptions to be used as input of a subsequent neural network layer, regressing and training an integral network, and outputting position point coordinates represented by an image and WiFi for positioning. The method comprises the steps of dividing positioning points into a plurality of positioning labels according to regions during regression training, introducing a classification loss function, and generating a final result by using classification loss assistance during regression.
Compared with the prior art, the embodiment of the invention has the following beneficial effects: (1) a tightly coupled fusion mode is provided, and fusion positioning is directly carried out by using WiFi and original data of images, so that the calculation steps are greatly reduced, and the efficiency is increased; (2) original data of WiFi and images are fully utilized, positioning accuracy is higher, and effects are better; (3) the problem that the influence of WiFi signal attenuation, shielding in images, illumination and the like is too large is effectively avoided.
Based on any of the above embodiments, fig. 6 is a schematic structural diagram of an indoor positioning device provided by the present invention, as shown in fig. 6, the device includes:
an obtaining unit 610, configured to obtain WiFi signal data and indoor image data of a current location;
the prediction unit 620 is configured to input WiFi signal data and indoor image data to the indoor positioning model to obtain a predicted position of a current position output by the indoor positioning model;
the indoor positioning model is used for respectively extracting signal features in WiFi signal data and image features in indoor image data, fusing the signal features and the image features and then performing positioning prediction.
According to the indoor positioning device provided by the embodiment of the invention, the signal characteristics in the WiFi signal data and the image characteristics in the indoor image data are respectively extracted through the indoor positioning model, the signal characteristics and the image characteristics are fused and then are positioned and predicted to obtain the predicted position of the current position, the problem of signal attenuation caused by positioning by independently adopting the WiFi signal data is solved, the problem of visual range limitation caused by positioning by independently adopting the indoor image data is also solved, the advantage complementation of data is realized, meanwhile, the WiFi signal data and the indoor image data are directly used for fusion positioning, and the precision and the efficiency of indoor positioning are improved.
Based on any embodiment, the indoor positioning model comprises a signal feature extraction layer, an image feature extraction layer and a fusion prediction layer;
inputting WiFi signal data and indoor image data into an indoor positioning model to obtain a predicted position of a current position output by the indoor positioning model, wherein the method comprises the following steps:
inputting WiFi signal data to a signal feature extraction layer to obtain signal features output by the signal feature extraction layer;
inputting indoor image data to an image feature extraction layer to obtain image features output by the image feature extraction layer;
and inputting the signal characteristics and the image characteristics into the fusion prediction layer to obtain the prediction position of the current position output by the fusion prediction layer.
Based on any of the above embodiments, the apparatus further comprises a training unit, the training unit comprising:
the sample set determining subunit is used for determining a sample set, wherein the sample set comprises a plurality of sample positioning point sequences, and each sample positioning point sequence comprises the actual positions of a plurality of sample positioning points, sample WiFi signal data and sample indoor image data;
the sample set training subunit is used for training the initial model based on the sample set to obtain an indoor positioning model;
the signal feature extraction layer of the initial model adopts a cyclic neural network model, the image feature extraction layer adopts a convolutional neural network model, and the fusion prediction layer adopts a fully-connected neural network.
Based on any of the above embodiments, the sample set determining subunit includes:
the candidate training set determining module is used for determining a candidate training set of the indoor positioning model based on the actual indoor position of each sample positioning point, the WiFi signal data of the sample and the indoor image data of the sample;
and the sample set determining module is used for determining the sample set based on the sample positioning points meeting the Gaussian distribution in the candidate training set.
Based on any of the above embodiments, the sample set determination module is configured to:
based on the length of a preset sequence, taking any sample positioning point in a candidate training set as a starting point, and selecting a sample positioning point meeting Gaussian distribution from the candidate training set as a sample positioning point sequence;
based on the plurality of sequences of sample anchor points, a set of samples is determined.
Based on any of the above embodiments, the candidate training set determining module is configured to:
determining the actual indoor position of each sample positioning point, sample WiFi signal data and sample indoor image data based on the set acquisition sequence;
and determining a candidate training set and a data test set of the indoor positioning model based on the acquisition sequence number of each sample positioning point.
Based on any embodiment above, the apparatus further comprises:
the loss function determining unit is used for determining a root mean square error based on the actual position and the predicted position of the sample positioning point in the sample set; classifying each sample positioning point in the sample set based on the sample positioning points and the minimum distance between the sample positioning points, and determining a classification error based on the cross entropy loss of each category; based on the root mean square error and the classification error, a loss function of the indoor positioning model is determined.
Based on any of the above embodiments, fig. 7 is a schematic structural diagram of an electronic device provided by the present invention, and as shown in fig. 7, the electronic device may include: a Processor (Processor)710, a communication Interface (Communications Interface)720, a Memory (Memory)730, and a communication Bus (Communications Bus)740, wherein the Processor 710, the communication Interface 720, and the Memory 730 communicate with each other via the communication Bus 740. Processor 710 may call logical commands in memory 730 to perform the following method:
acquiring WiFi signal data and indoor image data of a current position; inputting WiFi signal data and indoor image data into an indoor positioning model to obtain a predicted position of a current position output by the indoor positioning model; the indoor positioning model is used for respectively extracting signal features in WiFi signal data and image features in indoor image data, fusing the signal features and the image features and then performing positioning prediction.
In addition, the logic commands in the memory 730 can be implemented in the form of software functional units and stored in a computer readable storage medium when the logic commands are sold or used as independent products. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes a plurality of commands for enabling a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
The processor in the electronic device provided in the embodiment of the present invention may call a logic instruction in the memory to implement the method, and the specific implementation manner of the method is consistent with the implementation manner of the method, and the same beneficial effects may be achieved, which is not described herein again.
Embodiments of the present invention further provide a non-transitory computer-readable storage medium, on which a computer program is stored, where the computer program is implemented to perform the method provided in the foregoing embodiments when executed by a processor, and the method includes:
acquiring WiFi signal data and indoor image data of a current position; inputting WiFi signal data and indoor image data into an indoor positioning model to obtain a predicted position of a current position output by the indoor positioning model; the indoor positioning model is used for respectively extracting signal features in WiFi signal data and image features in indoor image data, fusing the signal features and the image features and then performing positioning prediction.
When the computer program stored on the non-transitory computer readable storage medium provided in the embodiments of the present invention is executed, the method is implemented, and the specific implementation manner of the method is consistent with the implementation manner of the method, and the same beneficial effects can be achieved, which is not described herein again.
The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware. With this understanding in mind, the above technical solutions may be embodied in the form of a software product, which can be stored in a computer-readable storage medium, such as ROM/RAM, magnetic disk, optical disk, etc., and includes commands for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method according to the embodiments or some parts of the embodiments.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (10)
1. An indoor positioning method, comprising:
acquiring WiFi signal data and indoor image data of a current position;
inputting the WiFi signal data and the indoor image data into an indoor positioning model to obtain a predicted position of the current position output by the indoor positioning model;
the indoor positioning model is used for respectively extracting signal features in the WiFi signal data and image features in the indoor image data, fusing the signal features and the image features and then performing positioning prediction.
2. The indoor positioning method according to claim 1, wherein the indoor positioning model includes a signal feature extraction layer, an image feature extraction layer, and a fusion prediction layer;
the inputting the WiFi signal data and the indoor image data to an indoor positioning model to obtain a predicted position of the current position output by the indoor positioning model includes:
inputting the WiFi signal data to the signal feature extraction layer to obtain signal features output by the signal feature extraction layer;
inputting the indoor image data to the image feature extraction layer to obtain image features output by the image feature extraction layer;
and inputting the signal characteristics and the image characteristics to the fusion prediction layer to obtain the prediction position of the current position output by the fusion prediction layer.
3. The indoor positioning method of claim 2, wherein the indoor positioning model is trained based on the following steps:
determining a sample set, the sample set comprising a plurality of sample anchor point sequences, each sample anchor point sequence comprising actual positions of a plurality of sample anchor points, sample WiFi signal data, and sample indoor image data;
training an initial model based on the sample set to obtain the indoor positioning model;
the signal feature extraction layer of the initial model adopts a cyclic neural network model, the image feature extraction layer adopts a convolutional neural network model, and the fusion prediction layer adopts a fully-connected neural network.
4. The indoor positioning method of claim 3, wherein the determining a sample set comprises:
determining a candidate training set of the indoor positioning model based on the actual indoor position of each sample positioning point, sample WiFi signal data and sample indoor image data;
and determining the sample set based on the sample positioning points meeting the Gaussian distribution in the candidate training set.
5. The indoor positioning method of claim 4, wherein the determining the sample set based on the sample positioning points satisfying the Gaussian distribution in the candidate training set comprises:
based on the length of a preset sequence, taking any sample positioning point in the candidate training set as a starting point, and selecting a sample positioning point satisfying Gaussian distribution from the candidate training set as a sample positioning point sequence;
determining the sample set based on a plurality of sequences of sample localization points.
6. The indoor positioning method of claim 4, wherein determining the candidate training set of the indoor positioning model based on the actual position of each sample positioning point indoors, the sample WiFi signal data and the sample indoor image data comprises:
determining the actual indoor position of each sample positioning point, sample WiFi signal data and sample indoor image data based on the set acquisition sequence;
and determining a candidate training set and a data test set of the indoor positioning model based on the acquisition sequence number of each sample positioning point.
7. The indoor positioning method according to any one of claims 1 to 6, wherein the loss function of the indoor positioning model is determined based on the following steps:
determining a root mean square error based on the actual position and the predicted position of the sample positioning point in the sample set;
classifying each sample positioning point in the sample set based on the sample positioning points and the minimum distance between the sample positioning points, and determining a classification error based on the cross entropy loss of each category;
determining a loss function for the indoor positioning model based on the root mean square error and the classification error.
8. An indoor positioning device, comprising:
the device comprises an acquisition unit, a processing unit and a processing unit, wherein the acquisition unit is used for acquiring WiFi signal data and indoor image data of a current position;
the prediction unit is used for inputting the WiFi signal data and the indoor image data into an indoor positioning model to obtain a predicted position of the current position output by the indoor positioning model;
the indoor positioning model is used for respectively extracting signal features in the WiFi signal data and image features in the indoor image data, fusing the signal features and the image features and then performing positioning prediction.
9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor, when executing the program, carries out the steps of the indoor positioning method according to any of claims 1 to 7.
10. A non-transitory computer readable storage medium, having a computer program stored thereon, wherein the computer program, when being executed by a processor, implements the steps of the indoor positioning method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110784734.3A CN113628272A (en) | 2021-07-12 | 2021-07-12 | Indoor positioning method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110784734.3A CN113628272A (en) | 2021-07-12 | 2021-07-12 | Indoor positioning method and device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113628272A true CN113628272A (en) | 2021-11-09 |
Family
ID=78379515
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110784734.3A Pending CN113628272A (en) | 2021-07-12 | 2021-07-12 | Indoor positioning method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113628272A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117474991A (en) * | 2023-10-24 | 2024-01-30 | 纬创软件(武汉)有限公司 | SpectFormer-based Poi positioning method and device |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105792353A (en) * | 2016-03-14 | 2016-07-20 | 中国人民解放军国防科学技术大学 | Image matching type indoor positioning method with assistance of crowd sensing WiFi signal fingerprint |
| CN109086350A (en) * | 2018-07-13 | 2018-12-25 | 哈尔滨工业大学 | A kind of mixed image search method based on WiFi |
| CN111028358A (en) * | 2018-10-09 | 2020-04-17 | 香港理工大学深圳研究院 | Augmented reality display method and device for indoor environment and terminal equipment |
| CN112312541A (en) * | 2020-10-09 | 2021-02-02 | 清华大学 | Wireless positioning method and system |
| CN112333642A (en) * | 2020-11-06 | 2021-02-05 | 南京邮电大学 | Indoor positioning method based on channel state information |
| CN112325883A (en) * | 2020-10-19 | 2021-02-05 | 湖南大学 | Indoor positioning method for mobile robot with WiFi and visual multi-source integration |
-
2021
- 2021-07-12 CN CN202110784734.3A patent/CN113628272A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105792353A (en) * | 2016-03-14 | 2016-07-20 | 中国人民解放军国防科学技术大学 | Image matching type indoor positioning method with assistance of crowd sensing WiFi signal fingerprint |
| CN109086350A (en) * | 2018-07-13 | 2018-12-25 | 哈尔滨工业大学 | A kind of mixed image search method based on WiFi |
| CN111028358A (en) * | 2018-10-09 | 2020-04-17 | 香港理工大学深圳研究院 | Augmented reality display method and device for indoor environment and terminal equipment |
| CN112312541A (en) * | 2020-10-09 | 2021-02-02 | 清华大学 | Wireless positioning method and system |
| CN112325883A (en) * | 2020-10-19 | 2021-02-05 | 湖南大学 | Indoor positioning method for mobile robot with WiFi and visual multi-source integration |
| CN112333642A (en) * | 2020-11-06 | 2021-02-05 | 南京邮电大学 | Indoor positioning method based on channel state information |
Non-Patent Citations (1)
| Title |
|---|
| 王鑫: "基于深度学习的图像与无线信号的融合定位关键技术研究", 基于深度学习的图像与无线信号的融合定位关键技术研究, pages 22 - 41 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117474991A (en) * | 2023-10-24 | 2024-01-30 | 纬创软件(武汉)有限公司 | SpectFormer-based Poi positioning method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111199564B (en) | Indoor positioning method and device of intelligent mobile terminal and electronic equipment | |
| CN110610166B (en) | Text region detection model training method and device, electronic equipment and storage medium | |
| CN106951484B (en) | Picture retrieval method and device, computer equipment and computer readable medium | |
| CN111862213A (en) | Positioning method and device, electronic equipment and computer readable storage medium | |
| US11113896B2 (en) | Geophysical sensor positioning system | |
| CN111639968B (en) | Track data processing method, track data processing device, computer equipment and storage medium | |
| CN110136058B (en) | Drawing construction method based on overlook spliced drawing and vehicle-mounted terminal | |
| CN110555428B (en) | Pedestrian re-identification method, device, server and storage medium | |
| CN111339976B (en) | Indoor positioning method, device, terminal and storage medium | |
| CN111664848B (en) | Multi-mode indoor positioning navigation method and system | |
| CN112464912B (en) | Robot end face detection method based on YOLO-RGGNet | |
| US11756205B2 (en) | Methods, devices, apparatuses and storage media of detecting correlated objects involved in images | |
| Alitaleshi et al. | EA-CNN: A smart indoor 3D positioning scheme based on Wi-Fi fingerprinting and deep learning | |
| WO2023231991A1 (en) | Traffic signal lamp sensing method and apparatus, and device and storage medium | |
| CN110647675A (en) | Method and device for recognition of stop point and training of prediction model and storage medium | |
| Nowakowski et al. | Topological localization using Wi-Fi and vision merged into FABMAP framework | |
| US20220300774A1 (en) | Methods, apparatuses, devices and storage media for detecting correlated objects involved in image | |
| CN114943870A (en) | Training method and device of line feature extraction model and point cloud matching method and device | |
| CN113628272A (en) | Indoor positioning method and device, electronic equipment and storage medium | |
| CN116824641A (en) | Gesture classification method, device, equipment and computer storage medium | |
| CN111310595A (en) | Method and apparatus for generating information | |
| Tsai et al. | Design of vision‐based indoor positioning based on embedded system | |
| CN111968030B (en) | Information generation method, apparatus, electronic device and computer readable medium | |
| CN116824686A (en) | Action recognition method and related device | |
| Nguyen et al. | An algorithm using YOLOv4 and DeepSORT for tracking vehicle speed on highway |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |