KR102572895B1 - Apparatus for PDR Based on Deep Learning using multiple sensors embedded in smartphones and GPS location signals and method thereof - Google Patents

Abstract

The present invention relates to a deep learning-based PDR positioning device and method using multiple sensors of a smartphone and a GPS location signal. According to the present invention, the deep learning-based PDR positioning device receives a GPS signal last received outside a building from a smartphone owned by a user, and sets a location coordinate value extracted from the GPS signal as a starting point; A data acquisition unit that receives sensor values measured every time the user walks indoors after the GPS signal is acquired and generates input data by pre-processing the sensor values; A learning unit that predicts the GPS longitude change by inputting and predicts the GPS latitude change by inputting the input data to the second learning model, and the user's movement by applying the predicted GPS longitude change and GPS latitude change to the starting point. It includes a location positioning unit for locating a path.
According to the present invention, by accumulating data according to the location of the smartphone that the user mainly uses while walking (during a call, texting, in a pocket, etc.), indoor location positioning can be performed without considering a separate calculation method according to various smartphone locations. It is possible and effective estimation is possible in situations where external communication is difficult, such as in a disaster environment, because additional communication equipment such as a Wi-Fi access point (AP) used in existing indoor positioning is not required.

Description

Apparatus for PDR Based on Deep Learning using multiple sensors embedded in smartphones and GPS location signals and method thereof}

The present invention relates to a deep learning-based PDR positioning device and method using multiple sensors and GPS location signals of a smartphone, and more specifically, combining a sensor value and a GPS signal that can be obtained when walking with a smartphone It relates to a deep learning-based PDR positioning device and method for positioning a user's position indoors using data.

Pedestrian Dead Reckoning (hereinafter referred to as "PDR") technology uses various sensors to identify movement information such as speed, acceleration, direction and distance of movement of a person, and calculates the relative position from the starting point. means technology.

The conventional pedestrian positioning (PDR) is updated according to Equation 1 below.

here represents the initial location of the smartphone user, Represents the estimated user's step length for every mth step, represents the direction angle of the device estimated at every mth step.

And, the stride ( ) is estimated using Equation 2 below.

here, represents the calibration constant, is the maximum value of the accelerometer, represents the minimum value of the accelerometer.

In addition, the direction angle ( ) is estimated using Equation 3 below and is affected by noise.

Conventional pedestrian positioning (PDR) has problems in estimating the length of a user's step and drift and accumulation error of a sensor generated along a walking path.

Therefore, in order to solve the above problem, a technique of estimating the user's current location by combining location positioning (PDR) using a sensor of a smartphone and an access point (AP) inside a building has been extensively researched. .

However, the location estimation method combining access points (AP) has the inconvenience of having to readjust the location one by one according to the user's step length, drift, and accumulation error. In addition, if an access point (AP) cannot be used, the exact location cannot be determined, and the unique biomechanical information that appears each time a user walks cannot be reflected in the stride length, making it difficult to estimate an accurate location indoors. .

The background technology of the present invention is disclosed in Korean Patent Registration No. 10-1452373 (Announced on October 13, 2014).

As described above, according to the present invention, to provide a deep learning-based PDR positioning device and method for locating a user's position indoors using data obtained by combining a sensor value and a GPS signal obtained when walking with a smartphone It is for

According to an embodiment of the present invention for achieving this technical problem, in the deep learning-based PDR positioning device using multiple sensors and GPS location signals of a smartphone, the GPS signal last received from outside the building from the smartphone owned by the user A location setting unit that receives and sets the location coordinate value extracted from the GPS signal as a starting point, receives a sensor value measured every time the user walks indoors after the GPS signal is acquired, and preprocesses the sensor value a data acquisition unit that generates input data by using a data acquisition unit to predict GPS longitude change by inputting the input data to a first learning model that has been previously learned, and a learning unit to predict GPS latitude change by inputting the input data to a second learning model and a positioning unit for locating the user's moving path by applying the predicted GPS longitude change and GPS latitude change to the starting point.

Further comprising a database for collecting and storing sensor values obtained using at least one sensor among acceleration, gyroscope, and geomagnetism installed in the smartphone and position coordinate values of GPS signals,

The learning unit pre-processes the sensor values stored in the database to obtain input data indicating the position and direction of the smartphone according to the change in GPS latitude and longitude of the previous and current steps, and converts the obtained input data into a first learning model. and outputting the GPS longitude change through the first learning model and outputting the GPS latitude change through the second learning model.

The input data may include at least one of a change amount of acceleration, an average amount of a total amount of acceleration, a geomagnetic value, and a change amount of a gyroscope.

The user's current GPS coordinate value estimated by applying the predicted GPS longitude change and GPS latitude change may be displayed in seconds using the following equation.

Here, Degree denotes degrees, Minutes denotes minutes, and C denotes a GPS coordinate value, and cut() denotes truncating decimal places.

The first learning model and the second learning model are implemented in the form of a Multi-Layer Perceptron (MLP) algorithm and use Tensor Flow. When the user is walking while sending text messages (texting mode) , In case of walking while making a call (talking mode), in case of walking while shaking a part of the body (swing mode), and in case of walking with a smartphone in a pocket (pocket mode), the nth A sensor value measured for each step is normalized with a characteristic value to estimate a GPS longitude change and a GPS latitude change at the nth step, respectively.

In addition, according to an embodiment of the present invention, in the deep learning-based PDR positioning method using a PDR positioning device, a GPS signal received last outside a building from a smartphone owned by a user is received, and a position extracted from the GPS signal is received. Setting a coordinate value as a starting point, receiving a sensor value measured whenever a user walks indoors after the GPS signal is acquired, and generating input data by preprocessing the sensor value, and generating input data by pre-processing the sensor value. 1 inputting the input data to a learning model to predict the GPS longitude change, inputting the input data to a second learning model to predict the GPS latitude change, and setting the predicted GPS longitude change and GPS latitude change to the starting point and positioning the user's movement path by applying to .

In this way, according to the embodiment of the present invention, the drift problem of the sensor, which is a problem in the existing pedestrian positioning (PDR), is corrected by the GPS signal, and it is robust to the effect of the accumulated error of the sensor drift and more accurate indoor positioning is possible. . In addition, according to an embodiment of the present invention, an economical pedestrian positioning (PDR) system can be implemented because additional equipment is not required to estimate the stride length.

In addition, according to an embodiment of the present invention, by accumulating data according to the smartphone location (during a call, texting, in a pocket, etc.) that the user mainly uses when walking, separate calculation methods are not considered according to various smartphone locations. Indoor positioning is possible without the need for indoor positioning, and effective estimation is possible in situations where external communication is difficult, such as in a disaster environment, because there is no need for additional communication equipment such as a Wi-Fi access point (AP) used in existing indoor positioning.

1 is a configuration diagram for explaining a deep learning-based PDR positioning device according to an embodiment of the present invention.
2 is a flowchart illustrating a method for positioning a user's position indoors using a deep learning-based PDR positioning device according to an embodiment of the present invention.
3 is an exemplary view showing a user's common smartphone location while walking.
4 is a diagram schematically illustrating a first learning model and a second learning model according to an embodiment of the present invention.

Hereinafter, preferred embodiments according to the present invention will be described in detail with reference to the accompanying drawings. In this process, the thickness of lines or the size of components shown in the drawings may be exaggerated for clarity and convenience of description.

In addition, terms to be described later are terms defined in consideration of functions in the present invention, which may vary according to the intention or custom of a user or operator. Therefore, definitions of these terms will have to be made based on the content throughout this specification.

Hereinafter, a PDR positioning device based on deep learning according to an embodiment of the present invention will be described in more detail with reference to FIG. 1 .

1 is a configuration diagram for explaining a deep learning-based PDR positioning device according to an embodiment of the present invention.

As shown in FIG. 1, the deep learning-based PDR positioning device 100 according to an embodiment of the present invention includes a location setting unit 110, a data acquisition unit 120, a learning unit 130, and a positioning unit ( 140) and a database 150.

First, the position setting unit 110 sets a user's positioning starting point. To elaborate, the location setting unit 110 receives location coordinate values extracted from GPS signals from the user's smart phone. At this time, the GPS signal represents the last received signal when the user is located outside the building. Then, the location setting unit 110 sets the received location coordinate value as a starting point.

The data acquisition unit 120 receives sensor values measured through a plurality of sensors built into the smart phone after receiving the GPS signal, and pre-processes the received sensor values to generate input data.

Here, the plurality of sensors include at least one of an acceleration sensor, a gyroscope sensor, and a geomagnetic sensor, and the input data indicates the location and direction of the smartphone according to the amount of change in GPS latitude and longitude of the previous and current steps.

The learning unit 130 builds two learning models, inputs sensor values obtained through a plurality of sensors and location coordinate values of GPS signals to the built two learning models, and outputs GPS longitude change and GPS latitude change, respectively. learn to do

To elaborate, the learning unit 130 inputs input data to the first model implemented in the form of a Multi-Layer Perceptron (MLP) algorithm and learns to output a GPS longitude variation.

In addition, the learning unit 130 inputs input data to a second model implemented in the form of a multi-layer perceptron (MLP) algorithm and learns to output a GPS latitude change amount.

The positioning unit 140 measures the user's movement path by applying the GPS longitude change and the GPS latitude change respectively obtained from the first model and the second model to the starting point.

Finally, the database 150 collects sensor values measured using a smart phone and location coordinate values of GPS signals. In detail, the database 150 collects sensor values measured through a plurality of sensors built into the smart phone whenever the user walks normally and GPS location coordinate values obtained as the user's location changes.

Hereinafter, a method of positioning a user's position indoors using the deep learning-based PDR positioning device 100 according to an embodiment of the present invention will be described in detail with reference to FIGS. 2 to 4 .

2 is a flowchart illustrating a method for positioning a user's location indoors using a deep learning-based PDR positioning device according to an embodiment of the present invention, and FIG. 3 shows a user's general smartphone location while walking. is an example

As shown in FIG. 2 , the deep learning-based PDR positioning device 100 according to an embodiment of the present invention collects sensor values and GPS location coordinate values measured according to user steps (S210).

The deep learning-based PDR positioning device 100 acquires measured sensor values and GPS location coordinate values from a smartphone owned by the user whenever the user walks.

To elaborate, since the smart phone includes at least one sensor among an accelerometer sensor, a gyroscope sensor, and a geomagnetic sensor, the plurality of sensors are used to provide motion information according to the location of the smartphone, direction, movement speed, etc. can be obtained.

As shown in FIG. 3, the location of the smartphone is when walking while sending a text message (texting mode), when walking while making a call (talking mode), when walking while shaking a part of the body (swing mode), and when walking while making a call (swing mode). It is different according to any one mode among the case of walking with the smartphone in the pocket mode, and the sensor value according to the location of the smartphone is measured differently.

Therefore, the deep learning-based PDR positioning device 100 receives sensor values and GPS location coordinate values obtained from points A and B, respectively, when the user moves one step from point A to point B, and receives the received sensor values. And the GPS location coordinate values are stored in the database 150 .

Next, the learning unit 130 learns the first learning model and the second learning model using the sensor values and GPS location coordinate values stored in the database 150 (S220).

To elaborate, the learning unit 130 builds a first learning model and a second learning model implemented in the form of a multi-layer perceptron (MLP) algorithm.

A multi-layer perceptron (MLP) has a structure of a neural network including one or more intermediate layers between an input layer and an output layer, and values transmitted from the input layer are transmitted to the output layer through the hidden layer. At this time, the multi-layer perceptron (MLP) compares the result value output through the output layer with the actual value, determines and applies a weight value that minimizes an error between the result value and the actual value. And, in the multi-layer perceptron (MLP), learning ends when the error between the result value of the output layer and the actual value falls within the allowable range.

Accordingly, the learning unit 130 inputs the input data obtained by preprocessing the sensor values stored in the database 150 to the first learning model, and acquires the GPS longitude change through the first learning model. Then, the learning unit 130 compares the GPS longitude change obtained through the first learning model with the GPS position coordinate value stored in the database, and then determines that the error between the GPS longitude change and the GPS position coordinate value stored in the database is within the allowable range. Iteratively learns the map until it is relevant.

In addition, the learning unit 130 inputs the input data obtained by preprocessing the sensor values stored in the database 150 to the second learning model, and acquires the GPS latitude change through the second learning model. Then, the learning unit 130 compares the GPS latitude change amount obtained through the second learning model with the GPS location coordinate value stored in the database, and then the error between the GPS latitude change amount and the GPS location coordinate value stored in the database falls within the allowable range. Iteratively learns the map until it is relevant.

Here, the input data input to the first learning model and the second learning model includes at least one of an acceleration change amount, an average amount of a total amount of acceleration magnitude, a geomagnetic value, and a gyroscope change amount.

4 is a diagram schematically illustrating a first learning model and a second learning model according to an embodiment of the present invention.

As shown in FIG. 4 , the first learning model and the second learning model according to an embodiment of the present invention are implemented using Tensor Flow. To explain this again, the sensor values stored in the database 150 are expressed in different measurement units. Therefore, as shown in FIG. 4, the learning unit 130 provides a batch normalization layer such as the Mul and Add layers shown in black, and uses the provided batch normalization layer for every nth step. The measured sensor values are normalized to the characterized values.

When estimating the location of the user who has entered the room after step S220 is completed, the location setting unit 110 sets a starting point using the GPS signal received from the user's smartphone (S230).

When the user enters the room, the user's smart phone can no longer acquire the user's location coordinates through GPS. Therefore, the location setting unit 110 receives the last GPS signal received outside the building from the smartphone, and sets the location coordinate value obtained from the received GPS signal as a starting point.

Next, the data acquisition unit 120 obtains input data by pre-processing sensor values measured through a plurality of sensors built into the smart phone (S240).

In other words, the data acquisition unit 120 receives a sensor value generated after receiving the last GPS signal. Then, the data acquisition unit 120 obtains input data reconstructed into data usable for positioning by pre-processing the received sensor values.

Here, the input data includes at least one of a change amount of acceleration, an average amount of the total amount of acceleration, a geomagnetic value, and a change amount of a gyroscope.

To explain this in more detail, when the user moves one step from point A to point B, a plurality of sensors built into the smartphone acquire at least one sensor value from among acceleration, gyroscope, and geomagnetism, and the acquired sensor value is stored in the dipstick. It is delivered to the running-based PDR positioning device 100.

Then, the data acquisition unit 120 obtains an average amount of acceleration change according to the user's step length and acceleration magnitude, and obtains a geomagnetic value indicating the user's moving direction and a change amount of the gyroscope.

Next, the positioning unit 140 inputs the input data including the acquired acceleration change amount, the average amount of acceleration magnitude, the geomagnetism value, and the change amount of the gyroscope to the first learning model and the second learning model where learning has been completed, respectively. A GPS longitude change and a GPS latitude change are obtained from the first learning model and the second learning model (S250).

Meanwhile, in general, latitude coordinate values and longitude coordinate values according to GPS signals are expressed using decimal points. Using latitude coordinates and longitude coordinates displayed to decimal places, the latitude change and longitude change measured as the user moves one step have too insignificant differences, making it difficult to locate the user's location.

Therefore, in an embodiment of the present invention, the GPS longitude change and the GPS latitude change are displayed in units of seconds using Equation 4 below.

Here, Degree denotes degrees, Minutes denotes minutes, and C denotes a GPS coordinate value, and cut() denotes truncating decimal places.

Next, the positioning unit 140 determines the user's moving path by applying the GPS latitude change predicted from the first learning model and the GPS latitude change predicted from the second learning model to the starting point (S260).

For example, if the GPS location coordinate values corresponding to the user's starting point are 37° 29' 45.6"N, 126° 57' 20.6"E, and the GPS longitude change predicted from the first learning model is +1 second (+1" ), and the GPS latitude change amount predicted from the second learning model is +1 second (+1").

Then, the location positioning unit 140 may acquire the location coordinates of the current user by correcting the change amount predicted from the first learning model and the second learning model. That is, since the positioning unit 140 can estimate the user's moving distance and direction angle using the acquired positional coordinates and sensor values, it is possible to position the user's moving path indoors.

In this way, the deep learning-based PDR positioning device according to an embodiment of the present invention is robust against the influence of sensor drift accumulation error by correcting the drift problem of the sensor, which is a problem in existing pedestrian positioning (PDR), with a GPS signal. Accurate indoor positioning is possible. In addition, according to an embodiment of the present invention, an economical pedestrian positioning (PDR) system can be implemented because additional equipment is not required to estimate the stride length.

In addition, the deep learning-based PDR positioning device according to an embodiment of the present invention accumulates data according to the smartphone location (during a call, texting, in a pocket, etc.) that the user mainly uses when walking, thereby determining various smartphone locations. Therefore, indoor positioning is possible without considering a separate calculation method, and additional communication equipment such as a Wi-Fi access point (AP) used in existing indoor positioning is not required, so it is effective in situations where external communication is difficult, such as in a disaster environment. it is possible to estimate

Although the present invention has been described with reference to the embodiments shown in the drawings, this is only exemplary, and those skilled in the art will understand that various modifications and equivalent other embodiments are possible therefrom. will be. Therefore, the true technical protection scope of the present invention should be determined by the technical spirit of the claims below.

100: PDR positioning device
110: position setting unit
120: data acquisition unit
130: learning unit
140: positioning unit
150: database

Claims (10)

In the deep learning-based PDR positioning device using multiple sensors and GPS location signals of a smartphone,
A location setting unit that receives the GPS signal last received outside the building from a smartphone owned by the user and sets the location coordinate value extracted from the GPS signal as a starting point;
a data acquiring unit receiving a sensor value measured whenever a user walks indoors after the point in time at which the GPS signal is acquired, and generating input data by pre-processing the sensor value;
A learning unit that inputs the input data into a first learning model that has been previously learned to predict a GPS longitude change amount, and inputs the input data to a second learning model to predict a GPS latitude change amount; and
A location positioning unit for locating a user's moving path by applying the predicted GPS longitude change and GPS latitude change to the starting point;
The first learning model and the second learning model,
Implemented in the form of a Multi-Layer Perceptron (MLP) algorithm, using Tensor Flow, when a user walks while sending text messages (texting mode), when walking while talking on the phone (talking mode), In response to either mode of walking while shaking a part of the body (swing mode) or walking with a smartphone in the pocket (pocket mode), the sensor value measured at every n-th step is normalized to a characteristic value A deep learning-based PDR positioning device that predicts the change in GPS longitude and the change in GPS latitude at the n-th step, respectively. According to claim 1,
Further comprising a database for collecting and storing sensor values obtained using at least one sensor among acceleration, gyroscope, and geomagnetism installed in the smartphone and position coordinate values of GPS signals,
The learning unit,
The sensor values stored in the database are pre-processed to obtain input data representing the position and direction of the smartphone according to the change in GPS latitude and longitude of the previous and current steps, and the obtained input data are used for first learning model and second learning A deep learning-based PDR positioning device for learning to output a GPS longitude change through the first learning model and output a GPS latitude change through the second learning model by inputting the input to the model. According to claim 1 or 2,
The input data is
A deep learning-based PDR positioning device including at least one of an acceleration change amount, an average amount of a total amount of acceleration magnitude, a geomagnetic value, and a gyroscope change amount. According to claim 1 or 2,
The user's current GPS coordinate value estimated by applying the predicted GPS longitude change and GPS latitude change,
Deep learning-based PDR positioning device displayed in units of seconds using the following equation.

(Where Degree is degrees, Minutes is minutes, and C indicates GPS coordinate values, and cut() indicates rounding off decimal places.) delete In the deep learning-based PDR positioning method using the PDR positioning device,
Receiving a GPS signal last received outside the building from a smartphone possessed by a user, and setting a location coordinate value extracted from the GPS signal as a starting point;
Receiving a sensor value measured whenever a user walks indoors after the point in time at which the GPS signal is acquired, and generating input data by pre-processing the sensor value;
predicting a GPS longitude change by inputting the input data to a first learning model that has been previously learned, and predicting a GPS latitude change by inputting the input data to a second learning model; and
Positioning a user's movement path by applying the predicted GPS longitude change and GPS latitude change to the starting point;
The first learning model and the second learning model,
Implemented in the form of a Multi-Layer Perceptron (MLP) algorithm, using Tensor Flow, texting mode, talk mode, swing mode, and pocket mode ), a deep learning-based PDR positioning method that predicts GPS longitude change and latitude change at the n-th step by normalizing the sensor value measured at every n-th step to a characteristic value corresponding to any one of the modes. According to claim 6,
Collecting sensor values obtained using at least one sensor among acceleration, gyroscope, and geomagnetism included in the smartphone and position coordinate values of GPS signals, and
The sensor values are preprocessed to obtain input data representing the position and direction of the smartphone according to the change in GPS latitude and longitude between the previous and current steps, and input the obtained input data to the first learning model and the second learning model. and learning to output a GPS longitude change through the first learning model and output a GPS latitude change through the second learning model. According to claim 6 or 7,
The input data is
A deep learning-based PDR positioning method including at least one of a change in acceleration, an average amount of the total amount of acceleration, a geomagnetic value, and a change in a gyroscope. According to claim 6 or 7,
The user's current GPS coordinate value estimated by applying the predicted GPS longitude change and GPS latitude change,
Deep learning-based PDR positioning method expressed in units of seconds using the following equation.

(Where Degree is degrees, Minutes is minutes, and C indicates GPS coordinate values, and cut() indicates rounding off decimal places.) delete