KR20230113884A - Indoor positioning system using multiple signals of smart phone and its operation method - Google Patents

Abstract

An indoor positioning system according to an embodiment of the present invention collects multiple signals sensed in correspondence with location information from a portable terminal, performs a corresponding imaging process, and outputs positioning information based on the similarity between the imaged data. Learning a similarity-based neural network to build an indoor coordinate positioning model based on an artificial intelligence neural network, and providing indoor positioning information using the indoor coordinate positioning model based on the artificial intelligence neural network.

Description

Indoor positioning system using multiple signals of smart phone and its operation method

The present invention relates to an indoor positioning system and its operating method. More specifically, an indoor positioning system using multiple signals of a smartphone that searches for indoor location information by mapping indoor location information corresponding to multiple signals that can be received by a smartphone and generating a radio wave map fingerprint corresponding to the multiple signals. and its operating method.

Recently, as GPS (Global Positioning System) has begun to be installed in most smartphones, location-based services have begun to be used in various areas of real life. In particular, as GPS can be used freely in various applications (APPs) of smartphones, the combination of location information and various services has been made.

However, since the signals generated by GPS satellites become very weak as they pass through more than 20,000 km of atmosphere, it is impossible to penetrate the interior of a building. Therefore, a location-based service such as a road guidance (navigation) service cannot be used indoors. This is because most location-based services depend on GPS.

To solve this problem, various indoor positioning methods have been proposed. The indoor positioning technology known so far is a method of positioning an indoor position using only wireless LAN (WiFi) AP information collected for wireless base station and indoor positioning, and a method of positioning indoors and outdoors using GPS/base station/WiFi/magnetic field information continuously. Positioning methods and the like are proposed.

In addition, data collection for indoor positioning is exemplified by fingerprinting methods in which a related person directly collects data or collects wireless LAN (WiFi) AP signal information using related software to create a map.

However, the accuracy of the indoor positioning technology is still insufficient even at the present time when the activity time in the indoor space is rapidly increasing according to the increase in the size of the building.

In particular, building and maintaining a map by collecting indoor signals is expensive because accurate drawings of buildings and skilled experts are required. There are deficiencies in actual use.

Furthermore, positioning technology using a geomagnetic field has poor versatility, has no uniqueness according to location, and can only be located through a predetermined path. Therefore, there is a problem in that all maps must be rebuilt when some magnetic fields are changed.

For this reason, there are problems that are difficult to apply in special environments such as temporarily constructed spaces, tunnels, parking lots, and exhibition halls.

The present invention has been made to solve the above problems, and collects various signals collected from a typical portable terminal as multiple signals, and uses an artificial intelligence neural network-based positioning network built using such multi-signal big data. Accordingly, an object of the present invention is to provide an indoor coordinate positioning system that can be applied to various environments while accurately locating indoor coordinates at low cost.

A system according to an embodiment of the present invention for solving the above problems is an indoor positioning system, comprising: a multi-signal collector for collecting multi-signals sensed in response to location information from a portable terminal; an imaging processing unit that performs imaging processing corresponding to the multiple signals; a similarity-based neural network learning unit constructing an indoor coordinate positioning model based on an artificial intelligence neural network to output positioning information based on a similarity between the imaged data and the imaged data; and a learning model-based indoor positioning service provider providing indoor positioning information using the artificial intelligence neural network-based indoor coordinate positioning model.

The present invention is a concept of globally sharing a table mapping multi-signal and indoor location information (what kind of building number, etc. (what building number, number of apartment buildings, etc.) is entered so that it can be spread globally, so that all the public can acquire indoor location information with the Indoor Name Server built in this way.

In addition, the present invention constructs a technology for storing radio map fingerprints used for indoor positioning in large quantities around the world and searching them at high speed, and an in-memory database using the same. To this end, when a request including a fingerprint generated by a user is received, an inverse index technology is included to quickly select a fingerprint that is likely to be similar among stored fingerprints. This inverse index is generated for each radio source included in the radio map fingerprint and stored separately. A hash map technology is included to store radio map fingerprints in large quantities and store all radio map fingerprints to enable high-speed random search. In addition, other indexes can be added as needed, such as location-based indexes and user-based indexes.

According to an embodiment of the present invention, it is possible to build an indoor coordinate positioning model based on an artificial intelligence neural network that outputs positioning information by imaging multiple signals sensed in correspondence with location information from a portable terminal, and a learning model based thereon. Based indoor positioning service can be provided.

Accordingly, various signals collected in a typical portable terminal are collected as multi-signals, and by using an artificial intelligence neural network-based positioning network built using such multi-signal big data, indoor coordinates are accurately located at low cost and in various environments. An indoor coordinate positioning system that can be applied to can be provided.

1 is a diagram schematically illustrating a system according to an embodiment of the present invention.
2 is a diagram illustrating a service platform for implementing an embodiment of the present invention.
3 is a diagram for explaining a multi-signal in more detail according to an embodiment of the present invention.
4 is a diagram for explaining a neural network construction process according to an embodiment of the present invention.
5 to 10 are diagrams for explaining characteristics and positioning processes of a neural network constructed according to an embodiment of the present invention.
11 is a diagram for explaining a location determination service process according to an embodiment of the present invention.
12 is a diagram for explaining implementation examples of various service systems according to an embodiment of the present invention.

The following merely illustrates the principles of the present invention. Therefore, those skilled in the art can invent various devices that embody the principles of the present invention and fall within the concept and scope of the present invention, even though not explicitly described or shown herein. In addition, it is to be understood that all conditional terms and embodiments listed herein are, in principle, expressly intended only for the purpose of making the concept of the present invention understood, and not limited to such specifically listed embodiments and conditions. It should be.

Further, it should be understood that all detailed descriptions reciting specific embodiments, as well as principles, aspects and embodiments of the present invention, are intended to encompass structural and functional equivalents of these matters. In addition, it should be understood that such equivalents include not only currently known equivalents but also equivalents developed in the future, that is, all devices invented to perform the same function regardless of structure.

Thus, for example, the block diagrams herein are to be understood as representing conceptual views of exemplary circuits embodying the principles of the present invention. Similarly, all flowcharts, state transition diagrams, pseudo code, etc., are meant to be tangibly represented on computer readable media and represent various processes performed by a computer or processor, whether or not the computer or processor is explicitly depicted. It should be.

The functions of various elements shown in the drawings including functional blocks represented by processors or similar concepts may be provided using dedicated hardware as well as hardware capable of executing software in conjunction with appropriate software. When provided by a processor, the functionality may be provided by a single dedicated processor, a single shared processor, or a plurality of separate processors, some of which may be shared.

In addition, the explicit use of terms presented as processor, control, or similar concepts should not be construed as exclusively citing hardware capable of executing software, but without limitation, digital signal processor (DSP) hardware, ROM for storing software (ROM), random access memory (RAM) and non-volatile memory. Other hardware for the governor's use may also be included.

In the claims of this specification, components expressed as means for performing the functions described in the detailed description include, for example, a combination of circuit elements performing the functions or all types of software including firmware/microcode, etc. It is intended to include any method that performs the function of performing the function, combined with suitable circuitry for executing the software to perform the function. Since the invention defined by these claims combines the functions provided by the various enumerated means and is combined in the manner required by the claims, any means capable of providing such functions is equivalent to that discerned from this disclosure. should be understood as

The above objects, features and advantages will become more apparent through the following detailed description in conjunction with the accompanying drawings, and accordingly, those skilled in the art to which the present invention belongs can easily implement the technical idea of the present invention. There will be. In addition, in describing the present invention, if it is determined that a detailed description of a known technology related to the present invention may unnecessarily obscure the subject matter of the present invention, the detailed description will be omitted.

The present invention is a concept of globally sharing a table mapping multi-signal and indoor location information (what kind of building number, etc. (what building number, number of apartment buildings, etc.) is entered so that it can be spread globally, so that all the public can acquire indoor location information with the Indoor Name Server built in this way.

In addition, the present invention constructs a technology for storing radio map fingerprints used for indoor positioning in large quantities around the world and searching them at high speed, and an in-memory database using the same. To this end, when a request including a fingerprint generated by a user is received, an inverse index technology is included to quickly select a fingerprint that is likely to be similar among stored fingerprints. This inverse index is generated for each radio source included in the radio map fingerprint and stored separately. A hash map technology is included to store radio map fingerprints in large quantities and store all radio map fingerprints to enable high-speed random search. In addition, other indexes can be added as needed, such as location-based indexes and user-based indexes.

Hereinafter, a preferred embodiment according to the present invention will be described in detail with reference to the accompanying drawings.

1 is a conceptual diagram schematically illustrating an entire system according to an embodiment of the present invention.

The entire system according to an embodiment of the present invention includes a portable terminal 200, an indoor coordinate positioning system 100, and a service providing device 300.

The indoor coordinate positioning system 100 may be connected to each portable terminal 200 and the service providing device 300 through a wired/wireless network to provide an indoor coordinate positioning service according to an embodiment of the present invention, and communicate with each other. can be performed.

Here, each network is a Local Area Network (LAN), a Wide Area Network (WAN), a Value Added Network (VAN), a Personal Area Network (PAN), a mobile communication network ( It can be implemented in all types of wired/wireless networks, such as a mobile radiocommunication network) or a satellite communication network.

In addition, the portable terminal 200 may be an individual device of any one of a mobile phone, a smart phone, a smart pad, and a PDA (Personal Digital Assistants), and transmits various sensor information collected from the environment indoors. It is composed of multiple signals for positioning, and the composed multiple signals together with location mapping information can be transmitted to the indoor coordinate positioning system 100.

In an embodiment of the present invention, the multiple signals may include at least one of a Wi-Fi signal, a Bluetooth signal, a GPS/GLONASS signal, a mobile network signal, a geomagnetic sensor signal, an acceleration sensor signal, and a barometer signal. The multi-signal may be configured by matrix combining signals of various environmental sensors, which are typically provided in ).

In addition, the indoor coordinate positioning system 100 can build an artificial intelligence neural network-based indoor coordinate positioning model through learning of multiple signals and location mapping information of the portable terminal 200, and indoor coordinate positioning based on an artificial intelligence neural network. A model-based indoor positioning service may be provided through the service providing device 300 .

More specifically, the indoor coordinate positioning system 100 includes a multi-signal collection unit 110, an imaging processing unit 120, a similarity-based neural network learning unit 130, and a learning model-based indoor positioning service providing unit 140. .

First, the multi-signal collection unit 110 collects multi-signals sensed from the portable terminal 200 corresponding to location information.

Here, the multi-signal may be information configured by converting all signals receivable from a smartphone into <KEY, VALUE> pairs, and may be normalized regardless of the type of smartphone.

Then, the imaging processing unit 120 performs imaging processing corresponding to the multiple signals.

Accordingly, the similarity-based neural network learning unit 130 builds an artificial intelligence neural network-based indoor coordinate positioning model to output positioning information based on the imaged data and the similarity between the imaged data.

The learning model-based indoor positioning service providing unit 140 provides indoor positioning information using the artificial intelligence neural network-based indoor coordinate positioning model.

Here, the imaging processing unit may include a process of generating a sensor image map corresponding to the multiple signals to which the location information is mapped.

The sensor image map may include constellation image objects having different color sizes and concentrations according to the types, numbers, and intensities of the multiple signals.

According to the configuration of the constellation image object, the normalized multiple signals can be converted into one constellation image, and the constellation image can serve as a signal map representing the collected location. In addition, constellation images can be converted into big data and effectively used to build an artificial intelligence neural network.

Further, the similarity-based neural network learning unit 130 uses a multi-dimensional activation function for similarity comparison between the constellation image objects to learn positioning prediction information in a multi-dimensional manner, a positioning neural network based on deep learning of the constellation image objects. , PNN) model can be built.

The location neural network model according to an embodiment of the present invention may be a PERCEPTRON neural network specialized for constellation image comparison, and may be configured such that there are perceptrons corresponding to all stars (image object pixels) of each image.

In addition, the constellation image may be an image connecting signal pixels mapped to the same location information, and an accelerated learning process may be performed by optimizing the perceptron learning and comparison process based thereon.

Furthermore, the similarity-based neural network learning unit 130 constructs a neural network by applying a three-dimensional or multi-dimensional activation function corresponding to two inputs, and optimizes a function of DISCRETE to form more accurate positioning information.

For example, the similarity-based distance can be calculated as the two constellation images are similar, and the similarity-based distance can be calculated as far as the shape of the constellation images is different. If learning is performed to predict the obtained location information, it is possible to provide a specialized location location service to enable location location in various special environments.

Also, the learning model-based indoor positioning service provider 140 may provide various service information based on indoor positioning information to the service providing device 300 according to a request from the service providing device 300 .

For example, service information includes indoor and outdoor positioning information provided using indoor positioning information, grid address search information, XR content service information, indoor and outdoor 3D modeling information, CCTV image analysis information, robot guidance information, disaster safety information, construction management cloud Various service information such as information may be exemplified.

According to the construction of such a system, the indoor coordinate positioning system 100 according to an embodiment of the present invention can build models suitable for various buildings and special environments, and has the advantage of reducing construction cost and time.

2 is a diagram illustrating a service platform for implementing an embodiment of the present invention.

Referring to FIG. 2 , the service platform according to an embodiment of the present invention may be divided into a user layer, a service layer, and a core layer, and the indoor coordinate positioning system 100 and the service providing device 300 interwork to provide individual, Service processes specialized for each service target such as building owners, advertisers, service companies, governments, and local governments can be performed based on the platform.

In addition, in the core layer, the multiple signals of the portable terminal 200 are converted into big data such as a global signal map, a signal map for each group, a signal map for each customer, and the like, and based on this, a positioning engine for each attribute can be configured. A floor number estimating unit according to another embodiment of the present invention is further provided, so that not only the two-dimensional positioning but also the number of floors within the building can be determined.

3 is a diagram for explaining a multi-signal in more detail according to an embodiment of the present invention.

As shown in FIG. 3, the multi-signal collected by the portable terminal 200 may include various information for each type. In addition to wireless signal sensor information of the portable terminal 200, a geomagnetic sensor, an acceleration sensor, A barometer, device information of the portable terminal 200, model information, MAC address information, and the like may also be included in the multi-signal.

4 is a diagram for explaining a neural network construction process according to an embodiment of the present invention.

Referring to FIG. 4 , the multi-signal collection unit 110 normalizes and pre-processes the collected multi-signals, and the imaging processing unit 120 and the similarity-based neural network learning unit 130 perform imaging and positioning neural network learning. , a big data learning model for actual positioning can be built.

4 shows a process of converting the multi-signal into a constellation image, the multi-signal SIGNAL can be converted into a pair of KEY and VALUE values. Then, the multiple signals are normalized by a normalization process, the KEY information is converted into X, Y coordinates and colors of constellation image object pixels by a hashing process, and the VALUE value is determined as intensity (INTENSITY) and converted into object density. can

Accordingly, as shown in FIG. 4 , a constellation image composed of constellation image objects representing indoor specific location information may be configured. Each type of sensor is represented by the color of a dot, which is an image object, the number of sensors is represented by the size of the dot, and the strength of the sensor is represented by the color density of the dot.

According to these constellation images, a composite signal receivable indoors can be configured as a sensor map image, and a location neural network model that predicts indoor location information through a deep learning process can be built.

5 to 10 are diagrams for explaining characteristics and positioning processes of a neural network constructed according to an embodiment of the present invention.

5 shows the result of constructing a Positioning Neural Network according to an embodiment of the present invention. As constellation image data is learned, the similarity between signals of two points is actually similar using comparison of similarity between constellation image data. It can be converted as a Euclidian distance.

Accordingly, when a representative constellation image for each location is configured, a location neural network usable as a positioning model can be constructed. This learning process is a configuration that cannot be implemented with a learning method such as an existing CNN.

6 shows the actual Euclidean distance calculation process for building a location neural network. The location neural network can be built by learning the similarity between two constellation images, and the similarity between the two constellations is converted into distance information to learn location information. indicates that it can be used for

FIG. 7 illustrates a result graph obtained by processing an active function dedicated to indoor constellation positioning based on a Euclidean distance based on a constellation image (constellation signal map image) according to an embodiment of the present invention.

As shown in FIG. 7 , the similarity-based neural network learning unit 130 according to an embodiment of the present invention learns positioning prediction information in a multi-dimensional manner using a multi-dimensional activation function for comparing the similarity between the constellation image objects. A positioning neural network (PNN) model based on deep learning of a constellation image object can be built. The positioning neural network model according to an embodiment of the present invention may be a PERCEPTRON neural network specialized for comparing constellation images, It can be configured so that there is a perceptron corresponding to every star (image object pixel) in each image.

In addition, the constellation image may be an image that connects signal pixels mapped to the same location information, and the weight between each compared constellation image is set to be the same, so that a speeded-up learning process is achieved by optimizing the perceptron learning and comparison process based on this. can be performed

In addition, performance can be optimized as an activation function dedicated to positioning is used as a 3-dimensional or multi-dimensional function corresponding to two inputs.

8 relates to a multi-dimensional activation function, and the location neural network according to an embodiment of the present invention uses information that cannot be located with an existing standardized activation function to obtain an accurate location using the output of applying a different multi-dimensional function for each function. make it possible to predict

For example, the variables that determine the shape of the function can be designed to fit indoor and outdoor positioning applications, such as the maximum value of positive output, the minimum value of negative output, the zero point on the x=y line of positive output, and the negative extraction power. A specially designed variable-based activation function, such as a point that becomes 0 on the line of x = 0 and y = 0 of , may be used.

9 is a diagram for explaining a method for determining the shape of an activation function that minimizes a loss function of a positional neural network according to an embodiment of the present invention. The output value is similarity and may be a function set to domain 0 to 1 and conjugate domain -1 to 1. there is.

Here, the loss function is determined as a distance between pairs that are evaluated as closest. For example, when each pair is generated for 3,000 sensor map constellation images, up to 9 million can be calculated. If the number of layers of the predicted result is incorrect, a larger value may be added to the error.

And, the learning process may include a process of determining a form of an activation function that minimizes the loss function. For example, in a 4-dimensional space, a neural network construction process that calculates and minimizes loss function values corresponding to the origin and the surrounding area may be performed as a learning process.

10 is a diagram for explaining an optimized learning algorithm reflecting the characteristics of a discrete function according to an embodiment of the present invention.

As shown in FIG. 10, the learning process of the positional neural network according to an embodiment of the present invention applies a gradient decent algorithm to a discontinuous function, and optimizes a total of 4 dimensions based on simulated annealing (SA). can be done

11 is a diagram for explaining a location determination service process according to an embodiment of the present invention.

As shown in FIG. 11, the positioning service process according to an embodiment of the present invention is implemented as each service process such as indoor/outdoor recognition, vertical layer, horizontal positioning (area), horizontal positioning (location tracking), and LAS location authentication. As it can be, it can be confirmed that the possibility of utilization in various services is high.

12 is a diagram for explaining implementation examples of various service systems according to an embodiment of the present invention.

As shown in FIG. 12, the service providing device 300 can build various service systems using the indoor coordinate positioning system 100, such as an indoor/outdoor positioning system, a grid address search system, an XR contents service system, and an indoor/outdoor positioning system. Various service systems such as a 3D modeling system, a CCTV image analysis system, a robot guidance system, a disaster safety system, and a construction management cloud system may be exemplified.

The method according to the present invention described above may be produced as a program to be executed on a computer and stored in a computer-readable recording medium. Examples of the computer-readable recording medium include ROM, RAM, CD-ROM, and magnetic tape. , floppy disks, and optical data storage devices.

The computer-readable recording medium is distributed to computer systems connected through a network, so that computer-readable codes can be stored and executed in a distributed manner. In addition, functional programs, codes, and code segments for implementing the method can be easily inferred by programmers in the technical field to which the present invention belongs.

In addition, although the preferred embodiments of the present invention have been shown and described above, the present invention is not limited to the specific embodiments described above, and the technical field to which the present invention belongs without departing from the gist of the present invention claimed in the claims. Of course, various modifications can be made by those skilled in the art, and these modifications should not be individually understood from the technical spirit or perspective of the present invention.

Claims (6)

In the indoor positioning system,
Using a smartphone, a radio wave map is created by collecting multiple signals from one's location, and the map is mapped to indoor location information (including at least one building address, building name, floor and number within the building), Build an Indian name server to share globally,
An indoor positioning system in which a user measures indoor location information by searching for indoor location information corresponding to the indoor name server using a propagation map of multiple signals collected using a smartphone at a user's location. According to claim 1,
The propagation map is
Characterized in that it is generated as a propagation map fingerprint including constellation image objects having different color sizes and concentrations according to the type, number and intensity of the multiple signals
indoor positioning system. According to claim 1
When an indoor location search request including a user-created radio map is received, indoor location information is detected by executing an inverse index-based radio map search to quickly select a radio map that is likely to be similar among stored radio maps.
indoor positioning system. According to claim 3
For the inverse index, hash map technology that is generated for each radio source included in the radio map fingerprint and stored separately, and stores all radio map fingerprints so that a large amount of radio map fingerprints can be searched at high speed. containing
indoor positioning system. In the operating method of the indoor positioning system,
Collecting multiple signals at one's location using a smartphone;
image processing into a propagation map fingerprint including constellation image objects having different color sizes and concentrations according to the types, numbers, and intensities of the collected multiplex signals;
mapping indoor location information corresponding to the imaged radio map fingerprint, storing it in a database, and providing a sharing service through a network;
Collecting multiple signals of a user's smartphone for indoor location measurement and transmitting them to the database through a network to search for a propagation map fingerprint of the collected multiple signals;
obtaining location information of a user's smartphone by extracting indoor location information mapped to the searched radio map fingerprint; containing
Operation method of indoor positioning system. According to claim 5,
constructing an artificial intelligence neural network-based indoor coordinate positioning model to output positioning information based on a similarity between the imaged data and the imaged data; and
Providing indoor positioning information using the artificial intelligence neural network-based indoor coordinate positioning model
Operation method of indoor positioning system.