CN114758364B

CN114758364B - Industrial Internet of things scene fusion positioning method and system based on deep learning

Info

Publication number: CN114758364B
Application number: CN202210120544.6A
Authority: CN
Inventors: 王天择; 孙奕髦; 董兵; 赖伟; 黄文炜
Original assignee: Sichuan University
Current assignee: Sichuan University
Priority date: 2022-02-09
Filing date: 2022-02-09
Publication date: 2022-09-23
Anticipated expiration: 2042-02-09
Also published as: CN114758364A

Abstract

The invention discloses an industrial Internet of things scene fusion positioning method and system based on deep learning, when position information of a position to be detected needs to be obtained, a position fingerprint identification network extracts features of fingerprint information received by a target to be detected through a convolutional neural network, then a feature image of the fingerprint information features is extracted through an image extraction layer, a prediction layer predicts fingerprint features of the fingerprint information based on the feature image, a full connection layer matches the fingerprint features with standard fingerprint information in a preset position fingerprint library, and position information corresponding to the successfully matched standard fingerprint information is output and serves as the position information of the target to be detected. The method and the device realize online positioning, eliminate the dependency of matching between position labels and improve the positioning robustness and precision. The position fingerprint identification network has good mobility, can be suitable for different environments, and improves the environmental adaptability to positioning.

Description

Industrial Internet of things scene fusion positioning method and system based on deep learning

Technical Field

The invention relates to the technical field of computers, in particular to an industrial internet of things scene fusion positioning method and system based on deep learning.

Background

The vigorous development of industrial manufacturing continuously promotes the technical change in the manufacturing field of China, and the appearance of the Internet of things and artificial intelligence technology makes concepts such as intelligent production and intelligent factories possible. In the intelligent modification, the traditional industrial production faces many difficulties, such as indoor positioning technology in an industrial scene. The indoor positioning technology is realized, personnel can be tracked and allocated in real time, each link of automatic production can be monitored, and the method has positive significance for improving the production efficiency.

Indoor positioning plays an important role in industrial scenarios. By the indoor positioning technology with higher precision, a manager can realize real-time monitoring and dynamic scheduling on production personnel, thereby eliminating hidden dangers and improving the working efficiency; meanwhile, a factory can acquire real-time positions of materials and vehicles, and automatic allocation, turnover, cargo transportation and the like of the materials are realized; in addition, a high-precision positioning technology is also a precondition for putting industrial robots and the like into automatic production.

Existing outdoor location services are mainly implemented by Global Positioning Satellite (GPS) technology. The global positioning system can provide high-precision positioning service for outdoor users, but has the following limitations: the GPS signal has very low common rate and higher signal receiving requirement, and no obstruction exists between an outdoor antenna and a satellite, so that a better positioning result can be achieved. However, when indoor positioning is involved, since satellite signals are blocked by buildings and then quickly attenuated after reaching indoors, indoor coverage requirements cannot be met, and indoor positioning by using GPS signals is almost impossible.

In industrial production, an indoor environment is far more complex than an outdoor environment, radio waves are easily blocked by obstacles, and reflection, refraction or scattering occurs to form non-line-of-sight propagation (namely NLOS, non-line-of-sight communication refers to indirect point-to-point communication between a receiver and a transmitter, the most direct explanation of the non-line-of-sight is that two points of sight of communication are blocked, the two points cannot see each other, and more than 50% of a Fresnel zone is blocked), so that the positioning accuracy is seriously influenced. In addition, the indoor production environment layout and topology are susceptible to human factors, which causes various signal propagation changes, thereby reducing the performance of the positioning technology based on the characteristic matching principle.

Most of the currently popular positioning systems are independent positioning systems, that is, most of them are designed for a special application environment, which leads to the following problems:

(1) the target detection has defects: most location systems rely on matches between tags, so they have difficulty detecting objects that are different tags co-existing with the target tag in the consent environment.

(2) Positioning robustness and precision are low: independent systems are limited in information acquisition and difficult to implement.

(3) The environmental adaptability is poor: an independent positioning system that performs well in one practical scenario may perform poorly in another practical scenario.

Disclosure of Invention

The invention aims to provide an industrial Internet of things scene fusion positioning method and system based on deep learning, which are used for solving the problems in the prior art.

In a first aspect, an embodiment of the present invention provides an industrial internet of things scene fusion positioning method based on deep learning, where the method includes:

acquiring fingerprint information received by a target to be detected, wherein the fingerprint information is used for representing the position characteristics of the target to be detected;

inputting the fingerprint information into a position fingerprint identification network trained in advance, and outputting the position information of the target to be detected by the position fingerprint identification network;

the position fingerprint identification network is composed of a convolutional neural network, a portrait extraction layer, a prediction layer, a full connection layer and a confrontation network, wherein the input of the convolutional neural network is the fingerprint information, and the input of the portrait extraction layer is the output of the convolutional neural network; the input of the prediction layer is the output of the sketch extraction layer, and the input of the countermeasure network is the output of the convolutional neural network and the output of the prediction layer; the prediction layer outputs the position information of the target to be detected, and the countermeasure network is used for eliminating the influence of environmental characteristics on the fingerprint characteristics output by the prediction layer; the input of the full connection layer is the output of the prediction layer, and the output of the prediction layer is the position information of the object to be measured.

Optionally, the training method of the location fingerprint identification network includes:

determining a plurality of fingerprint data of a plurality of predetermined positions, setting 20% of the plurality of fingerprint data of each predetermined position in a verification set, and setting 80% of the plurality of fingerprint data of each predetermined position in a training set; the fingerprint data in the verification set is marked with a preset position represented by the fingerprint data in advance;

inputting the training set and the verification set into a convolutional neural network, and respectively extracting training features in the training set and verification features in the verification set through the convolutional neural network;

extracting a first characteristic portrait of the training characteristics through a portrait extraction layer, and extracting a second characteristic portrait of the verification characteristics through the portrait extraction layer;

mapping the first characteristic portrait to a potential space through a prediction layer to obtain a first potential characteristic, and mapping the second characteristic portrait to the potential space to obtain a second potential characteristic; obtaining a first prediction vector y1 based on the first potential feature; obtaining a second prediction vector based on the second potential feature; obtaining a first loss function of the unlabeled data based on the first prediction vector, and obtaining a second loss function of the labeled data based on the second prediction vector;

performing fusion operation on the training features, the first prediction vector, the verification features and the second prediction vector through an antagonistic network to obtain fusion data; mapping the fusion data to a second space respectively to obtain mapping characteristics; obtaining a third loss function based on the mapping characteristics;

obtaining a model loss function based on the first loss function, the second loss function and the third loss function;

and when the model loss function is converged or the model training times are greater than a set value, determining that the training of the position fingerprint identification network is finished, and determining that a first prediction vector output by the prediction layer is the position information of the target to be detected.

Optionally, the performing a fusion operation on the training features, the first prediction vector, the verification features, and the second prediction vector to obtain fusion data includes:

obtaining a first fusion vector based on the training features, the first prediction vector and a second prediction vector;

obtaining a second fusion vector based on the verification feature, the first prediction vector and a second prediction vector;

and carrying out weighted summation on the first fusion vector and the second fusion vector to obtain fusion data.

In a second aspect, an embodiment of the present invention provides an industrial internet of things scene fusion positioning system based on deep learning, where the system includes:

the acquisition module is used for acquiring fingerprint information received by a target to be detected, and the fingerprint information is used for representing the position characteristics of the target to be detected;

the positioning module is used for inputting the fingerprint information into a position fingerprint identification network trained in advance, and the position fingerprint identification network outputs the position information of the target to be detected;

the position fingerprint identification network consists of a convolutional neural network, a portrait extraction layer, a prediction layer, a full connection layer and a confrontation network, wherein the input of the convolutional neural network is the fingerprint information, and the input of the portrait extraction layer is the output of the convolutional neural network; the input of the prediction layer is the output of the sketch extraction layer, and the input of the countermeasure network is the output of the convolutional neural network and the output of the prediction layer; the prediction layer outputs the position information of the target to be detected, and the confrontation network is used for eliminating the influence of environmental characteristics on the fingerprint characteristics output by the prediction layer; the input of the full connection layer is the output of the prediction layer, and the output of the prediction layer is the position information of the object to be measured.

Compared with the prior art, the embodiment of the invention achieves the following beneficial effects:

the embodiment of the invention provides an industrial Internet of things scene fusion positioning method and system based on deep learning, wherein the method comprises the following steps: acquiring fingerprint information received by a target to be detected, wherein the fingerprint information is used for representing the position characteristics of the target to be detected; inputting the fingerprint information into a position fingerprint identification network trained in advance, and outputting the position information of the target to be detected by the position fingerprint identification network; the position fingerprint identification network is composed of a convolutional neural network, a portrait extraction layer, a prediction layer, a full connection layer and a confrontation network, wherein the input of the convolutional neural network is the fingerprint information, and the input of the portrait extraction layer is the output of the convolutional neural network; the input of the prediction layer is the output of the sketch extraction layer, and the input of the countermeasure network is the output of the convolutional neural network and the output of the prediction layer; the prediction layer outputs the position information of the target to be detected, and the countermeasure network is used for eliminating the influence of environmental characteristics on the fingerprint characteristics output by the prediction layer; the input of the full connection layer is the output of the prediction layer, and the output of the prediction layer is the position information of the object to be measured.

When the position information of the position to be detected needs to be obtained, a signal (for example, Wifi) needs to be sent to the signal receiving terminal of the position to be detected through a predetermined AP, then signal information (fingerprint information) received by the signal receiving terminal is obtained, the fingerprint information is input into a position fingerprint identification network trained in advance, and the position fingerprint identification network can identify the position information of the position to be detected based on the fingerprint information. The position fingerprint identification network extracts the characteristics of fingerprint information received by a target to be detected through a convolutional neural network, then extracts a characteristic image of the fingerprint information characteristics through an image extraction layer, after the characteristic image is obtained, a prediction layer predicts the fingerprint characteristics of the fingerprint information based on the characteristic image, a full connection layer matches the fingerprint characteristics with standard fingerprint information in a preset position fingerprint database, and position information corresponding to the standard fingerprint information which is successfully matched is output and serves as the position information of the target to be detected. The standard fingerprint information uniquely characterizes one location information. Therefore, online positioning is realized, and the positioning accuracy is improved. In addition, the position fingerprint identification network comprises a countermeasure network, and the countermeasure network is used for eliminating the influence of the environmental characteristics on the fingerprint characteristics output by the prediction layer, so that the accuracy of the fingerprint characteristics output by the prediction layer is improved, and the accuracy of positioning is further improved.

Drawings

Fig. 1 is a flowchart of an industrial internet of things scene fusion positioning method based on deep learning according to an embodiment of the present invention.

Fig. 2 shows a schematic structure of a location fingerprinting network.

FIG. 3 shows a schematic representation of location fingerprint construction for signal strength of one of a variety of signals.

Fig. 4 is a schematic block structure diagram of an electronic device according to an embodiment of the present invention.

The labels in the figure are: a bus 500; a receiver 501; a processor 502; a transmitter 503; a memory 504; a bus interface 505.

Detailed Description

The present invention will be described in detail below with reference to the accompanying drawings.

Example 1

Existing outdoor location services are mainly implemented by Global Positioning Satellite (GPS) technology. The global positioning system can provide high-precision positioning service for outdoor users, but has the following limitations: the GPS signal common rate is very low, the signal receiving requirement is higher, no obstruction exists between an outdoor antenna and a satellite, and a better positioning result can be achieved. However, when indoor positioning is involved, since satellite signals are blocked by buildings and then quickly attenuated after reaching indoors, indoor coverage requirements cannot be met, and indoor positioning by using GPS signals is almost impossible.

In industrial production, an indoor environment is far more complex than an outdoor environment, radio waves are easily blocked by obstacles, and are reflected, refracted or scattered to form non-line-of-sight propagation (namely NLOS, the non-line-of-sight communication refers to indirect point-to-point communication between a receiver and a transmitter, the most direct explanation of the non-line-of-sight is that two points of sight of communication are blocked, the two points cannot see each other, and more than 50% of a Fresnel zone is blocked), so that the positioning accuracy is seriously influenced. In addition, the indoor production environment layout and topology are susceptible to human factors, which causes various signal propagation changes, thereby reducing the performance of the positioning technology based on the feature matching principle.

In consideration of the huge development potential and the wide development prospect of indoor positioning and the difficult points of the existing indoor positioning technology analyzed in the above, the application provides a positioning method based on the combination of multi-sensor data fusion and the deep learning technology so as to reduce the indoor positioning difficulty and improve the indoor positioning precision.

Before the solution proposed by the present invention is explained, the concept of "location fingerprint" needs to be introduced. As a "human identification card", human fingerprints are widely used in the field of identification. Uniqueness, local correlation and identifiability are the characteristics of fingerprints, and thus the concept of fingerprint location is introduced into indoor location. A "location fingerprint" is a fingerprint that associates a feature of a location in an actual environment with one or more features of the environment in which the location is located, and a one-to-one correspondence is achieved by specifying the features, i.e., a location corresponds to a unique fingerprint. This fingerprint may be one or more dimensions depending on the characteristics of the environment in which it is located, for example when the device to be located is receiving or transmitting information, then the fingerprint may be a characteristic or characteristics of the information or signal (most commonly signal strength). The common location fingerprint positioning modes include three, if the equipment to be positioned is sending signals, the fixed receiving equipment which is pre-installed senses the signals or information of the equipment to be positioned to realize positioning, and the mode is remote positioning or network positioning. If the device to be positioned receives signals or information of some fixed sending devices, then the position of the device to be positioned is estimated according to the detected characteristics, and the mode is self-positioning. If the device to be positioned transmits all the detected features to the server, the server uses the obtained features to estimate the position of the device, which is a hybrid positioning.

The method aims at the problems that the traditional method for indoor positioning through a single signal is high in information acquisition difficulty and low in positioning result accuracy. The positioning problem of an actual industrial scene can be effectively solved by adopting the fusion of various signals; various sensors are used for collecting various common signals indoors, such as Wireless Fidelity (WiFi), Visible Light Communication (VLC), and the like, and complete data can be collected under the condition of low difficulty. Compared with a single signal method, when one signal loses value due to some reason, more accurate positioning can be realized by making up information through other signals. When the signal is complete, more accurate positioning result can be achieved due to the multi-dimensionality of the signal. For this reason, the fingerprint information and fingerprint data mentioned in the present application are the above-mentioned "location fingerprint", which may be a single signal, such as a WiFi signal, a visible light signal, and an Ultra Wide Band (UWB) signal, or may be formed by combining a plurality of signals mentioned above, and it should be noted that the nature of VLC, UWB, and WiFi positioning is actually a base station type positioning, and positioning is performed by radiating a fingerprint that gradually weakens outward by taking a WiFi AP and a UWB base station as their centers and a light source. Therefore, the fingerprint Information and fingerprint data of the present application may be Channel State Information (CSI), Received Signal Strength Indication (RSSI), or data formed by combining Information such as CSI and RSSI. That is, the location fingerprint can be of various types, and any "location-unique" characteristic can be used as a location fingerprint, such as the multipath structure of the communication signal at a location, whether an access point or base station is detected at a location, the Received Signal Strength (RSS) of the communication signal from a base station detected at a location, the round trip time or delay of the communication signal at a location, as a location fingerprint, or a combination thereof to form a location fingerprintThe signal intensity or the visible light signal intensity is weakened along with the increase of the propagation distance in the space propagation process, and the closer the receiving end equipment is to the signal source, the stronger the signal intensity of the signal source is, the farther the receiving end is from the sender, and the weaker the received signal intensity is. A unique fingerprint may be formed based on the strength of the signal received by the terminal device. Considering cost and practicality aspects, currently, the Received radio Signal Strength (Received Signal Strength,

) Are commonly used as fingerprints in indoor positioning. In the indoor environment where the wireless signal deployment is finished, the wireless signals are distinguished at different positions and are distributed relatively stably in space and time, and the wireless signals are distributed only at a preset reference point

The fingerprint collection can be carried out, and the position of an Access Point (AP) does not need to be known. The positioning method and the positioning system aim at the problems of positioning, positioning accuracy and the like of indoor scenes such as large workshops, plants, warehouses and the like, and serve as an optional implementation mode, so that the CSI is adopted to be suitable for adopting (VLC, UWB and WiFi are fused to receive signal strength) as position fingerprints.

With reference to fig. 1, the method for scene fusion and positioning of the industrial internet of things based on deep learning includes:

s101: and acquiring fingerprint information received by the target to be detected.

In the embodiment of the present invention, the fingerprint information is used to characterize the location characteristics of the target to be measured, and optionally, the fingerprint information may be information such as CSI, RSSI, RSS, or information formed by combining CSI, RSSI, and RSS. In the embodiment of the present invention, the receiver of the target to be detected receives the fingerprint signal transmitted by the transmitter in the scene and performs fusion to obtain the fingerprint information, which may specifically be:

and obtaining data samples in a set time period, wherein the set time period can be 1-15 minutes, the data samples in the set time period comprise data sampled every second in the set time period, namely the data samples comprise data sampled for multiple times. The sampled data may include one or more of CSI, RSSI, RSS, etc. information. As an example, one piece of fingerprint information is represented by [ a1, a2, a3, … …, a60 ]. Wherein, a1, a2, a3, … … and a60 respectively represent CSI information acquired at 1 st second, 2 nd second, 3 rd second and … … th 60 th second.

S102: and inputting the fingerprint information into a position fingerprint identification network trained in advance, and outputting the position information of the target to be detected by the position fingerprint identification network.

The position fingerprint identification network is composed of a convolutional neural network, a portrait extraction layer, a prediction layer, a full connection layer and a confrontation network, wherein the input of the convolutional neural network is the fingerprint information, and the input of the portrait extraction layer is the output of the convolutional neural network. The input to the prediction layer is the output of the image layer, and the input to the countermeasure network is the output of the convolutional neural network and the output of the prediction layer. The prediction layer outputs the position information of the target to be detected, and the countermeasure network is used for eliminating the influence of environmental characteristics on the fingerprint characteristics output by the prediction layer. The input of the full connection layer is the output of the prediction layer, and the output of the prediction layer is the position information of the object to be measured. As shown in fig. 2.

By adopting the scheme, when the position information of the position to be detected needs to be obtained, a signal (such as Wifi) needs to be sent to the signal receiving end of the position to be detected through the preset AP, then the signal information (fingerprint information) received by the signal receiving end is obtained, the fingerprint information is input into the position fingerprint identification network trained in advance, and the position fingerprint identification network can identify the position information of the position to be detected based on the fingerprint information. The position fingerprint identification network extracts the characteristics of fingerprint information received by a target to be detected through a convolutional neural network, then extracts a characteristic image of the fingerprint information characteristics through an image extraction layer, after the characteristic image is obtained, a prediction layer predicts the fingerprint characteristics of the fingerprint information based on the characteristic image, a full connection layer matches the fingerprint characteristics with standard fingerprint information in a preset position fingerprint database, and position information corresponding to the standard fingerprint information which is successfully matched is output and serves as the position information of the target to be detected. The standard fingerprint information uniquely characterizes one location information. Therefore, online positioning is realized, and the positioning accuracy is improved. In addition, the position fingerprint identification network comprises a countermeasure network, and the countermeasure network is used for eliminating the influence of the environmental characteristics on the fingerprint characteristics output by the prediction layer, so that the accuracy of the fingerprint characteristics output by the prediction layer is improved, and the accuracy of positioning is further improved.

The training method of the position fingerprint identification network comprises the following steps:

a1, determining a plurality of fingerprint data of a plurality of preset positions, setting 20% of the plurality of fingerprint data of each preset position in a verification set, and setting 80% of the plurality of fingerprint data of each preset position in a training set. The fingerprint data in the verification set is pre-labeled with a predetermined location characterized by the fingerprint data.

In the embodiment of the present invention, the predetermined location is a specific area within the positioning scenario, for example, if the positioning scenario is a factory including 10 workshops, each workshop can be set to be a predetermined location. Optionally, a signal receiver is arranged in the predetermined position, and a plurality of fingerprint data of each predetermined position are obtained according to the above manner of obtaining the fingerprint information of the target to be detected. For example, the fingerprint information of the object to be detected is [ a1, a2, a3, … …, a60], and the fingerprint data of one of the predetermined positions may be [ b1, b2, b3, … …, b60], [ b61, b62, b63, … …, b120], [ b121, b122, b123, … …, b180 ]. I.e. the fingerprint data may comprise a plurality of pieces of historical fingerprint information. If the fingerprint data has 10 pieces of historical fingerprint information, 8 pieces of fingerprint information are arranged in the training set, 2 pieces of fingerprint information are arranged in the verification set, and the historical fingerprint information arranged in the verification set is subjected to standard to determine the position information represented by the historical fingerprint information in the verification set.

And A2, inputting the training set and the verification set into a convolutional neural network, and respectively extracting the training features in the training set and the verification features in the verification set through the convolutional neural network.

A3, extracting a first feature image of the training feature by the image extraction layer, and extracting a second feature image of the verification feature by the image extraction layer. Specifically, the portrait extraction layer is a full connection layer and is fully connected with the convolutional neural network, the portrait extraction layer maps the training features and the verification features into a portrait space to respectively obtain a first feature portrait and a second feature portrait, specifically, the training features are mapped through a softplus function to obtain the first feature portrait, and the verification features are mapped through the softplus function to obtain the second feature portrait.

A4, mapping the first feature image into the potential space through the prediction layer to obtain a first potential feature, and mapping the second feature image into the potential space to obtain a second potential feature. Obtaining a first prediction vector based on the first potential feature; a second prediction vector is derived based on the second potential feature.

In particular, a first feature image is mapped into the potential space, resulting in a first potential feature, specifically H1= W × P1+ b, H1 representing the first potential feature, and P1 representing the first feature image. And projecting the second feature image into the potential space to obtain a second potential feature, specifically H2= W × P2+ b, P2 represents the second feature image, and H2 represents the second potential feature. W, b are mapping parameters, and the values may be W =1, b = 0.5.

Obtaining a first prediction vector based on the first potential feature through a softmax activation function; and obtaining a second prediction vector based on the second potential feature through the softmax activation function. Specifically, the first potential feature is input into a softmax activation function, and the softmax activation function outputs the first prediction vector. The second potential feature is input into a softmax activation function, which outputs a second prediction vector.

A5, obtaining a first loss function of the unlabeled data based on the first prediction vector, and obtaining a second loss function of the labeled data based on the second prediction vector.

In the embodiment of the present invention, the steps a4, a5 are obtained by a prediction layer operation.

Specifically, the first loss function is represented by:

（1）

where L1 is the first loss function, X1 represents the number of fingerprint data in the training set.

Representing the ith first prediction vector.

The second loss function is represented by:

（2）

where, the L2 second loss function,

the actual feature vector representing the data for the ith fingerprint in the verification set is pre-extracted. X2 represents the amount of fingerprint data in the authentication set.

And A6, carrying out fusion operation on the training features, the first prediction vectors, the verification features and the second prediction vectors to obtain fusion data. Specifically, the fusion operation specifically comprises: first, a first fusion vector is obtained based on the training features, the first prediction vector and the second prediction vector. And secondly, obtaining a second fusion vector based on the verification feature, the first prediction vector and the second prediction vector. And then carrying out weighted summation on the first fusion vector and the second fusion vector to obtain fusion data.

Obtaining a first fusion vector based on the training features, the first prediction vector and the second prediction vector, specifically obtaining the first fusion vector by a calculation mode of the following formula: r1= y1+ a y2-p z1, wherein R1 represents the first fused vector, y1 represents the first prediction vector, y2 represents the second prediction vector, z1 represents the training feature, a, p are weight parameters, a + p = 1.

And obtaining a second fusion vector based on the verification feature, the first prediction vector and the second prediction vector, specifically by a calculation mode of the following formula: r2= y2+ a y1-p z2, wherein R2 represents the second fusion vector and z2 represents the verification feature.

The fused data F is represented in the following manner: f = R2+ a × R2-p × R1, wherein F represents fused data.

For this reason, for each fingerprint data, the manner of obtaining the first fusion vector, and the fusion data of the fingerprint data is referred to as the manner of obtaining the first fusion vector, and the fusion data as above, and details are not repeated here.

By adopting the scheme, the data characteristics of the verification data set and the training data are comprehensively considered in the fusion data, and the accuracy of the characterization of the fusion characteristics on the position is improved.

And A7, mapping the fused data to a second space respectively to obtain mapping characteristics.

Specifically, the specific mapping mode for mapping the fusion data to the second space through the softmax function to obtain the mapping characteristics is as follows:

（3）

wherein the content of the first and second substances,

a mapping feature representing the ith fingerprint data,

is a mapping parameter of the second space, and the value can be

。

A8, obtaining a third loss function based on the mapping characteristics, specifically according to the mode shown in formula (4):

（4）

where L3 denotes a third loss function,

a unique heat vector expressing the ith fingerprint data.

In the embodiment of the present invention, the steps A6-A8 are obtained by the operation of the countermeasure network.

A9, obtaining a model loss function based on the first loss function, the second loss function and the third loss function.

The model loss function is specifically calculated in the form of L = L2+ a L1-p L3 (5)

Where L represents the loss function of the model.

And when the model loss function is converged or the model training times are greater than a set value, determining that the training of the position fingerprint identification network is finished, and determining that the first prediction vector output by the prediction layer is the position information of the target to be measured.

By adopting the scheme, the constraint capacity of the loss function on the model can be improved, the correlation between the marked data and the unmarked data is considered, the influence of the network eliminating environmental characteristics on the fingerprint characteristics output by the prediction layer is resisted (particularly, the formula (5) is embodied), and the accuracy of the model predicting position information is improved. The trained model is used for detecting the target position to be detected, the matching dependency among the labels is eliminated, the fingerprint data of the position to be detected is directly obtained for the position to be detected, the fingerprint data is input into the model, the position information of the position can be directly detected, the positioning independent of the labels is realized, and the positioning robustness and the positioning precision are improved. Due to the adoption of the position fingerprint identification network with the structure, the network has good mobility, can be suitable for different environments, and improves the environmental adaptability of positioning.

In the embodiment of the present invention, the manner of obtaining the fingerprint information received by the target to be detected includes:

the method comprises the steps of collecting fingerprint data received by a target to be detected, and specifically collecting CSI (channel state information) or RSS (received signal strength) information of the target to be detected at the position of the target to be detected.

Of signals

Or the received power depends on the position of the receiver.

The signal decreases with increasing distance, and is generally negative, and the signal is considered to be good at-60 to-70 dBm.

The acquisition is relatively simple because it is necessary for most wireless communication devices to operate properly. Many communication system needs

The information is used to sense the quality of the link to perform functions such as handoff and adapting the transmission rate, and

is not affected by the signal bandwidth. The RSS information is adopted as important reference data of indoor positioning, so that the positioning accuracy can be improved.

In free space, without any obstacles, the signal is emitted from the emission source in a spherical shape in all directions, without any distinction in each direction, so that the power of the signal is inversely proportional to the square of the distance: p ℃ 1d 2.

That is, power, but the unit of attenuation is typically expressed in dB, it is easy to understand

In relation to the distance, the distance between the sensor and the sensor,

attenuation is proportional to the logarithm of the distance, assuming a reference distance d0 and a distance over it are known

Is composed of

(d0) Then, RSS (d) = RSS (d0) -10 × n × log (d 0).

In the embodiment of the application, the preset fixed signal emission source is averaged at different distances from the signal emission source

Is proportional to the logarithm of the distance, and in the simplest case, the RSS can be expressed as:

RSS(d)=Pt-K-10*q*ln(d)

where q is called the path loss exponent, Pt is the transmit power, and K is a constant that depends on the environment and frequency, and may take values from 0.1 to 20. RSS can be used to calculate the distance between the device to be located and the AP and the light source, where the obtained distance is used to perform trilateration of the mobile device for location determination, and this method may cause large errors due to the influence of the actual environment (called shadow fading). When the device to be positioned receives signals from a plurality of transmission sources, we can use the signals from the plurality of transmission sources

Form a R

The vector serves as a fingerprint associated with the location. The method comprises the steps of measuring fingerprint data of the same position by different sensors through a data layer fusion technology, firstly carrying out fusion processing on the measured data from a plurality of different sensors, obtaining and extracting combined data characteristics to construct a position fingerprint database.

In the embodiment of the invention, the establishment of the corresponding relation between the position and the fingerprint is carried out in an off-line stage. In the off-line stage, in order to collect fingerprints at various positions and construct a database, comparison is performed in a specified areaAnd (4) carrying out complicated survey and collecting good data as a training set. The position coordinates obtained by indoor positioning calculation refer to coordinates in a local coordinate system in the current environment, and are not latitude and longitude. At each data acquisition point, we averaged the data from each AP light source by sampling the data over a period of time (5 to 15 minutes, acquired approximately once per second)

The device may be oriented differently at the time of acquisition.

For this project we deploy several APs, UWB base stations and LED light sources (LiFi's APs are usually integrated on the ceiling and UWB positioning base stations are distributed at the geometrical edges of the positioning area) which can be used for communication as well as for positioning.

In the data acquisition stage, the complicated experiment is carried out at each data acquisition point through equipment, and an equipment hardware system comprises a photodiode, a 0UWB (ultra wide band) tag, a WiFi (wireless fidelity) module and the like. Firstly, the equipment obtains various relevant characteristics of a data acquisition point through a relevant module, namely VLC signal intensity, UWB signal intensity and WiFi signal intensity, and then sends the characteristics to an upper computer through a WiFi module. At each data acquisition point, we averaged the data from each AP light source by sampling the data over a period of time (5 to 15 minutes, acquired approximately once per second)

The device may have a different orientation at the time of acquisition.

To clear the technical solution of the embodiments of the present invention, it is set that the geographic area to be measured is covered by a rectangular grid, as shown in fig. 3, in this scenario, there are 4 rows and 8 columns of grids (total 32 grid points), 1 AP and 1 LED light source, in this example, the fingerprint on each grid point is a two-dimensional vector z = [ x1, x2 ] = [ x1 ], and the fingerprint on each grid point is a two-dimensional vector z = [ x2 ]]Where x1 is from the average of the APs

，

Averaging from LEDs

. That is, the fingerprint data z at each predetermined position is represented in a manner of z = [ x1, x2 ]]。

These two-dimensional fingerprints are acquired in the area shown by each grid point, and the coordinates of the grid points and the corresponding fingerprints form a database, which is also referred to as a labeling phase (radio map). The right part of fig. 3 shows these fingerprints in a two-dimensional vector space (signal space), whereas in a more general scenario, assuming N APs, UWB base stations and LED light sources, the fingerprints are then

Is an N-dimensional vector.

The hybrid network location for which the present application is directed is less costly than those location systems that require additional equipment.

Most studies assume that the location fingerprints are obtained by collecting data at virtual grid points, for example, in a 100 x 100 area, dividing the area into 50 x 50 grids (i.e., 2m x 2m per grid), and collecting a set of fingerprints in the middle of each grid, where each set of fingerprints is recorded by the grid points that include information received from each WiFi AP, UWB base station, and visible light

5 to 15 minutes, in some cases we also use different measuring devices or set different device orientations to take measurements and collect data. Such collection work is extremely tedious, because this project is to the locatable and positioning accuracy scheduling problem of indoor scenes such as large-scale workshop, factory building, warehouse, etc. moreover need periodically to be updated in order to adapt to environmental change. In the above example, the application uses a single device and fixed orientation, and the whole collection process takes 2500 x 5=125000 minutes, approaching 9 days, not onlyThe labor and material consumption and the like are high, and the efficiency is low. Efficiency is improved by preprocessing by reducing the number of fingerprint acquisitions.

In a complex indoor environment, particularly, the project aims at indoor scenes such as large workshops, plants, warehouses and the like, WiFi, UWB and VLC

The signal is susceptible to multipath effects and ambient temperature, and reflections, refractions, or even equipment problems, of the signal within the room can result in the signal being collected

The value suffers from fluctuations, even packet loss. When we are doing data collection, if it is collected

Value, especially stronger

The value is lost, and even if the subsequent positioning algorithm is good, a large error can be caused, so that the project adopts an integrated filtering method combining gradient filtering and Kalman filtering. The method has the main idea that firstly, the gradient filtering method is utilized to carry out preliminary filtering on the acquired data, missing data is filled, and the occurrence of data acquisition process is avoided

The value is lost, and then the Kalman filtering method is utilized to carry out secondary filtering processing on the data, so that the data can be further smoothed

Data and noise reduction pair

The impact of the data.

That is, after acquiring the fingerprint data received by the target to be detected, the method for obtaining the fingerprint information received by the target to be detected further includes: and preprocessing the fingerprint data.

The specific pretreatment mode may be:

in the indoor space to be collected, the intelligent mobile equipment can collect a group at any collection point

And (4) data. Due to a single acquisition

The fluctuation is large, therefore, in the actual collecting process, a time interval is set for collecting

The data is set to 1 second here. During this time, the data collected is averaged and represented by RSS (i),

+1) represents the data subsequent to the data acquired consecutively, i being a positive integer greater than or equal to 1. The data collected each time are processed by an integrated filtering method, and the detailed processing steps are as follows:

step 1, firstly, the first step is to

The data is judged if

+1) =0, if data loss is determined, replacing with data acquired last time, and if n times of loss occur continuously, and the value of n may be 3-5 times, stopping replacing with the last time data, which indicates that the position of the reference point is far away from the coverage of the AP. That is, for the mth data acquisition, if RSS (m) =0, RSS (m) = RSS (m-1) is determined, and m is a positive integer greater than or equal to 2. If rss (m) =0 appears n times in succession, data processing is performed in the manner of step 2.

And 2, predicting and gradient filtering the data. Specifically, the method comprises the following steps:

obtained by the first two steps

The rss (prediction) mode of the prediction data of +1) is formula (6).

RSS(predict)=2*RSS(i)-RSS(i-1) (6)

I.e. the prediction difference is made for the missing data.

And then for the predicted prediction data

Performing gradient filtration, wherein the specific gradient filtration mode is as follows:

RSS(predict)1= RSS(predict)+sign(RSS(predict))+c (7)

where c is equal to the variance between RSS (predict) and RSS (i + 1).

Through adopting above scheme, can improve the accuracy of indoor location.

In summary, the industrial internet of things scene fusion positioning method based on deep learning provided by the embodiment of the invention includes:

firstly, fingerprint data received by a target to be detected is collected, wherein the fingerprint data is information capable of representing a certain position. The fingerprint data is preprocessed, environmental influences and data residual errors are removed, the fingerprint data is corrected, then the fingerprint data is input into a trained position fingerprint identification network, the characteristics of the corrected fingerprint data are extracted through a convolutional neural network in the position fingerprint identification network, then a characteristic portrait of the characteristics of the fingerprint data is extracted through a portrait extracting layer, then the fingerprint characteristics of the fingerprint data are predicted through a predicting layer based on the characteristic portrait, then the fingerprint characteristics are matched with standard fingerprints in a preset fingerprint library based on a full connecting layer, and position information corresponding to the standard fingerprints successfully matched with the fingerprint characteristics is output as position information of a target to be detected. Thereby realizing the positioning of the target to be measured. The target to be detected can be a target area to be detected, or a person, an animal, a robot or an object carrying equipment capable of receiving fingerprint information.

The fingerprint database stores standard fingerprints and position information with one-to-one relationship in advance, and one standard fingerprint corresponds to one position information. I.e. a standard fingerprint uniquely characterizes a location.

The fingerprint features are matched with the standard fingerprints in a preset fingerprint database, the Euclidean distance or the similarity between the fingerprint features and the standard fingerprints can be calculated, then the position information corresponding to the standard fingerprint with the minimum Euclidean distance is taken as the position information of the target to be detected, or the position information corresponding to the standard fingerprint with the maximum similarity is taken as the position information of the target to be detected. The similarity can be a cosine value of an included angle between the fingerprint feature and the standard fingerprint.

Example 2

The embodiment of the invention provides an industrial Internet of things scene fusion positioning system based on deep learning, which comprises an acquisition module and a positioning module, wherein:

the acquisition module is used for acquiring fingerprint information received by the target to be detected, and the fingerprint information is used for representing the position characteristics of the target to be detected.

And the positioning module is used for inputting the fingerprint information into a position fingerprint identification network trained in advance, and the position fingerprint identification network outputs the position information of the target to be detected.

In the embodiment of the invention, the position fingerprint identification network is composed of a convolutional neural network, a portrait extraction layer, a prediction layer, a full connection layer and a countermeasure network. The input of the convolutional neural network is the fingerprint information, and the input of the portrait extraction layer is the output of the convolutional neural network. The input of the prediction layer is the output of the portrait extraction layer, and the input of the countermeasure network is the output of the convolutional neural network and the output of the prediction layer. The prediction layer outputs the position information of the target to be detected, and the countermeasure network is used for eliminating the influence of environmental characteristics on the fingerprint characteristics output by the prediction layer. The input of the full connection layer is the output of the prediction layer, and the output of the prediction layer is the position information of the object to be measured. As shown in fig. 2.

In the embodiment of the invention, the training method of the position fingerprint identification network comprises the following steps:

A4, mapping the first feature image into the potential space through the prediction layer to obtain the first potential feature, and mapping the second feature image into the potential space to obtain the second potential feature. Obtaining a first prediction vector based on the first potential feature; a second prediction vector is derived based on the second latent features.

And A6, carrying out fusion operation on the training features, the first prediction vector, the verification features and the second prediction vector to obtain fusion data. Specifically, the fusion operation specifically comprises: first, a first fusion vector is obtained based on the training features, the first prediction vector and the second prediction vector. And secondly, obtaining a second fusion vector based on the verification feature, the first prediction vector and the second prediction vector. And then carrying out weighted summation on the first fusion vector and the second fusion vector to obtain fusion data.

Obtaining a second fusion vector based on the verification feature, the first prediction vector and the second prediction vector, specifically obtaining the second fusion vector by a calculation mode of the following formula: r2= y2+ a y1-p z2, wherein R2 represents the second fusion vector and z2 represents the verification feature.

The fused data F is represented in the following manner: f = R2+ a R2-p R1, wherein F represents fusion data.

in the embodiment of the invention, the steps A6-A8 are obtained by the operation of the countermeasure network.

And A9, obtaining a model loss function based on the first loss function, the second loss function and the third loss function.

And when the model loss function is converged or the model training times are greater than a set value, determining that the training of the position fingerprint identification network is finished, and determining that a first prediction vector output by the prediction layer is input of the position information of the target to be measured.

By adopting the scheme, the constraint capacity of the loss function on the model can be improved, the correlation between the marked data and the unmarked data is considered, the influence of the network-resistant elimination environment characteristics on the fingerprint characteristics output by the prediction layer is realized (particularly, the formula (5) is embodied), and the accuracy of the model prediction position information is improved. The trained model is used for detecting the target position to be detected, the matching dependency between the labels is eliminated, the fingerprint data of the position to be detected is directly obtained for the position to be detected, and the position information of the position can be directly detected by inputting the fingerprint data into the model, so that the positioning independent of the labels is realized, and the positioning robustness and the positioning precision are improved. Due to the adoption of the position fingerprint identification network with the structure, the network has good mobility, can be suitable for different environments, and improves the environmental adaptability of positioning. In addition, the industrial internet of things scene fusion positioning system based on deep learning further comprises a preprocessing module, and the preprocessing module is used for preprocessing fingerprint data.

The specific pretreatment mode may be:

And (4) data. Due to a single acquisition

The data is set to 1 second here. During which time the collected data is averaged

It is shown that,

the latter data representing the data acquired consecutively, i being a positive integer greater than or equal to 1.The data collected each time are processed by an integrated filtering method, and the detailed processing steps are as follows:

and (3) processing by a filtering method, wherein the detailed processing steps are as follows:

step 1, firstly, the

The data is judged if

+1) =0, if the data is determined to be lost, the data acquired last time is adopted for replacement, if the loss situation occurs for n times continuously, and the value of n can be 3-5 times, the replacement with the last time data is stopped, and it is indicated that the position of the reference point is far away from the coverage area of the AP. That is, for the mth data acquisition, if RSS (m) =0, RSS (m) = RSS (m-1) is determined, and m is a positive integer greater than or equal to 2. If rss (m) =0 appears n times in succession, data processing is performed in the manner of step 2.

through adopting above scheme, can improve the accuracy of indoor location.

With regard to the system in the above embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.

An embodiment of the present invention further provides an electronic device, as shown in fig. 4, which includes a memory 504, a processor 502, and a computer program stored on the memory 504 and executable on the processor 502, where the processor 502, when executing the program, implements the steps of any one of the deep learning based industrial internet of things scene fusion positioning methods described above.

Where in fig. 4 a bus architecture (represented by bus 500) is shown, bus 500 may include any number of interconnected buses and bridges, and bus 500 links together various circuits including one or more processors, represented by processor 502, and memory, represented by memory 504. The bus 500 may also link together various other circuits such as peripherals, voltage regulators, power management circuits, and the like, which are well known in the art, and therefore, will not be described any further herein. A bus interface 505 provides an interface between the bus 500 and the receiver 501 and transmitter 503. The receiver 501 and the transmitter 503 may be the same element, i.e. a transceiver, providing a means for communicating with various other apparatus over a transmission medium. The processor 502 is responsible for managing the bus 500 and general processing, and the memory 504 may be used for storing data used by the processor 502 in performing operations.

The algorithms and displays presented herein are not inherently related to any particular computer, virtual machine, or other apparatus. Various general purpose systems may also be used with the teachings herein. The required structure for constructing such a system will be apparent from the description above. Moreover, the present invention is not directed to any particular programming language. It is appreciated that a variety of programming languages may be used to implement the teachings of the present invention as described herein, and any descriptions of specific languages are provided above to disclose the best mode of the invention.

In the description provided herein, numerous specific details are set forth. However, it is understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.

Various component embodiments of the invention may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art will appreciate that a microprocessor or Digital Signal Processor (DSP) may be used in practice to implement some or all of the functionality of some or all of the components in an apparatus according to an embodiment of the invention. The present invention may also be embodied as apparatus or device programs (e.g., computer programs and computer program products) for performing a portion or all of the methods described herein. Such programs implementing the present invention may be stored on computer-readable media or may be in the form of one or more signals. Such a signal may be downloaded from an internet website, or provided on a carrier signal, or provided in any other form.

It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means can be embodied by one and the same item of hardware. The usage of the words first, second and third, etcetera do not indicate any ordering. These words may be interpreted as names.

Claims

1. An industrial Internet of things scene fusion positioning method based on deep learning is characterized by comprising the following steps:

the position fingerprint identification network is composed of a convolutional neural network, a portrait extraction layer, a prediction layer, a full connection layer and a confrontation network, wherein the input of the convolutional neural network is the fingerprint information, and the input of the portrait extraction layer is the output of the convolutional neural network; the input of the prediction layer is the output of the portrait extraction layer, and the input of the countermeasure network is the output of the convolutional neural network and the output of the prediction layer; the prediction layer outputs the position information of the target to be detected, and the countermeasure network is used for eliminating the influence of environmental characteristics on the fingerprint characteristics output by the prediction layer; the input of the full connection layer is the output of the prediction layer, and the output of the prediction layer is the position information of the object to be measured.

2. The deep learning-based industrial internet of things scene fusion positioning method as claimed in claim 1, wherein the training method of the location fingerprint identification network comprises the following steps:

mapping the first characteristic portrait to a potential space through a prediction layer to obtain a first potential characteristic, and mapping the second characteristic portrait to the potential space to obtain a second potential characteristic; obtaining a first prediction vector based on the first potential feature; obtaining a second prediction vector based on the second potential feature; obtaining a first loss function of unmarked data based on the first prediction vector, and obtaining a second loss function of marked data based on the second prediction vector;

3. The deep learning-based industrial internet of things scene fusion positioning method as claimed in claim 2, wherein the fusion operation of the training features, the first prediction vector, the verification features and the second prediction vector to obtain fusion data comprises:

4. An industrial internet of things scene fusion positioning system based on deep learning, the system comprising:

5. The deep learning-based industrial internet of things scene fusion positioning system as claimed in claim 4, wherein the training method of the location fingerprint identification network comprises the following steps:

mapping the first characteristic portrait to a potential space through a prediction layer to obtain a first potential characteristic, and mapping the second characteristic portrait to the potential space to obtain a second potential characteristic; obtaining a first prediction vector based on the first potential feature; obtaining a second prediction vector based on the second potential feature; obtaining a first loss function of the unlabeled data based on the first prediction vector, and obtaining a second loss function of the labeled data based on the second prediction vector;

performing fusion operation on the training features, the first prediction vector, the verification features and the second prediction vector through an antagonistic network to obtain fusion data; mapping the fusion data to second spaces respectively to obtain mapping characteristics; obtaining a third loss function based on the mapping characteristics;

6. The deep learning-based industrial internet of things scene fusion positioning system according to claim 5, wherein the fusion operation of the training features, the first prediction vector, the verification features and the second prediction vector to obtain fusion data comprises: