WO2022101997A1

WO2022101997A1 - State estimation system, state estimation method, state estimation device, and state estimation program

Info

Publication number: WO2022101997A1
Application number: PCT/JP2020/041956
Authority: WO
Inventors: 理一工藤; 馨子高橋; 友規村上; 智明小川
Original assignee: 日本電信電話株式会社
Priority date: 2020-11-10
Filing date: 2020-11-10
Publication date: 2022-05-19
Also published as: JP7473843B2; JPWO2022101997A1

Abstract

A state estimation system 1 that estimates the state of wireless communication terminals 2 or the peripheral environment of the wireless communication terminals comprises: wireless communication units 11 that wirelessly receive communication signals transmitted by the wireless communication terminals 2, and acquires channel information relating to electromagnetic wave propagation between said units and the wireless communication terminals from the communication signals; an input feature amount generation unit 12 that generates a channel feature amount using the channel information; and a real-world communication model control unit 14 that detects the state of the wireless communication terminals or the peripheral environment of the wireless communication terminals by inputting the channel feature amount into a real-world communication model in which machine learning is used to model the relationship between the channel feature amount relating to electromagnetic wave propagation between the wireless communication devices and the state of the wireless communication devices or the peripheral environment of the wireless communication devices.

Description

State estimation system, state estimation method, state estimation device, and state estimation program

The present invention relates to a state estimation system for estimating the state of a wireless communication device and the surrounding environment, a state estimation method, a state estimation device, and a state estimation program.

The realization of IOT (Internet of things), in which various devices are connected to the Internet, is progressing. Various devices such as automobiles, drones, and construction machinery vehicles are being connected wirelessly. As wireless communication standards, for wireless LAN (Local Area Network), Bluetooth (registered trademark), cellular communication by LTE and 5G, LPWA (Low Power Wide Area) communication for IOT, and car communication specified by the standardization standard IEEE802.11. Supported wireless communication standards such as ETC (Electronic Toll Collection System), VICS (Vehicle Information and Communication System), and ARIB-STD-T109 used are also developing, and are expected to spread in the future.

The above wireless communication equipment has introduced MIMO (Multiple input multiple output) communication technology using multiple antennas in order to achieve high throughput and reliability performance. MIMO communication technology can improve throughput and reliability performance by using channel information that shows how radio waves propagate between the transmitting side and the receiving side. The transmitting side wireless communication device supports a function of transmitting a feedback signal for transmitting channel information to the receiving side wireless communication device (see Non-Patent Document 1).

By using the channel information related to radio wave propagation, it was possible to improve the throughput and reliability performance related to the communication of wireless communication equipment. On the other hand, depending on the state of the wireless communication device such as position, posture, movement, type of wireless communication device such as laptop computer, smartphone, and static or dynamic objects existing around the wireless communication device, it is possible to communicate with the communication partner. Since the radio wave propagation environment changes, it may affect the communication quality and have a great influence on the services and wireless communication systems realized by the wireless communication by the wireless communication device. In particular, the higher the frequency used for wireless communication, the stronger the straightness of the radio wave, and the more easily it is affected by the communication quality.

Therefore, the present invention focuses on the fact that the channel information not only includes information related to the communication of the wireless communication device but also reflects the state, type, and surrounding environment of the wireless communication device at the time of wireless communication, and the channel information is provided. It is used to estimate the real-world state of the surrounding environment of wireless communication devices and wireless communication terminals.

The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a technique capable of estimating the state of the surrounding environment of a wireless communication device or a wireless communication terminal in the real world.

The state estimation system of one aspect of the present invention is a state estimation system that estimates the state of the wireless communication terminal or the surrounding environment of the wireless communication terminal, and receives a communication signal transmitted from the wireless communication terminal wirelessly from the communication signal. A wireless communication device that acquires channel information related to radio wave propagation between the wireless communication terminal and itself, a generation device that generates the channel feature amount using the channel information, and the channel feature amount between the wireless communication devices. By inputting the relationship between the channel feature quantity related to radio wave propagation and the state of the surrounding environment of the wireless communication device or wireless communication terminal into the real-world communication model modeled by machine learning, the wireless communication terminal or wireless communication terminal can be used. It is equipped with a detection device that detects the state of the surrounding environment.

The state estimation system of one aspect of the present invention is a state estimation system that estimates the state of the wireless communication terminal or the surrounding environment of the wireless communication terminal, and receives a communication signal transmitted from the wireless communication terminal wirelessly from the communication signal. A wireless communication device that acquires channel information related to radio wave propagation between the wireless communication terminal and itself, a first generation device that generates a channel feature amount using the channel information, and a physical of the wireless communication terminal. A measuring device that measures the state information of any one or more of the state, the type of the wireless communication terminal, and the surrounding environment of the wireless communication terminal, and the wireless that should detect at least one state information of the state information. The target state information that is the state of the communication terminal is used, the information used for detection other than the target state information is used as auxiliary information, the training data is output to the second generator, and the target state information is output using the training data. It is equipped with a third generator that generates a world communication model.

In the state estimation method of one aspect of the present invention, in the state estimation method of estimating the state of the wireless communication terminal or the surrounding environment of the wireless communication terminal, the wireless communication device wirelessly receives the communication signal transmitted from the wireless communication terminal. , The channel information regarding the radio wave propagation between the wireless communication terminal and itself is acquired from the communication signal, the generator generates the channel feature amount using the channel information, and the detection device generates the channel feature amount. Is input to the real-world communication model modeled by machine learning about the relationship between the channel feature amount related to radio wave propagation between wireless communication devices and the state of the surrounding environment of the wireless communication device or wireless communication terminal. Detects the state of the surrounding environment of the terminal or wireless communication terminal.

The state estimation device of one aspect of the present invention is a state estimation device that estimates the state of the surrounding environment of a wireless communication terminal or a wireless communication terminal, and is a channel feature amount related to radio wave propagation between wireless communication devices and the wireless communication device or wireless communication terminal. A storage unit that stores a real-world communication model that models the relationship with the state of the surrounding environment by machine learning, and the wireless communication terminal acquired from a communication signal wirelessly transmitted from the wireless communication terminal and itself. By inputting the channel feature amount to the real-world communication model and the generation unit that generates the channel feature amount using the channel information related to the radio wave propagation between the radio communication terminals, the peripheral environment of the wireless communication terminal or the wireless communication terminal can be obtained. A detection unit for detecting a state is provided.

The state estimation program of one aspect of the present invention is a state estimation program that causes a computer to function as the state estimation device.

According to the present invention, it is possible to provide a technique capable of estimating the state of the surrounding environment of a wireless communication device or a wireless communication terminal in the real world.

FIG. 1 is a diagram showing an overall configuration example of a state estimation system. FIG. 2 is a diagram showing an example of generating a real-world communication model. FIG. 3 is a diagram showing an input / output example of a real-world communication model. FIG. 4 is a diagram showing an operation example of the state estimation system. FIG. 5 is a diagram showing an example of an experimental area. FIG. 6 is a diagram showing an example of setting a target position. FIG. 7 is a diagram showing a processing example of deep learning. FIG. 8 is a diagram showing the experimental results. FIG. 9 is a diagram showing the experimental results. FIG. 10 is a diagram showing a hardware configuration example of the state estimation system.

Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In the description of the drawings, the same parts are designated by the same reference numerals and the description thereof will be omitted.

[Outline of the invention]
The present invention estimates the state information of a wireless communication terminal and its surrounding environment in the real world by using channel information and channel features related to radio wave propagation transmitted by the wireless communication terminal, specifically, from the state information. The purpose is to estimate the selected target state information.

"Channel information" is information indicating how radio waves propagate between the transmitting side and the receiving side. That is, "channel information" is information indicating the state of radio wave propagation between a plurality of antennas provided on the transmitting side and a plurality of antennas provided on the receiving side in MIMO communication technology, and is propagation loss or propagation loss. It is the information obtained from the phase rotation information of the radio wave and the radio wave.

The "channel feature amount" is a numerical value obtained from "channel information" or a numerical value obtained by performing various operations described later on "channel information".

The "state information of the wireless communication terminal" is, for example, the state of the wireless communication terminal such as position, posture, orientation, height, speed, acceleration, rotation, and movement, and is put in a bag held in the hand. Information about the physical state of wireless communication terminals such as laptop computers, smartphones, connected cars, robots, transport devices, types of wireless communication terminals such as drones, and objects that exist around the wireless communication terminals. ..

The "target state information" is the state information to be predicted among the "state information of the wireless terminal".

The present invention is an actual model of the relationship between "channel feature amount" and "target state information", or "channel feature amount" and "wireless communication terminal state information" and "target state information" by machine learning. By generating a world communication model in advance and inputting the channel feature amount of the channel information transmitted from the wireless communication terminal to the real world communication model at a predetermined timing, the actual wireless communication terminal at the predetermined timing is input. Estimate the state information in the world (that is, the target state information).

[Configuration of state estimation system]
FIG. 1 is a diagram showing an overall configuration example of the state estimation system 1 according to the present embodiment. The state estimation system 1 is a state estimation system or a state estimation device that estimates the state information of the wireless communication terminal 2 and its surrounding environment.

The wireless communication terminal 2 will be described. The wireless communication terminal 2 is, for example, an automobile, a drone, or a construction machine vehicle having a wireless communication function. The wireless communication terminal 2 is one or a plurality, and can wirelessly communicate with the state estimation system 1 via the wireless communication network 5. The wireless communication terminal 2 includes a wireless communication unit 21 that performs wireless communication by MIMO communication via a plurality of antennas. One wireless communication terminal 2 may include a plurality of wireless communication units 21.

The state estimation system 1 will be described. The state estimation system 1 is, for example, a base station installed in a main place. The state estimation system 1 may be, for example, a wireless communication terminal, a wireless communication device, a wireless communication device, or a wireless communication device capable of communicating with the wireless communication terminal 2.

As shown in FIG. 1, the state estimation system 1 includes, for example, a plurality of wireless communication units 11, an input feature amount generation unit 12, an auxiliary information generation unit 13, a real-world communication model control unit 14, and a real-world. A communication model storage unit 15 and a network network 16 are provided.

The wireless communication unit 11 is a communication unit that performs wireless communication by MIMO communication via a plurality of antennas. A plurality of wireless communication units 11 can be provided, and FIG. 1 shows two wireless communication units as an example, the first wireless communication unit 11a and the second wireless communication unit 11b. The wireless communication unit 11 has a function of wirelessly receiving a communication signal transmitted from the wireless communication terminal 2. The wireless communication unit 11 has a function of acquiring channel information related to radio wave propagation between the wireless communication terminal 2 and its own wireless communication unit 11 from the received communication signal. The wireless communication unit 11 has a function of outputting the acquired channel information to the input feature amount generation unit 12 via the network network 16. The wireless communication unit 11 may be configured by a wireless communication device.

The "communication signal" is a signal from which channel information transmitted from the wireless communication unit 11 or the wireless communication terminal 2 can be obtained. The communication signal is, for example, a feedback signal including information for feeding back channel information, or a pilot signal. The pilot signal may be any signal known between the devices performing wireless communication.

The input feature amount generation unit 12 has a function of generating a channel feature amount related to radio wave propagation between the wireless communication terminal 2 and the state estimation system 1 by using the channel information acquired from the communication signal. The input feature amount generation unit 12 has a function of inputting the generated channel feature amount to the real-world communication model used by the real-world communication model control unit 14. The input feature amount generation unit 12 may be composed of an input feature amount generation device, a generation device, a first generation device, and a generation unit.

"Channel feature amount" is a feature amount obtained by converting channel information into various forms. The channel information can be expressed in a time-direction form, a frequency-direction form, and a coded form, and is converted in various forms for each wireless communication system. For example, the channel information between the transmitting antenna Nt and the receiving antenna Nr can be expressed as a channel matrix H of Mr × Mt. An example of the "channel feature amount" is a coefficient obtained by performing arithmetic processing on the channel matrix H and the channel matrix H.

For this channel matrix H, QR decomposition, singular value decomposition, correlation matrix such as ^H ^H H and H H H, QR decomposition / eigenvector decomposition of the correlation matrix, and difference information in their frequency direction or time direction, etc. Vectors, phases, levels, etc. obtained by various linear processes can also be generated.

Alternatively, the result of using the arrival direction estimation of the radio wave using multiple antennas or the input information to be used for the arrival direction estimation can be used. Alternatively, compressed information can be used to feed back the channel information. For example, in the case of wireless LAN, as the compressed information of the channel information, the unitary matrix obtained by singular value decomposition or QR decomposition is compressed into angle information with respect to the channel matrix obtained by receiving a known signal. It is quantized and feeds back to the receiving side corresponding to a plurality of subcarriers (subchannels in which frequencies are finely divided) (see Non-Patent Document 1). In addition to the matrix information compressed in this way, features can be generated by using averaging, QR decomposition, and difference information.

The auxiliary information generation unit 13 acquires the state information of the wireless communication terminal 2 from the camera 3 and the sensor 4, and the physical state such as the physical position, speed, direction, and acceleration of the wireless communication terminal 2, and the type of the wireless communication terminal 2. , It has a function to measure surrounding environment information such as the position and number of objects around the wireless communication terminal 2.

Further, the auxiliary information generation unit 13 is via the network 16 or information that is the state information of the wireless communication terminal 2 and does not become the target state information, the target state information that is the target state information but the corresponding time is different, and the network network 16. Information such as weather, date, time, event, weather, temperature, physical position and movement of people and things, composition, size, classification / model information of wireless communication terminal 2 and its owners and possessions. -It has a function to generate contract information, preference information, operation information, and communication mode information as auxiliary information.

"Auxiliary information" is used as input information to the real-world communication model or to select the real-world communication model. Alternatively, the "auxiliary information" may be used as teacher data used when updating various coefficients constituting the real-world communication model when training (learning) the real-world communication model. For example, when auxiliary information is generated once inside the real-world communication model and then used for output of target state information, the auxiliary information can be used as teacher data. The auxiliary information generation unit 13 may be composed of a measuring device such as a camera / sensor and an auxiliary information generation device.

The real-world communication model control unit 14 uses machine learning to pre-model the relationship between the channel features between wireless communication devices and the state of the wireless communication device and its surrounding environment from the real-world communication model storage unit 15. By reading out the communication model and inputting the channel feature amount related to the radio wave propagation and the auxiliary information into the real-world communication model, the state of the wireless communication terminal 2 and its surrounding environment in the real world is detected, and the actual state is detected. It has a function to output the state information of the wireless communication terminal 2 and its surrounding environment in the world. The real-world communication model control unit 14 may be composed of a real-world communication model utilization device, a detection device, and a detection unit.

As described above, the "state information of the wireless communication terminal" is, for example, the state of the wireless communication terminal such as position, posture, orientation, height, speed, acceleration, rotation, and movement, and the state of the bag held in the hand. Physical status of wireless communication terminals such as those inside, types of wireless communication terminals such as laptop computers, smartphones, connected cars, robots, transport devices, drones, objects existing around wireless communication terminals, Information about.

The target state information of the wireless communication terminal 2 for which at least one of the above state information should be detected was used, and the information other than the channel feature amount used for detecting the target state information was used as auxiliary information. For example, the real-world communication model control unit 14 outputs the target state information by using the training data generation unit 14a for generating the training data for forming the real-world communication model that outputs the target state information and the training data. It includes a real-world communication model generation unit 14b that generates a real-world communication model, and a real-world communication model utilization unit 14c that inputs channel feature quantities and auxiliary information into the real-world communication model and uses the real-world communication model. .. The training data generation unit 14a may be configured by a second generation device. The real-world communication model generation unit 14b may be configured by a third generation device.

The real-world communication model storage unit 15 has a function of storing a real-world communication model in which the relationship between the channel features between wireless communication devices and the state of the wireless communication device and its surrounding environment is pre-modeled by machine learning. Be prepared. The real-world communication model includes, for example, a channel feature amount related to radio wave propagation generated by the input feature amount generation unit 12, auxiliary information generated by the auxiliary information generation unit 13, a plurality of wireless communication units 11 and a plurality of wireless communication terminals. It is a learning model generated by using the channel feature amount obtained from at least one radio communication of each radio communication with 2. The real-world communication model storage unit 15 may be composed of a real-world communication model storage device and a storage unit.

[How to generate a real-world communication model]
FIG. 2 is a diagram showing an example of generating a real-world communication model. As the machine learning algorithm, any algorithm such as a support vector machine, a neural network, a decision tree, a gradient boosting decision tree, a random forest, and a gradient boost can be used.

The real-world communication model control unit 14 prepares the channel feature amount obtained from the feedback information or the pilot signal from the wireless communication terminal 2 and the state information (including the target state information) of the wireless communication terminal 2 as training data. Of these, training (machine learning) is performed using a predetermined machine learning algorithm so that at least one or more parameters of the wireless communication terminal 2 can be output as target state information, and coefficients such as coefficients, biases, and determination trees are calculated. By updating, a real-world communication model capable of outputting the state information (target state information) of the wireless communication terminal 2 from the channel feature amount is generated. At this time, information other than the state information of the wireless communication terminal 2 may be used as auxiliary information. The real-world communication model is a model that outputs target state information using channel features or channel features and auxiliary information.

Here, the state information of the wireless communication terminal 2 includes the physical state of the wireless communication terminal 2 (position, speed, rotation, direction, height, holding in hand, in the back), and the type of the wireless communication terminal 2 (in the back). XRs for notebook PCs, smartphones, connected cars, robots, drones, virtual reality (VR: Virtual Reality), augmented reality (AR: Augmented Reality), compound reality (MR: Mixed Reality), alternative reality (SR: Substitutional Reality), etc. Terminal), the physical state of peripheral objects of wireless communication terminal 2.

The auxiliary information is the state information of the wireless communication terminal 2, which is not the target state information, or the target state information, but the corresponding time is different, and the target state information can be obtained via the network network 16. Information such as weather, date, time, event, weather, temperature, physical position and movement of people and things, composition / size / classification / model information / contract information of wireless communication terminal 2 and its owners and possessions. -Preference information / operation information / communication mode information, parameters related to wireless communication, modes such as terminal power consumption, terminal / terminal owner / behavior pattern / preference / mounting position information of the terminal's load, terminal owner's Contract information, appearance or communication recognition information that identifies the terminal / terminal owner / equipment of the terminal, video information from peripheral cameras / sensors, temperature, humidity, light, altitude, tilt, obtained from peripheral sensors, Sensing information such as speed and acceleration can be used.

In the real-world communication model, as an example, the physical state of the wireless communication terminal 2 as the target state information, of which the position information is output, is based on the position positioning by GPS, wireless LAN, Bluetooth, etc., or the camera image of the wireless communication terminal 2. Using a wireless communication terminal that can acquire the position by self-position estimation using LiDAR (Light Detection and Ringing) technology and extraction processing of position information from surveillance cameras and sensing information, it is combined with the input signal to the real-world communication model. By inputting the training data to the real-world communication model control unit 14, training can be performed by machine learning and the real-world communication model can be calculated.

When outputting speed, rotation, orientation, height, etc. as physical states, teacher data for them is generated as training data. Velocity information can be extracted using the accelerometer of the terminal, time-series data of position information, etc., rotation, rotation speed, and direction can be output from the gyro sensor and video / sensor information of the terminal, and height from the barometer. Can be estimated and output using a camera or sensor output that reflects the terminal or the owner / property of the terminal. For example, light cannot be detected from the output of the camera / sensor of the wireless communication terminal 2 in the state of holding it in the hand, in the bag, etc. You can judge the state of touching your ear, holding it in your hand, etc. by taking a direction that cannot be detected, or you can pick it up from the camera or sensor information that shows the terminal or the owner / property of the terminal. You can also determine the state of holding, in the back, etc., or manually judge from the image.

When the target status information to be output is the type of the wireless communication terminal 2, the communication information and the status information of various types of terminals are collected in advance and used as training data, so that any type of terminal can be used. You can generate a real-world communication model that outputs what you have.

When the target state information to be output is the peripheral environment information of the wireless communication terminal 2, the detection result of the peripheral environment by the camera or sensor provided in the wireless communication terminal 2, or the camera 3 or sensor separately installed in the environment. Training data is generated using the detection result of 4, the output result of the position detection device provided by the surrounding object to be detected, or the surrounding environment information generated by human manual judgment, and the information of the surrounding environment is obtained. You can generate a real-world communication model to output. For example, an object capable of measuring its own position can be used as training data by moving around the wireless communication terminal 2 and combining it with the communication information of the wireless communication terminal 2.

The teacher data may be collected in advance and then used by generating a real-world communication model, a state information collecting device separately connected to a network, human work, the above-mentioned physical state, type, and surroundings. It is a wireless communication terminal 2 that can collect environmental information in some form, and the teacher data is collected from other wireless communication terminals 2 by permitting the provision of data, and the training data is gradually increased or generated. You may also update the real-world communication model.

Further, as the target state information, one may be selected from the physical state, the type, and the surrounding object information of the wireless communication terminal 2 described above, or a plurality of them may be selected.

Therefore, the real-world communication model control unit 14 has, for example, training data labeled with information on the state of the wireless communication terminal 2, the type of the wireless communication terminal 2, and information on objects located around the wireless communication terminal 2, and wireless communication. It is possible to generate a real-world communication model that learns the relationship between channel features related to radio wave propagation between devices and trains them to output labeled information corresponding to the input channel features.

The channel feature amount may be measured from a plurality of channel information received simultaneously by the plurality of wireless communication units 11, may be obtained for a plurality of frequencies, or may be used for sound waves, light, or the like. Channel information may also be used, or communication method, algorithm, modulation method, and error correction code information may be combined and generated.

The channel information may be obtained from the received signal received by the first wireless communication unit 11a of the wireless communication terminal 2 or the feedback information included in the received signal.

Note that the feedback information does not necessarily have to be transmitted by the wireless communication terminal 2 (here, the first wireless communication terminal 2a shown in FIG. 1) to the first wireless communication unit 11a. For example, the feedback information transmitted by the first wireless communication terminal 2a to the second wireless communication unit 11b may be used. When controlled in this way, the channel information between the first wireless communication terminal 2a and the first wireless communication unit 11a as well as the channel information between the first wireless communication terminal 2a and the second wireless communication unit 11b are controlled. There is a merit that two radio wave propagation information can be acquired because it can also be acquired.

In this case, the input feature amount generation unit 12 has a channel output from the second wireless communication unit 11b in addition to the channel information output from the first wireless communication unit 11a and the radio wave propagation information developed from the channel information. Information and radio wave propagation information developed from the channel information can be further used.

As a result, the detection accuracy for detecting the state of the wireless communication terminal 2 is improved.

The wireless communication unit 11 may be an external wireless communication unit that is not connected to the network network 16. In this case, for example, the feedback information between the external wireless communication unit and the first wireless communication terminal 2a is received by the first wireless communication unit 11a connected to the network network 16, and the external wireless communication is performed. Channel information between the unit and the first wireless communication terminal 2a can be used.

For example, the first wireless communication unit 11a is a network network unconnected wireless communication unit (second wireless communication unit connected to the network network 16) other than the first wireless communication unit 11a from the first wireless communication terminal 2a. 11b), the communication signal including the channel information transmitted to is received. The input feature amount generation unit 12 includes the received power of the communication signal received from the first wireless communication terminal 2a, the first wireless communication terminal 2a included in the communication signal, the wireless communication unit not connected to the network network, and the first. The channel feature amount is generated by using the channel information with and from the wireless communication unit 11b of 2.

When the wireless communication unit 11 is a wireless communication system such as an orthogonal wave frequency division method that divides into a plurality of frequencies and a channel information corresponding to a plurality of frequencies can be obtained, all of the channel information of each frequency is obtained. You may use it, select frequency channel information with a high reception level, take the sum or average, or perform linear signal processing on the information obtained by the sum or average.

For example, the input feature amount generation unit 12 communicates from the first wireless communication terminal 2a received by the second wireless communication unit 11b (which may be the wireless communication unit not connected to the network network) other than the first wireless communication unit 11a. Using the channel information of a plurality of frequency channels acquired from the signal, information obtained by averaging the information obtained by linear calculation from the channel information for different frequencies is generated.

[How to use the real-world communication model]
FIG. 3 is a diagram showing an input / output example of a real-world communication model. In FIG. 2, if a real-world communication model is constructed, target state information can be output using channel features, channel features, and auxiliary information. As described above, the target state information is at least one of the physical state, type, or peripheral object information of the wireless communication terminal 2.

[Operation of state estimation system]
FIG. 4 is a diagram showing an operation example of the state estimation system 1. In this operation example, the first wireless communication unit 11a uses the pilot signal of the wireless signal transmitted by the first wireless communication terminal 2a to the first wireless communication unit 11a, or the first wireless communication terminal 2a is the first. Using the feedback signal included in the wireless signal transmitted to the wireless communication unit 11a of 1 or another wireless communication unit (for example, the second wireless communication unit 11b), the channel information related to the first wireless communication terminal 2a can be obtained. It is a state estimation method when acquiring.

Step S1;
The first radio communication unit 11a receives a radio signal or a pilot signal including feedback information of channel information transmitted from the first radio communication terminal 2a. From the received feedback information or pilot signal, the first wireless communication unit 11a is between the first wireless communication terminal 2a and itself, or between the first wireless communication terminal 2a and the second wireless communication unit 11b. Acquires channel information related to radio wave propagation in. After that, the first wireless communication unit 11a outputs the acquired channel information to the input feature amount generation unit 12 via the network network 16.

Step S2;
The input feature amount generation unit 12 uses the channel information (first channel information) output from the first wireless communication unit 11a between the first wireless communication terminal 2a and the first wireless communication unit 11a. , Or generate a channel feature amount related to radio wave propagation between the first wireless communication terminal 2a and the second wireless communication unit 11b. At this time, the input feature amount generation unit 12 outputs channel information (second channel information) from one or more wireless communication units such as the second wireless communication unit 11b and another wireless communication unit at the same timing. If so, the channel feature amount is generated by using the second channel information together. After that, the input feature amount generation unit 12 inputs the generated channel feature amount to the real-world communication model of the real-world communication model control unit 14.

For example, the input feature amount generation unit 12 uses the first channel information as the channel feature amount, and uses the received power calculated from the received signal of the pilot signal, the channel matrix, the correlation matrix of the channel matrix, and a plurality of those matrices. Matrix obtained by acquiring and performing linear calculation corresponding to the frequency of, unitary matrix and diagonal matrix obtained by linear calculation, phase, amplitude, real number component, and imaginary number component of those matrices are generated.

For example, the input feature amount generation unit 12 uses the first channel information as the channel feature amount, and the radio communication terminal 2 extracted from the feedback signal notifies the communication partner of the received power, the channel matrix, and the correlation of the channel matrix. Matrix, matrix obtained by acquiring those matrices corresponding to multiple frequencies and performing linear calculation, unitary matrix and diagonal matrix obtained by linear operation on those matrices, phase, amplitude, and real component of those matrices. , Imaginary component, generate transmission mode information to be selected by the sender. If it is a standard for wireless LAN, if it is IEEE802.11, the information of "Beamforming report" described in Non-Patent Document 1 can be used.

For example, the input feature amount generation unit 12 uses the second channel information as the channel feature amount, and the received power and channel obtained from the received signal of the radio packet including the feedback information transmitted by the first wireless communication terminal 2a. Matrix, correlation matrix of channel matrix, matrix obtained by acquiring those matrices corresponding to multiple frequencies and performing linear operation, unitary matrix and diagonal matrix obtained by linear operation on those matrices, and of those matrices Generates phase, amplitude, real and imaginary components.

Step S3;
The auxiliary information generation unit 13 can also acquire the state information of the first wireless communication terminal 2a, the camera / sensor information that can be collected via the network network 16, the weather, and the like and generate the auxiliary information. After that, the auxiliary information generation unit 13 inputs the generated auxiliary information into the real-world communication model of the real-world communication model control unit 14. Note that step S3 may be executed at a timing prior to or at the same timing as step S2.

Step S4;
The real-world communication model control unit 14 reads the real-world communication model from the real-world communication model storage unit 15, and inputs the channel feature amount or the channel feature amount and auxiliary information into the real-world communication model. Outputs the target state information of the wireless communication terminal 2a in the real world.

For example, the real-world communication model control unit 14 is housed in a bag, which is held in the hand, and the state of the wireless communication terminal such as position, posture, orientation, height, speed, acceleration, rotation, and movement. Detects information about the physical state of wireless communication terminals such as laptop computers, smartphones, connected cars, robots, transport devices, types of wireless communication terminals such as drones, and objects that exist around the wireless communication terminals. Use. For example, the physical state can be used for management, log acquisition, and tracking of wireless communication terminals, and the type determination can be used for services that change the required quality according to the type of wireless communication terminal. The object determination can be used for a security system such as detection of harmful animals in a field, determination of an intruder, a system for determining the number of communication terminals in a specific area, and a system for determining density and congestion.

[Experimental result]
The experiments conducted to demonstrate the effect of this embodiment and the results of the experiments are shown in FIGS. 5 to 8. FIG. 5 is a diagram showing an example of an experimental area. FIG. 6 is a diagram showing an example of setting a target position of a movement destination to which the autonomous traveling robot 6 moves in the experimental area. An IEEE 802.11ac wireless communication terminal 2 was mounted on an autonomous traveling robot 6 capable of accurately measuring its own position, and communicated with the second wireless communication unit 11b in FIG. Further, a wireless communication terminal capable of receiving feedback information was installed as the first wireless communication unit 11a in the laboratory. The autonomous traveling robot 6 keeps moving while randomly selecting eight target positions set in the central area in FIG. This position was predicted from the feedback information obtained by the first wireless communication unit 11a.

For the feedback information, the angles φ and ψ of the compressed beamforming feedback matrix obtained by IEEE802.11ac were used as the phase information. Since the second wireless communication unit 11b has four antennas and the wireless communication terminal 2 has two antennas, the angles φ and ψ are set to φ11, φ21, φ31, φ32, ψ21, ψ31, in the “Beamforming report”. A total of 10 angles of ψ41, ψ22, ψ32, and ψ42 can be obtained. In the frequency direction, data obtained from 52 subcarriers can be used, so that 520 of 52 × 10 can be obtained as the angle information. As training data, the relationship between the highly accurate robot position and channel information was trained by deep learning shown in FIG. 7, and a real-world communication model was generated. The channel features are composed of time-series data and are input to the GRU (Gated Recurent Unit) of the hidden layer 1 and dimension 35, the output of 35 is input to the fully connected layer, and the first and second stages are respectively. 35 is output, and the element 2 corresponding to the position information is output in the final layer. The outputs of the fully connected layers in the first and second stages were activated by ReLU (Rectified Linear Unit). In the training, the learning rate was 0.0002, and an optimization algorithm by Adam (Adaptive moment estimation) was used. In IEEE802.11ac, the downlink channel information in the wireless communication terminal is converted into a unitary matrix, and SNR (Signal-to-Noise Ratio) information and a plurality of orthogonal wave frequency divisions are performed by the "Beamforming report" described in Non-Patent Document 1. It is fed back as angle information for the subcarrier of the multiplex method. Although the details are described in the non-patent documents, the V matrix corresponding to the right singular matrix of the channel matrix can be obtained by performing the matrix operation using the angle information.

Here, Mr is the number of receiving antennas, which is 2 in this experiment. Also, since the imaginary part of the last element of each column vector of the V matrix is always 0, if the V matrix is obtained as a matrix of Mt × Mr, the meaningful information is the numerical value of the real part and the imaginary part of each element. Therefore, the numerical value of Mt × Mr-Mr becomes meaningful information. In the case of a 4 × 1 matrix, 7 elements of the real part 4 and the imaginary part 3 are obtained, and in the case of obtaining a 4 × 2 matrix, a total of 14 elements of the real part 8 and the imaginary part 6 are meaningful information. Yes, the imaginary part of the last element in each column is 0, so it does not have to be used.

The position estimation accuracy was verified by the following five methods.

First method (use of power information);
The first method is to receive two SNR information included in the feedback information to the second wireless communication unit 11b and two antennas 1 when the first wireless communication unit 11a receives a signal including the feedback information. A total of four channel feature quantities, the RSSI information in and 2 are acquired every 200 ms, and the time-series data for 5 seconds is trained to output the position information of the wireless communication terminal by the deep learning in FIG. This is a method when the position of the wireless communication terminal is predicted from the SNR information and RSSI information obtained in.

Second method (use of power information + angle information sine cosine);
In the second method, in addition to the first method, the sine / cosine information of 10 angle information corresponding to 52 subcarriers is used as the channel feature amount, and a total of 1044 channel feature amounts are acquired every 200 ms, and 5 The time-series data per second was trained to output the position information of the wireless communication terminal by the deep learning of FIG. 7, and the position of the wireless communication terminal was predicted by the newly obtained SNR information, RSSI information, and sine cosine information. The method of the case.

Third method (power information utilization + V matrix element);
In the third method, in addition to the first method, a 4 × 1 V matrix is calculated from 10 angle information corresponding to 52 subcarriers, and information on the real part and the imaginary part of the V matrix (real part 4). , Imaginary part 3) is used as the channel feature amount, 4 + 7 × 52 = a total of 368 channel feature amounts are acquired every 200 ms, and the time-series data for 5 seconds is obtained by deep learning in FIG. 7 to obtain the position information of the wireless communication terminal. This is a method when the position of the wireless communication terminal is predicted by training to output and using the newly obtained SNR information, RSSI information, and V matrix element information.

Fourth method (power information utilization + average of V matrix elements);
In the fourth method, in addition to the first method, after summing the V matrices of all subcarriers, each element is divided by a constant so that each column vector of the V matrix becomes a unit vector, and the average. The elements of 4 + 14 = 18 obtained as the real part and the imaginary part (real part 8, imaginary part 6) of the V matrix are set as channel feature quantities, acquired every 200 ms, and the time-series data for 5 seconds is obtained by the deep learning in FIG. This is a method in which the position of the wireless communication terminal is predicted by training to output the position information of the wireless communication terminal and using the newly obtained SNR information, RSSI information, and average V matrix information.

Fifth method (power information utilization + V matrix element + V matrix element average);
In the fifth method, in addition to the fourth method, the element of 4 + 14 + 14 = 32, which is obtained by adding the elements of the V matrix (real part 8, imaginary part 6) in the subcarrier of ± 14 from the DC component, is used as the channel feature amount. Acquired every 200 ms and trained to output the position information of the wireless communication terminal by the deep learning of FIG. 7 for 5 seconds, and newly obtained SNR information, RSSI information, and average V matrix information. This is a method when the position of the wireless communication terminal is predicted by the V matrix of two subcarriers.

FIG. 8 shows the result of evaluating the deviation between the value of the position information predicted by each method and the measured value estimated by the self-position estimated by the autonomous traveling robot 6 using the LiDAR technique by MAE (Minimum Absolute Error). It is a figure. As shown in FIG. 6, the position information output by the real-world communication model by deep learning can be predicted by the prepared channel features, and is highly accurate by performing processing such as averaging processing and using a plurality of information. It turns out that it is possible to make it. By simply monitoring the wireless packet transmitted by the wireless communication terminal, it is possible to understand that the spatial position of the wireless communication terminal can be detected with an error accuracy of 1 m or less. Here, the V matrix is averaged, but since the V matrix is preprocessed so that the imaginary part of the last element of each column vector becomes 0, the average of the V matrix is that is, the average of the correlation matrix of the channel matrix. Is equivalent to. Therefore, it is considered that the same effect can be obtained even if the channel matrix obtained from the pilot signal and its correlation matrix are averaged with different subcarriers.

FIG. 9 shows a plot of the predicted X-axis and Y-axis positions and the actual position comparison with respect to time. FIG. 9A is a position on the X-axis, and FIG. 9B is a position on the Y-axis. The solid line is the predicted position, and the broken line is the measured position. The vertical axis does not match the coordinate system of FIG. As shown in FIG. 9, it can be seen that the positions of the X and Y coordinates of the autonomous traveling robot 6 can be accurately detected by using the channel features.

[Effect of this embodiment]
According to the present embodiment, in the state estimation system 1 that estimates the state of the wireless communication terminal 2 and the surrounding environment of the wireless communication terminal, the communication signal transmitted from the wireless communication terminal 2 is wirelessly received, and the communication signal is used as described above. The wireless communication unit 11 that acquires the channel information related to the radio wave propagation between the wireless communication terminal and itself, the input feature amount generation unit 12 that generates the channel feature amount using the channel information, and the channel feature amount. By inputting the relationship between the channel feature amount related to radio wave propagation between wireless communication devices and the state of the surrounding environment of the wireless communication device or wireless communication terminal into a real-world communication model modeled by machine learning, the wireless communication terminal or Since it includes a real-world communication model control unit 14 that detects the state of the peripheral environment of the wireless communication terminal, it is possible to estimate the state of the peripheral environment of the wireless communication terminal 2 and the wireless communication terminal in the real world.

[others]
The state estimation system or the state estimation device in the above-described embodiment may be realized by a computer. In that case, a state estimation program for realizing each of the components of the state estimation system or the state estimation device is recorded on a computer-readable recording medium, and the state estimation program recorded on the recording medium is stored in the computer system. It may be realized by loading and executing.

"Computer system" includes hardware such as OS and peripheral devices. For example, as shown in FIG. 10, it is a general-purpose computer system including a CPU 901, a memory 902, a storage 903, a communication device 904, an input device 905, and an output device 906.

"Computer readable recording medium" refers to a portable medium such as a flexible disk, magneto-optical disk, ROM, CD-ROM, or a storage device such as a hard disk built in a computer system. Further, a "computer-readable recording medium" is a communication line for transmitting a program via a network such as the Internet or a communication line such as a telephone line, and dynamically holds the program for a short period of time. It may also include a program that holds a program for a certain period of time, such as a volatile memory inside a computer system that is a server or a client in that case. Further, the above program may be for realizing a part of the above-mentioned components, and may be further realized by combining the above-mentioned components with a program already recorded in the computer system. Often, it may be realized by using hardware such as PLD (Programmable Logic Device) or FPGA (Field Programmable Gate Array).

As described above, the embodiment of the present invention has been described in detail with reference to the drawings, but the specific configuration is not limited to this embodiment, and the design and the like within a range not deviating from the gist of the present invention are also included.

1: State estimation system 11: Wireless communication unit 12: Input feature amount generation unit 13: Auxiliary information generation unit 14: Real-world communication model control unit 14a: Training data generation unit 14b: Real-world communication model generation unit 14c: Real-world communication Model utilization unit 15: Real-world communication model storage unit 16: Network network 2: Wireless communication terminal 21: Wireless communication unit 3: Camera 4: Sensor 5: Wireless communication network 6: Autonomous traveling robot 901: CPU
902: Memory 903: Storage 904; Communication device 905: Input device 906: Output device

Claims

In a state estimation system that estimates the state of a wireless communication terminal or the surrounding environment of a wireless communication terminal
A wireless communication device that wirelessly receives a communication signal transmitted from a wireless communication terminal and acquires channel information related to radio wave propagation between the wireless communication terminal and itself from the communication signal.
A generator that generates channel features using the channel information,
By inputting the channel feature amount into a real-world communication model that models the relationship between the channel feature amount related to radio wave propagation between wireless communication devices and the state of the surrounding environment of the wireless communication device or wireless communication terminal by machine learning. , A detection device that detects the state of the wireless communication terminal or the surrounding environment of the wireless communication terminal, and
A state estimation system equipped with.
The generator is
As the channel feature amount, the received power, the channel matrix, the correlation matrix of the channel matrix, the channel matrix or the arithmetic matrix obtained by linearly performing the correlation matrix in association with a plurality of frequencies, the channel matrix or the correlation matrix. , The channel matrix or the diagonal matrix obtained by linearly calculating the correlation matrix, the channel matrix, the correlation matrix, the arithmetic matrix, the unity matrix, and the diagonal matrix. The state estimation system according to claim 1, wherein any one or more of the phase, amplitude, real number component, imaginary number component, and transmission mode information of one or more of the matrices is generated.
The generator is
Information obtained by linear calculation from channel information for different frequencies using channel information of a plurality of frequency channels acquired from communication signals from the wireless communication terminal received by a second wireless communication device other than the wireless communication device. The state estimation system according to claim 1 or 2, which generates information obtained by averaging the two.
The wireless communication device is
A communication signal including channel information transmitted from the wireless communication terminal to a second wireless communication device other than the wireless communication device is received.
The generator is
A channel feature amount is generated by using the received power of the communication signal received from the wireless communication terminal and the channel information between the wireless communication terminal and the second wireless communication device included in the communication signal. The state estimation system according to any one of claims 1 to 3.
The real-world communication model is
Training data labeled with information of any one or more of the state of the wireless communication terminal, the type of the wireless communication terminal, and an object located around the wireless communication terminal, and a channel related to radio wave propagation between wireless communication devices. The state estimation system according to any one of claims 1 to 4, which is a model for learning the relationship between the feature amount and the input channel feature amount and training to output the labeled information corresponding to the input channel feature amount.
In a state estimation system that estimates the state of a wireless communication terminal or the surrounding environment of a wireless communication terminal
A wireless communication device that wirelessly receives a communication signal transmitted from a wireless communication terminal and acquires channel information related to radio wave propagation between the wireless communication terminal and itself from the communication signal.
A first generator that generates channel features using the channel information,
A measuring device that measures the state information of any one or more of the physical state of the wireless communication terminal, the type of the wireless communication terminal, and the surrounding environment of the wireless communication terminal.
A second generation of training data in which at least one of the state information is the target state information that is the state of the wireless communication terminal to be detected, and information used for detection other than the target state information is used as auxiliary information. Generation device and
Using the training data, a third generator that generates a real-world communication model that outputs the target state information, and
A state estimation system equipped with.
In the state estimation method for estimating the state of the wireless communication terminal or the surrounding environment of the wireless communication terminal,
The wireless communication device wirelessly receives a communication signal transmitted from the wireless communication terminal, acquires channel information related to radio wave propagation between the wireless communication terminal and itself from the communication signal, and obtains channel information.
The generator uses the channel information to generate channel features.
The detection device creates a real-world communication model in which the channel feature amount is modeled by machine learning about the relationship between the channel feature amount related to radio wave propagation between wireless communication devices and the state of the surrounding environment of the wireless communication device or wireless communication terminal. A state estimation method for detecting the state of the wireless communication terminal or the surrounding environment of the wireless communication terminal by inputting.
In a state estimation device that estimates the state of the wireless communication terminal or the surrounding environment of the wireless communication terminal
A storage unit that stores a real-world communication model that models the relationship between the channel features related to radio wave propagation between wireless communication devices and the state of the surrounding environment of the wireless communication device or wireless communication terminal by machine learning.
A generation unit that generates channel features using channel information related to radio wave propagation between the wireless communication terminal and itself acquired from a communication signal transmitted wirelessly from the wireless communication terminal.
A detection unit that detects the state of the wireless communication terminal or the surrounding environment of the wireless communication terminal by inputting the channel feature amount into the real-world communication model.
A state estimator equipped with.
A state estimation program that causes a computer to function as the state estimation device according to claim 8.