JP2020085595A - Indoor state estimating method and indoor state estimating system - Google Patents

Abstract

To estimate the operation state of an indoor facility using CSIs of a wireless LAN radio wave.SOLUTION: The indoor state estimating method comprises a search phase for collecting CSIs of a wireless LAN radio wave in advance and discovering a motif due to the operation of an indoor facility and an estimation phase for detecting the operation of the indoor facility using the discovered motif. The search phase includes: a step 1 for estimating Doppler effects from the CSIs and estimating the arrival direction of a radio wave from the phase difference of the CSIs observed by a plurality of receive antennas of a wireless LAN receiver; a step 2 for calculating a Matrix profile from the time-series data of Doppler effects and taking the time-series pattern of Doppler effects where repeatedly occurring similarity is high as a motif candidate; a step 3 for combining the motif candidate and the information of a radio wave arrival direction and clustering it; and a step 4 for determining that the similar time-series patterns of Doppler effects repeatedly occurring to a given radio wave arrival direction are motifs due to the operation of the indoor facility.SELECTED DRAWING: Figure 2

Description

The present invention relates to an indoor state estimation method and an indoor state estimation system for detecting and analyzing Wi-Fi radio waves and estimating an operation state of indoor equipment.

Research on methods for detecting the state of indoor equipment, such as opening and closing doors, is underway. Door open/closed detection can be applied to a wide range of fields such as crime prevention systems that detect intrusion of people, home automation such as air conditioning/lighting control, and monitoring of the elderly. The research so far has been realized by installing an acceleration sensor, a gyro sensor, a vibration sensor, and a switch sensor in each indoor facility, and collecting opening/closing information by wireless communication (Non-Patent Document 1). However, these methods require a sensor to be installed in each indoor facility, which requires time and effort in terms of battery replacement, sensor sensor replacement, etc., resulting in high installation and management costs.

Also, with the recent widespread use of wireless LANs, wireless LAN devices are installed in various indoor environments such as offices and ordinary homes, so wireless LAN radio waves can be used inexpensively. Since changes in the propagation of wireless LAN radio waves occur due to changes in the environment such as operation of indoor equipment and movement of people, changes in the environment can be estimated from propagation information of the wireless LAN radio waves. However, in an indoor environment, wireless LAN radio waves are reflected and shielded by various obstacles such as furniture and walls, so that the propagation of wireless LAN radio waves is complicated. In many research fields such as indoor position estimation and action recognition of people using wireless LAN radio waves, estimations have been performed using supervised machine learning by associating environmental changes with changes in wireless LAN radio wave propagation characteristics. I've been told. However, these methods require the collection of learning data to which many labels have been added in advance in the target environment, so that there is a problem that the cost at the time of system introduction is large.

Further, channel state information (CSI: Channel State Information) is applied to estimate the state of an indoor environment as propagation information of a wireless LAN radio wave. CSI represents changes in amplitude and phase due to the effects of multipath such as propagation loss of wireless LAN radio waves and reflection/diffraction, by an absolute value of a complex number and a deviation angle. In this research on the position estimation of a person using CSI, the position estimation that does not require prior learning is performed by estimating the Doppler effect due to the movement of the person and the direction of arrival of the wireless LAN radio wave reflected by the person (non- Patent Document 2). When the object moves, the path length of the radio wave reflected by the object changes, and the CSI phase component observed by the receiver changes. Therefore, the Doppler effect due to the movement of the object is estimated from the time change of the CSI phase component. can do. In addition, by using an antenna array in which the receiving antennas are arranged in a straight line, there is a difference in the path length of the radio waves received by each receiving antenna depending on the direction of arrival of the radio waves, so that each receiving antenna is observed. The arrival direction of the wireless LAN radio wave to the antenna array can be estimated using the CSI.

Philipose, M., Fishkin, KP, Perkowitz, M., Patterson, DJ, Fox, D., Kautz, H. and H ahnel, D.: Inferring activities from interactions with objects, Pervasive Computing, IEEE, Vol.3, No.4, pp.50-57 (2004). Li, X., Li, S., Zhang, D., Xiong, J., Wang, Y. and Mei, H.: Dynamic-music: accurate device-free indoor localization, Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing, pp.196-207 (2016). Yeh, C.-CM, Zhu, Y., Ulanova, L., Begum, N., Ding, Y., Dau, HA, Silva, DF, Mueen, A. and Keogh, E.: Matrix profile I: all pairs similarity joins for time series: a unifying view that includes motifs, discords and shapelets, Proceedings of the 16th International Conference on Data Mining, pp. 1317-1322 (2016).

However, since the Doppler effect obtained from CSI is caused not only by the door but also by the movement of various objects, it is difficult to directly estimate the operation of the door from the time change of the estimated Doppler effect. However, compared with the movement of people, it is considered that the time change of the Doppler effect due to the opening/closing operation of the door is similar to each other. Further, since one end of the door is fixed, the arrival direction of the radio wave reflected by the door is constant.

An object of the present invention is to provide an indoor state estimation method and an indoor state estimation system capable of estimating the operation state of indoor equipment using CSI of wireless LAN radio waves.

A first invention is an indoor state estimating method for analyzing a wireless LAN radio wave transmitted and received between a wireless LAN transmitter and a wireless LAN receiver arranged indoors to estimate a motif due to an operation of indoor equipment. It is provided with a search phase in which channel state information of LAN radio waves (hereinafter referred to as CSI) is collected in advance to discover a motif due to an operation of the indoor equipment, and an estimation phase in which an operation of the indoor equipment is detected using the discovered motif. In the search phase, the Doppler effect is estimated from the CSI and the arrival direction of the radio wave is estimated from the phase difference of the CSI observed at a plurality of receiving antennas of the wireless LAN receiver, and the Matrix profile is calculated from the time series data of the Doppler effect. Of the Doppler effect with a high degree of similarity, which is calculated as a motif candidate, step 2; step 3 of combining the motif candidate and the information of the arrival direction of the radio wave; and step 3 of the constant arrival of the radio wave. Step 4 of determining a time-series pattern of similar Doppler effect repeatedly occurring with respect to a direction as a motif due to operation of indoor equipment.

In the indoor state estimating method according to the first aspect of the present invention, when the door opening and closing is the motif caused by the operation of the indoor equipment, the step 3 occurs first when the motif candidates belonging to different clusters are continuously observed. Let the cluster of motif be the cluster corresponding to the opening operation of the door, and the cluster of motif generated later be the cluster corresponding to the closing operation of the door.

A second aspect of the present invention is an indoor state estimation system that analyzes a wireless LAN radio wave transmitted and received between a wireless LAN transmitter and a wireless LAN receiver arranged indoors and estimates a motif due to an operation of indoor equipment. It is provided with a searching means for collecting LAN radio channel state information (hereinafter, referred to as CSI) in advance and discovering a motif due to the operation of the indoor equipment, and an estimating means for detecting the operation of the indoor equipment by using the found motif. The search means estimates the Doppler effect from the CSI, and estimates the direction of arrival of the radio wave from the phase difference of the CSI observed at a plurality of receiving antennas of the wireless LAN receiver, and the Matrix from the time series data of the Doppler effect. A profile for calculating a profile, a means for finding a candidate that uses a time-series pattern of the Doppler effect with a high degree of similarity that is a repeated candidate, and a candidate for clustering that combines the candidate and information about the direction of arrival of radio waves. It is provided with a motif determining means for determining a time series pattern of similar Doppler effect repeatedly generated for a constant arrival direction of radio waves as a motif due to operation of indoor equipment.

In the indoor state estimating system according to the second aspect of the present invention, when the opening/closing of the door is carried out by the operation of the indoor equipment, the motion candidate clustering means is configured to generate the first motion when different motion candidates are continuously observed. In this configuration, the cluster of is a cluster corresponding to the operation of opening the door, and the cluster of the motif generated later is a cluster corresponding to the operation of closing the door.

In the present invention, by calculating the Matrix profile from the time series data of the Doppler effect estimated from the CSI, the time series pattern of the Doppler effect with a high degree of repetitive similarity is found as a motif candidate. Then, the discovered candidate motions are combined with the arrival direction information of the reflected wave and clustered, and the similar time series pattern of the Doppler effect that is repeatedly generated in a certain direction is found as the motif by the operation of the indoor equipment.

Here, as an example of indoor equipment, when there are doors that open inside and outside, it is not known in which direction the movement of the door plate corresponds to the opening or closing operation. Therefore, in general doors, it is noted that the closing operation often occurs immediately after the opening operation of the door, and when different motifs are continuously observed, the cluster of motifs generated earlier is Corresponds to "open operation", and associates a cluster of motifs that occur later to "close operation". When multiple doors are installed, the direction of the door with respect to the receiver is calculated using the floor plan of the room prepared in advance to estimate which door was opened or closed from the arrival direction of the reflected wave. To do. Thereby, the cost of acquiring the learning data in advance can be reduced.

It is a figure which shows the open/close state of a door and the change of the propagation of a wireless LAN radio wave. It is a figure which shows the structural example of a search phase. P(v) when opening/closing the inward door It is a figure which shows an example of the time change of. It is a figure which shows Matrix profile at the time of repeating opening and closing operation of a door.

First, the basic idea of the proposed method is explained.
FIG. 1 shows changes in the open/closed state of a door and the propagation of wireless LAN radio waves.
In FIG. 1, assuming that one or more wireless LAN transmitters 11 and wireless LAN receivers 12 each equipped with a wireless LAN module are installed in the target room, the state of the indoor equipment is set as a detection target. To do. Hereinafter, the open/closed state of the door 13 will be described as the detection target as indoor equipment.

Since the wireless LAN radio wave hits an obstacle and is reflected, the radio wave from the direction of the detection target changes depending on the open/close state of the door 13 that is the detection target. For example, when the inwardly opening door 13 is opened, the path length is shortened, which causes a negative Doppler effect, and when the door 13 is closed, the path length is lengthened, which causes a positive Doppler effect.

The proposed method consists of a search phase that discovers a motif that indicates the operating state of a door from data that is not collected in advance and a presumption phase that detects the door operation using the discovered motif.

FIG. 2 shows a configuration example of the search phase.
2, in the search phase, the CSI time-series data of the reflected wave of the wireless LAN radio wave is input to the Doppler effect estimation section 21 and the reflected wave arrival direction estimation section 22 which are CSI processing means, and the Doppler effect and the reflected wave arrival are received. Estimate the direction. Next, the motif candidate finding unit 23 finds a pattern having a high repetitive occurrence as a motif candidate from the estimated Doppler effect by using Matrix profile (Non-Patent Document 3) or another method. Since the door is fixed indoors, it is considered that the Doppler effect due to door operation has a high degree of similarity with each other. When multiple doors are installed or motion candidates include movements of people or the like, the motion candidate clustering unit 24 uses the Doppler effect similarity and the arrival direction information to cluster the motion candidates. Then, the motif determining unit 25 determines the motif at the time of operating the door.

Here, an example of estimating the Doppler effect due to the door operation from the CSI time-series data using the MUSIC (MUltiple SIgnal Classification) algorithm is shown, but other methods may be used.

In the following, (1) preprocessing in the Doppler effect estimation unit 21 and (2) estimation of the change rate of the path length, (3) discovery of the motion candidate in the motion candidate discovery unit 23, (4) reflection in the reflected wave arrival direction estimation unit 22 Wave arrival direction estimation and (5) clustering of motif candidates in the clustering candidate of motifs will be described in this order.

(1) Pre-processing The CSI observed by the wireless LAN receiver has static components such as direct waves and dynamic components due to radio waves reflected by moving objects. Since only the dynamic component is necessary for estimating the Doppler effect, the static component is removed from the CSI observed as preprocessing. When the object reflected by the wireless LAN radio wave moves, the path length of the wireless LAN radio wave changes, and the phase component of CSI changes. Since wireless LAN radio waves are reflected to various indoors, if the number of routes is L and the rate of change of the i-th route length is v _i , the CSI value h(Δt) at a time Δt after a certain time is It is expressed by the following equation.

Here, A _i is the amplitude of the radio wave on the i-th route, and τ _i is the arrival time of the radio wave on the i-th route. Further, f is the frequency of the wireless LAN radio wave, and c is the speed of light.

However, since the wireless LAN radio wave transmitter/receiver is not actually synchronized, a random phase offset θ ₀ exists for each packet. Therefore, CSI is obtained as h(Δt)exp(−jθ ₀ ). Therefore, the phase offset is canceled by performing the complex conjugate multiplication between the CSIs observed by the two antennas. If h ₁ and h ₂ are CSI obtained at the receiving antenna 1 and the receiving antenna 2, respectively, the CSI obtained by performing the complex conjugate multiplication is

Is expressed as When the CSI obtained by each antenna i is divided into a static component h _si composed of a direct wave or the like and a dynamic component h _mi due to a reflected wave or the like during door operation, h _c is represented by the following equation.

The first term on the right side of equation (2) is the multiplication of static components. Since the influence of the static component such as the direct wave is larger than that of the dynamic component due to the door operation, it is necessary to subtract the static component (first term on the right side) from h _c . At this time, the average value of h _c observed over a plurality of packets is used as the static component. The fourth term on the right-hand side of Eq. (2) is the multiplication of the dynamic components, and its absolute value is small, so it has little influence on the Doppler effect estimation. From this, h _{cm obtained} by subtracting the static component from h _c can be approximated by the following equation.

Formula (3) In the first term and the second term on the right side of the above, h _m1 includes the change speed v _i of the path length to be obtained, and h _m2 includes −v _i whose sign is opposite to the change speed of the path length to be obtained. Be done. Therefore, when the Doppler effect is obtained from h _cm , v _i and −v _i are obtained, so only the absolute value of the Doppler effect can be estimated. Therefore, the constant α is subtracted from h ₁ and the constant β ² is added to h ₂ to reduce the influence of the second term and to easily estimate v _i .

(2) Path length change rate estimation The path length change rate v _i is obtained by applying the MUSIC algorithm to the pre-processed CSI value h _cm . Since CSI is obtained for each subcarrier, h _cm is obtained N _s for each packet. Here, the preprocessed CSIs obtained at time Δt are arranged for each subcarrier.
Is defined. However, h _cm (Δt, fi) is preprocessed CSI in the subcarrier of frequency f _i . Then, the CSI observation matrix H observed over M packets is represented by the following equation.

When the number of paths of the wireless LAN radio and L, the eigenvalue is large L eigenvectors signal subspace, the eigenvectors corresponding to the remaining M-L eigenvalues are noise subspace E _N. Then, P(v) expressed by the following equation using the noise subspace E _N Is calculated while changing v.

P(v) V corresponding to the peak of is the estimated change rate of the path length. However, the wireless LAN radio noise is large, since the influence of the -v _i path length rate of change in polarity is opposite to be obtained there, P (v) The value of P(v) is used instead of the value of v corresponding to the peak of.

Fig. 3 shows P(v) when the inside door is opened and closed. An example of the change over time of is shown.
In FIG. 3, it can be seen that when the door is opened, the path length becomes short, so that the negative Doppler effect occurs, and when the door is closed, the path length becomes long, so that the positive Doppler effect occurs.

(3) Discovery of motif candidates In this procedure, repeated motif candidates are discovered from the P(v) time series data estimated using the Matrix profile. First, the similarity matrix is obtained by dividing the time series data of P(v) by time windows having a window width W and calculating the reciprocal of the Euclidean distance between the time windows as the similarity. The similarity s is expressed by the following equation.

Here, P ₁ (t,v) and P ₂ (t,v) are P(v) calculated for velocity v at time t in each time window of time series data of P(v) for which similarity calculation is performed. Is. The maximum value of the similarity other than the diagonal components for each row of the similarity matrix is the Matrix profile. A row with a large Matrix profile value (time window) means that the time series pattern is repeated twice or more, so a time window with a Matrix profile value greater than a specific threshold is a candidate for a motif. In particular, since the path length of the reflected wave changes greatly in the door and it is fixed in the room, the degree of similarity when opening and closing the door is greater than when operating other daily objects such as windows or moving furniture such as chairs. Become.

FIG. 4 shows a Matrix profile when the opening/closing operation of the door is repeated.
In FIG. 4, since a person freely walks in the room during each door operation, it can be seen that the similarity degree is low when the person walks and the similarity degree is high when the opening/closing operation is performed. ..

(4) Direction of arrival estimation First, in order to remove random phase offset θ ₀ for each packet or remove static components and extract dynamic components by door operation, the absolute value of amplitude is the largest and the variance is small. Preprocessing (1) is performed using the complex conjugate of the CSI of the receiving antenna. Then, the arrival direction is estimated using the preprocessed CSI.

When the arrival direction of the wireless LAN radio wave is θ, the phase difference Δφ of the radio wave between the two antennas is expressed by the following equation.

Here, d is the distance between the antennas, and λ is the wavelength of the wireless LAN radio wave. Therefore, the arrival direction of the radio wave can be estimated from the CSI phase differences observed by the plurality of receiving antennas. However, the phase component of CSI has not only the information on the arrival direction but also the influence of the Doppler effect and the arrival time of the radio wave described above. Therefore, the arrival direction is estimated from the CSI by simultaneously estimating these three parameters. The arrival direction, the arrival time, and the changing speed of the path length of the i-th propagation path are set to θ _i , τ _i , and v _i , respectively. With reference to the 1st receiving antenna, 1st subcarrier, and CSI at time 0, the rth receiving antenna, sth subcarrier, and CSI value at time t, h _r (r _, s _, t) is Is approximated by.

Further, f _c is the center frequency of the wireless LAN radio wave, and f δ is the frequency difference between subcarriers.

For simplicity
Then, in the estimation of the direction of arrival, Θ that maximizes the logarithmic likelihood function represented by the following equation is obtained by a SAGE (Space Alternating Generalized Expectation Maximization) algorithm which is an extension of the EM algorithm.

It is a theoretical value of CSI calculated using the estimated parameters.

In the SAGE algorithm, E step and M step are sequentially performed for each propagation path, and this is repeated until convergence. In the E step, the CSI value of the propagation path of interest is calculated by the following equation.

Where ^Θ is the estimate in the previous step. In the M step, the parameter of the propagation path of interest is optimized using the following equation.

M is the number of packets. The finally obtained θ _i is the arrival direction of the reflected wave from the door.

(5) When multiple doors exist in the clustering environment of candidate motifs, the motif found in (3) cannot distinguish which door was opened or closed. In addition, the movement of an object other than a door such as a person may be detected as a motif. On the other hand, since the door is fixed in the room, in addition to the Doppler effect, the similarity of the arrival direction at the time of operating the door is large. Therefore, first, the similarity with the largest similarity and each time window are calculated. Then, among time windows whose similarity is equal to or greater than a threshold value, a set of time windows whose difference in arrival direction is equal to or less than the threshold value is set as a cluster of time windows during a certain door operation. Then, a similar process is performed using the motif with the highest similarity among the time windows not selected as clusters, and a new cluster is formed. By repeating the above process twice as many times as the number of doors in the environment, a cluster is created for each opening and closing operation of the doors in the environment.

It is considered that two clusters with similar arrival directions estimated in (4) correspond to opening and closing of each door. In addition, when the door continues to open and close, it often occurs in the order of opening and then closing. Therefore, when it is detected that a certain door is opened and closed continuously, the cluster corresponding to the operation of opening the clutter that occurs first and the cluster corresponding to the operation of closing the cluster that occurs later are set.
The above is the search phase.

Next, the estimation phase will be described.
In the estimation phase, the Doppler effect and the arrival direction of the reflected wave are estimated from the estimation data. Then, by calculating the similarity between the found motif and the estimation data, it is estimated which door was opened and closed. That is, it is determined which cluster the time windows of the estimation data belong to, or whether it does not belong to any cluster. For that purpose, the average value of the difference in the arrival direction between the estimation data time window and each motif (time window) in each cluster is calculated, and the estimation data time window in which the arrival direction difference is smaller than the threshold value is calculated. Calculate the average Doppler effect distance for a combination of clusters.

Here, DTW (Dynamic Time Warping) is used to calculate the distance of the similarity between the motif and the estimation data in order to cope with the fluctuation in the time taken to open and close the door. If there is a cluster whose average distance is smaller than the threshold value, the time window of the estimation data belongs to the cluster with the shortest average distance. Further, it is assumed that the time window in which there is no cluster in which the difference in the direction of arrival or the average distance of the Doppler effect is smaller than the threshold does not belong to any cluster. With this, when it is estimated that the time window for estimation data belongs to a certain cluster, it is estimated which door was opened or closed based on the door operation and the position of the corresponding door. can do.

11 wireless LAN transmitter 12 wireless LAN receiver 13 door 21 Doppler effect estimation unit 22 reflected wave arrival direction estimation unit 23 motif candidate discovery unit 24 motif candidate clustering unit 25 motif determination unit

Claims (4)

In an indoor state estimation method for analyzing a wireless LAN radio wave transmitted and received between a wireless LAN transmitter and a wireless LAN receiver arranged indoors and estimating a motif due to operation of indoor equipment,
A search phase in which the channel state information (hereinafter referred to as CSI) of the wireless LAN radio waves is collected in advance to discover a motif due to the operation of the indoor equipment, and an estimation phase in which the operation of the indoor equipment is detected using the found motif. With and
The search phase is
Step 1 of estimating the Doppler effect from the CSI and estimating the arrival direction of the radio wave from the phase difference of the CSI observed at a plurality of receiving antennas of the wireless LAN receiver;
Step 2 of calculating a Matrix profile from the time series data of the Doppler effect, and using the time series pattern of the Doppler effect having a high degree of repetitive similarity as a motif candidate,
Clustering by combining the candidate motion and the information on the arrival direction of the radio wave;
A step 4 of determining a time-series pattern of similar Doppler effect repeatedly generated for a certain arrival direction of the radio wave as a motif due to operation of the indoor equipment. In the indoor state estimation method according to claim 1,
When the door is opened and closed by the operation of the indoor equipment,
In the step 3, when the motif candidates belonging to different clusters are continuously observed, the cluster of the motif generated first is set as a cluster corresponding to the operation of opening the door, and the cluster of the motif generated later is set to the door. An indoor state estimation method characterized by forming a cluster corresponding to the closing operation of. In an indoor state estimation system that analyzes a wireless LAN radio wave transmitted and received between a wireless LAN transmitter and a wireless LAN receiver placed indoors and estimates a motif due to operation of indoor equipment,
Search means for collecting channel state information (hereinafter, referred to as CSI) of the wireless LAN radio wave in advance and discovering a motif due to operation of the indoor equipment, and estimating means for detecting operation of the indoor equipment using the found motif. With and
The searching means is
CSI processing means for estimating the Doppler effect from the CSI and estimating the arrival direction of a radio wave from the phase difference of the CSI observed at a plurality of receiving antennas of the wireless LAN receiver,
Calculating a Matrix profile from the time-series data of the Doppler effect, a motif candidate discovering means that sets the time-series pattern of the Doppler effect with high similarity that is repeated as a motif candidate,
A candidate candidate clustering unit for clustering by combining the candidate candidates and information on the arrival direction of the radio wave;
An indoor state estimation system, comprising: a time determination pattern of a similar Doppler effect that is repeatedly generated with respect to a constant arrival direction of the radio wave, as a moment determined by the operation of the indoor equipment. .. The indoor state estimation system according to claim 3,
When the door is opened and closed by the operation of the indoor equipment,
When the different motif candidates are continuously observed, the motif candidate clustering means sets the cluster of the motif that occurs first as a cluster corresponding to the operation of opening the door, and the cluster of the motif that occurs later on the door. An indoor state estimation system characterized by having a cluster structure corresponding to a closing operation.