JP7067703B2 - Indoor state estimation method and indoor state estimation system - Google Patents

Description

The present invention relates to an indoor state estimation method and an indoor state estimation system that detect and analyze Wi-Fi radio waves to estimate the operating state of indoor equipment.

Research is underway on methods for detecting the state of indoor equipment, such as opening and closing doors. Door open / close detection can be applied to a wide range of fields such as crime prevention systems that detect the intrusion of people, home automation such as air conditioning / lighting control, and watching over elderly people. Previous studies have been realized by installing acceleration sensors, gyro sensors, vibration sensors, and switch sensors in each indoor facility and collecting open / close information by wireless communication (Non-Patent Document 1). However, these methods require sensors to be installed in each of the indoor equipment, which requires time and effort in terms of battery replacement and replacement in the event of sensor failure, resulting in high installation and management costs.

Further, with the spread of wireless LAN in recent years, wireless LAN devices are installed in various indoor environments such as offices and general households, so that wireless LAN radio waves can be used inexpensively. Since changes in the propagation of wireless LAN radio waves occur due to changes in the environment such as the operation of indoor equipment and the movement of people, changes in the environment can be estimated from the propagation information of the wireless LAN radio waves. However, in an indoor environment, the propagation of wireless LAN radio waves is complicated because the wireless LAN radio waves are reflected and shielded by various obstacles such as furniture and walls. In research fields such as indoor position estimation and behavior recognition of people using many wireless LAN radio waves so far, estimation is performed using supervised machine learning by associating environmental changes with changes in the propagation characteristics of wireless LAN radio waves. It has been broken. However, since these methods require the collection of learning data with many labels in the target environment in advance, there is a problem that the cost at the time of system introduction is large.

Further, as the propagation information of the wireless LAN radio wave, channel state information (CSI: Channel State Information) is applied to the state estimation of the indoor environment. CSI expresses the change in amplitude and phase due to the influence of multipath such as propagation loss of wireless LAN radio wave and reflection / diffraction by the absolute value and argument of a complex number. In the study of human position estimation using this CSI, the position estimation that does not require prior learning is performed by estimating the Doppler effect due to the movement of the person and the arrival direction of the wireless LAN radio wave reflected by the person (non-pre-learning). Patent Document 2). As the object moves, the path length of the radio waves reflected by the object changes, and the phase component of the CSI observed by the receiver changes. Therefore, the Doppler effect due to the movement of the object is estimated from the time change of the phase component of the CSI. can do. In addition, by using an antenna array in which the receiving antennas are lined up in a straight line, the path length of the radio waves received by each receiving antenna differs depending on the arrival direction of the radio waves, so that it is observed by each receiving antenna. It is possible to estimate the arrival direction of the wireless LAN radio wave to the antenna array by using the CSI.

Philipose, M., Fishkin, K.P., Perkowitz, M., Patterson, D.J., Fox, D., Kautz, H. and H ahnel, D .: Inferring activities from interactions with objects, Pervasive Computing, IEEE, Vol.3, No.4, pp.50-57 (2004). Li, X., Li, S., Zhang, D., Xiong, J., Wang, Y. and Mei, H .: Dynamic-music: accurate device-free indoor localization, Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing, pp.196-207 (2016). Yeh, C.-C.M., Zhu, Y., Ulanova, L., Begum, N., Ding, Y., Dau, H.A., Silva, D.F., Mueen, A. and Keogh, E .: Matrix profile I: all pairs similarity joins for time series: a unifying view that includes motifs, discords and shapelets, Proceedings of the 16th International Conference on Data Mining, pp. 1317-1322 (2016).

However, since the Doppler effect obtained from CSI is caused by the movement of various objects, not limited to the door, it is difficult to directly estimate the operation of the door from the estimated time change of the Doppler effect. However, it is considered that the time change of the Doppler effect due to the opening and closing operation of the door is similar to each other as compared with the movement of people and the like. Moreover, since one end of the door is fixed, the direction of arrival of the radio wave reflected by the door is also constant.

An object of the present invention is to provide an indoor state estimation method and an indoor state estimation system capable of estimating the operating state of indoor equipment using the CSI of a wireless LAN radio wave.

The first invention is an indoor state estimation method in which a wireless LAN radio wave transmitted and received between a wireless LAN transmitter and a wireless LAN receiver arranged indoors is analyzed and a motif is estimated by operating an indoor facility. It has a search phase that collects LAN radio wave channel status information (hereinafter referred to as CSI) in advance to discover motifs by operating indoor equipment, and an estimation phase that detects operations of indoor equipment using the discovered motifs. In the search phase, the Doppler effect is estimated from the CSI, and the arrival direction of radio waves is estimated from the phase difference of the CSI observed by multiple receiving antennas of the wireless LAN receiver. Step 2 which uses the time-series pattern of the Doppler effect with high recurring similarity as the motif candidate, step 3 where the motif candidate and the information of the arrival direction of the radio wave are combined and clustered, and the constant arrival of the radio wave. It has step 4 of determining a time-series pattern of similar Doppler effects that repeatedly occur in a direction as a motif by operating indoor equipment.

In the indoor state estimation method of the first invention, when the motif by operating the indoor equipment is used to open and close the door, step 3 occurs earlier when the motif candidates belonging to different clusters are continuously observed. The motif cluster is defined as the cluster corresponding to the door opening operation, and the motif cluster generated later is defined as the cluster corresponding to the door closing operation.

The second invention is in an indoor state estimation system that analyzes wireless LAN radio waves transmitted and received between a wireless LAN transmitter and a wireless LAN receiver arranged indoors and estimates a motif by operating indoor equipment. It is equipped with a search means for collecting LAN radio wave channel status information (hereinafter referred to as CSI) in advance to discover a motif by operating indoor equipment, and an estimation means for detecting operation of indoor equipment using the discovered motif. The search means are a CSI processing means that estimates the Doppler effect from the CSI and estimates the arrival direction of radio waves from the phase difference of the CSI observed by multiple receiving antennas of the wireless LAN receiver, and a Matrix from the time-series data of the Doppler effect. A motif candidate finding means that calculates a profile and uses a time-series pattern of the Doppler effect that occurs repeatedly and has a high degree of similarity as a motif candidate, and a motif candidate clustering means that clusters by combining the motif candidate and information on the arrival direction of radio waves. It is equipped with a motif determination means that determines the time-series pattern of similar Doppler effects that repeatedly occur in a certain direction of arrival of radio waves as a motif by operating indoor equipment.

In the indoor state estimation system of the second invention, when the motif by operating the indoor equipment is to open and close the door, the motif candidate clustering means is the motif that occurred earlier when different motif candidates are continuously observed. The cluster is a cluster that corresponds to the operation of opening the door, and the cluster of motif that occurs later is the cluster that corresponds to the operation of closing the door.

In the present invention, by calculating the Matrix profile from the time-series data of the Doppler effect estimated from CSI, a time-series pattern of the Doppler effect with high recurring similarity is discovered as a motif candidate. Then, the found motif candidates are clustered by combining with the arrival direction information of the reflected wave, and the time-series pattern of the similar Doppler effect that repeatedly occurs in a certain direction is discovered as a motif by operating the indoor equipment.

Here, as an example of indoor equipment, when there are doors that open inward and outward, it is unknown in which direction the movement of the door plate corresponds to the operation of opening or closing. Therefore, paying attention to the fact that in general doors, the closing operation often occurs immediately after the door opening operation occurs, and when different motifs are observed continuously, the cluster of motifs that occurred earlier is referred to as " Correspond to "Open operation" and correspond to "Close operation" to the cluster of motif that occurred later. Then, when multiple doors are installed, by calculating the direction of the door with respect to the receiver using the floor plan of the room prepared in advance, it is estimated which door was opened and closed from the direction of arrival of the reflected wave. do. As a result, it is possible to reduce the cost of acquiring the learning data in advance.

It is a figure which shows the open / closed state of a door, and the change of the propagation of a wireless LAN radio wave. It is a figure which shows the structural example of a search phase. P (v) when opening and closing the inward door It is a figure which shows an example of the time change of. It is a figure which shows the Matrix profile when the door opening and closing operation is repeated.

First, the basic idea of the proposed method will be explained.
FIG. 1 shows the open / closed state of the door and the change in the propagation of the wireless LAN radio wave.
In FIG. 1, it is assumed that one or more wireless LAN transmitters 11 and wireless LAN receivers 12 equipped with wireless LAN modules are installed indoors, and the state of the indoor equipment is detected. do. Hereinafter, the open / closed state of the door 13 will be described as an indoor facility as a detection target.

Since the wireless LAN radio wave hits an obstacle and is reflected, the radio wave from the direction of the detection target changes depending on the open / closed state of the door 13 to be detected. For example, when the door 13 that opens inward is opened, the path length is shortened, so that a negative Doppler effect is produced, and when the door 13 is closed, the path length is long, so that a positive Doppler effect is produced.

The proposed method consists of a search phase for discovering a motif indicating the operation status of the door from unlabeled data collected in advance, and an estimation phase for detecting the operation of the door using the discovered motif.

FIG. 2 shows a configuration example of the search phase.
In FIG. 2, in the search phase, the CSI time-series data of the reflected wave of the wireless LAN radio wave is input to the Doppler effect estimation unit 21 and the reflected wave arrival direction estimation unit 22, which are CSI processing means, and the Doppler effect and the reflected wave arrive. Estimate the direction. Next, the motif candidate discovery unit 23 discovers a pattern having a high degree of similarity that repeatedly occurs as a motif candidate by using a Matrix profile (Non-Patent Document 3) or another method from the estimated Doppler effect. Since the door is fixed indoors, it is considered that the Doppler effect due to the door operation has a high degree of similarity to each other. Since there are cases where multiple doors are installed or the motif candidate includes movements of people, etc., the clustering unit 24 of the motif candidate uses information on the similarity of the Doppler effect and the arrival direction to perform clustering of the motif candidate. Then, the motif determination unit 25 determines the motif when the door is operated.

Here, an example of estimating the Doppler effect by operating the door from the time series data of CSI using the MUSIC (MUltiple SIgnal Classification) algorithm is shown, but other methods may be used.

Hereinafter, (1) preprocessing and (2) path length change rate estimation in the Doppler effect estimation unit 21, (3) motif candidate discovery in the motif candidate discovery unit 23, and (4) reflection in the reflected wave arrival direction estimation unit 22. The wave arrival direction estimation and (5) motif candidate clustering in the motif candidate clustering unit 24 will be described in this order.

(1) Preprocessing The CSI observed by the wireless LAN receiver has static components such as direct waves and dynamic components due to radio waves reflected by moving objects. Since only the dynamic component is required to estimate the Doppler effect, the static component is removed from the CSI observed as a pretreatment. When an object that reflects the wireless LAN radio wave moves, the path length of the wireless LAN radio wave changes, and the phase component of the CSI changes. Since wireless LAN radio waves are reflected by various things indoors, if the number of paths is L and the rate of change of the i-th path length is v _i , the CSI value h (Δt) at the time after Δt from a certain time is It is expressed by the following equation.

Here, A _i is the amplitude of the radio wave of the i-th path, and τ _i is the arrival time of the radio wave of the i-th path. Further, f is the frequency of the wireless LAN radio wave, and c is the speed of light.

However, in reality, since the transmitter / receiver of the wireless LAN radio wave is not synchronized, there is a random phase offset θ ₀ for each packet. Therefore, CSI is obtained as h (Δt) exp (−jθ ₀ ). Therefore, the phase offset is canceled by performing complex conjugate multiplication between the CSIs observed by the two antennas. Assuming that h ₁ and h ₂ are the CSIs obtained by the receiving antenna 1 and the receiving antenna 2, respectively, the CSI obtained by performing the complex conjugate multiplication is

と表される。各アンテナｉで得られるＣＳＩを直接波等で構成される静的成分ｈ_siとドア操作時の反射波等による動的成分ｈ_miに分割すると、ｈ_c は次式で表される。

It is expressed as. When the CSI obtained by each antenna i is divided into a static component h _si composed of a direct wave or the like and a dynamic component h _mi due to a reflected wave or the like during door operation, h _c is expressed by the following equation.

The first term on the right-hand side of Eq. (2) is the multiplication of static components. Since the influence of static components such as direct waves is larger than that of dynamic components due to door operation, it is necessary to subtract the static components (first term on the right side) from h _c . At this time, the average value of h _c observed over a plurality of packets is used as the static component. The fourth term on the right-hand side of Eq. (2) is the multiplication of dynamic components, and since the absolute value is small, it has little effect on the Doppler effect estimation. From this, h _cm obtained by subtracting the static component from h _c can be approximated by the following equation.

Equation (3) In the first and second terms on the right side of_m1The rate of change of the path length you want to find_iIs included, h_m2The change speed of the path length you want to find is opposite to the positive and negative -v._iIs included. Therefore, h_cmIf you ask for the Doppler effect from, v_iAnd -v_iTherefore, only the absolute value of the Doppler effect can be estimated. So h₁Subtract the constant α from h₂Constant β²By adding, the influence of the second term is reduced, and v_i Make it easier to estimate.

を定義する。ただし、ｈ_cm(Δt、fi) は周波数ｆ_i のサブキャリアにおける前処理済みのＣＳＩである。すると、Ｍパケットにわたって観測したＣＳＩの観測行列Ｈは次式で表される。 (2) Estimating the change rate of the path length The change rate v _i of the path length is obtained by applying the MUSIC algorithm to the preprocessed CSI value h _cm . Since CSI is obtained for each subcarrier, h _cm is obtained for N _s per packet. Here, the preprocessed CSI obtained at time Δt is arranged for each subcarrier.

Is defined. However, h _cm (Δt, fi) is the preprocessed CSI in the subcarriers of frequency _fi . Then, the observation matrix H of CSI observed over the M packet is expressed by the following equation.

Assuming that the number of paths of the wireless LAN radio wave is L, the L eigenvectors having a large eigenvalue are the signal subspace, and the eigenvectors corresponding to the remaining ML eigenvalues are the noise subspace E._NIs. Then, the noise subspace E_NP (v) expressed by the following equation using Is calculated while changing v.

P (v) V corresponding to the peak of is the estimated change rate of the path length. However, the noise of the wireless LAN radio wave is large, and the change speed of the desired path length is opposite to the positive and negative -v._iBecause there is also the influence of P (v) Use the value of P (v) instead of the value of v corresponding to the peak of.

FIG. 3 shows P (v) when the door that opens inward is opened and closed. An example of the time change of is shown.
In FIG. 3, it can be seen that when the door is opened, the path length is shortened, so that a negative Doppler effect is generated, and when the door is closed, the path length is long, so that a positive Doppler effect is generated.

(3) Discovery of motif candidates In this procedure, motif candidates that occur repeatedly are discovered from the time-series data of P (v) estimated using the Matrix profile. First, the time series data of P (v) is divided by the time window of the window width W, and the reciprocal of the Euclidean distance between each time window is calculated as the similarity to obtain the similarity matrix. The similarity s is expressed by the following equation.

Here, P ₁ (t, v) and P ₂ (t, v) calculate the velocity v at time t in each time window of the time series data of P (v) for which the similarity is calculated. Is. The maximum value of similarity other than the diagonal component for each row of the similarity matrix is the Matrix profile. A row with a large Matrix profile value (time window) means that the time series pattern has occurred repeatedly more than once, so a time window with a Matrix profile value greater than a specific threshold is a candidate for motif. In particular, the path length of the reflected wave of the door changes greatly and is fixed in the room, so the degree of similarity when opening and closing the door is large compared to the operation of other everyday objects such as windows and the movement of furniture such as chairs. Become.

FIG. 4 shows a Matrix profile when the door opening / closing operation is repeatedly performed.
In FIG. 4, since a person walks freely in the room during each door operation, it can be seen that the similarity is low when the person walks and the similarity is high when the opening / closing operation is performed. ..

(4) Arrival direction estimation
First, a random phase offset θ for each packet₀In order to remove the static component and extract the dynamic component by door operation, the preprocessing of (1) is performed using the complex conjugate of the CSI of the receiving antenna with the largest absolute value of amplitude and the smallest variance. conduct. After that, the arrival direction is estimated using the preprocessed CSI.

Assuming that the arrival direction of the wireless LAN radio wave is θ, the phase difference Δφ of the radio wave between the two antennas is expressed by the following equation.

Here, d is the distance between the antennas, and λ is the wavelength of the wireless LAN radio wave. Therefore, the arrival direction of radio waves can be estimated from the phase difference of CSI observed by a plurality of receiving antennas. However, the phase component of CSI includes not only the information of the arrival direction but also the influence of the above-mentioned Doppler effect and the influence of the arrival time of the radio wave. Therefore, by estimating these three parameters at the same time, the direction of arrival is estimated from the CSI. Let θ _i , τ _i , and v _i be the arrival direction, arrival time, and change speed of the path length of the i-th propagation path, respectively. With reference to the 1st receiving antenna, the 1st subcarrier, and the CSI at time 0, the rth receiving antenna, the sth subcarrier, and the CSI value at time t h _r (r _, s _, t) are given by the following equations. Is approximated by.

Further, f _c is the center frequency of the wireless LAN radio wave, and f δ is the difference in frequency between the subcarriers.

とすると、到来方向の推定では次式で表される対数尤度関数が最大になるΘをＥＭアルゴリズムの拡張であるＳＡＧＥ（Space Alternating Generalized Expectation Maximization) アルゴリズムによって求める。 For simplification

Then, in the estimation of the arrival direction, the Θ that maximizes the log-likelihood function expressed by the following equation is obtained by the SAGE (Space Alternating Generalized Expectation Maximization) algorithm, which is an extension of the EM algorithm.

推定されたパラメータを用いて計算されたＣＳＩの理論値である。

It is a theoretical value of CSI calculated using the estimated parameters.

In the SAGE algorithm, the E step and the M step are sequentially performed for each propagation path, and this is repeated until they converge. In the E step, the CSI value of the propagation path of interest is calculated by the following equation.

Here, ^ Θ is the estimated value in the previous step. In the M step, the parameters of the propagation path of interest are optimized using the following equation.

Ｍはパケット数である。最終的に求められたθ_i をドアによる反射波の到来方向とする。

M is the number of packets. Let θ _i finally obtained be the direction of arrival of the reflected wave by the door.

(5) Clustering of motif candidates
If there are multiple doors in the environment, the motif found in (3) cannot tell which door was opened or closed. Also, the movement of objects other than doors, such as people, may be detected as motifs. On the other hand, since the door is fixed in the room, in addition to the Doppler effect, the degree of similarity in the arrival direction when operating the door is also large. Therefore, first, the motif with the highest similarity and the similarity of each time window are calculated. Then, among the time windows whose similarity is equal to or higher than the threshold value, a set of time windows whose difference in the arrival direction is equal to or lower than the threshold value is defined as a cluster of time windows at the time of a certain door operation. Then, among the time windows that were not selected as clusters, the same process is performed using the motif with the highest similarity, and a new cluster is formed. By repeating the above process twice as many times as the number of doors in the environment, a cluster is created for each door opening / closing operation in the environment.

It is considered that two clusters with similar arrival directions estimated in (4) correspond to the opening and closing of each door. In addition, when doors are opened and closed continuously, they often occur in the order of opening and then closing. Therefore, when it is detected that a certain door has been opened and closed continuously, the cluster corresponding to the operation of opening the clutter that occurred earlier and the cluster corresponding to the operation of closing the cluster that occurred later are used.
The above is the search phase.

Next, the estimation phase will be described.
In the estimation phase, the Doppler effect and the direction of arrival of the reflected wave are first estimated from the estimation data. Then, by calculating the similarity between the found motif and the estimation data, it is estimated which door was opened and closed and when. That is, it is determined which cluster formed by each time window of the estimation data belongs to, or does not belong to any cluster. For that purpose, the average value of the difference in the arrival direction between the time window of the estimation data and each motif (time window) in each cluster is calculated, and the time window of the estimation data in which the difference in the arrival direction is smaller than the threshold value is calculated. Calculate the mean distance of the Doppler effect for a combination of clusters.

Here, DTW (Dynamic Time Warping) is used to calculate the distance between the motif and the estimation data in order to deal with the fluctuation of the time required for opening and closing the door. Then, if there is a cluster whose average distance is smaller than the threshold value, it is assumed that the time window of the estimation data belongs to the cluster having the closest average distance. It is also assumed that the time window in which there is no cluster whose arrival direction difference or average distance of the Doppler effect is smaller than the threshold value does not belong to any cluster. As a result, if it is estimated that the time window of the estimation data belongs to a cluster, it is estimated from the operation of the corresponding door and the position of the door which door was opened or closed. can do.

11 Wireless LAN transmitter 12 Wireless LAN receiver 13 Door 21 Doppler effect estimation unit 22 Reflected wave arrival direction estimation unit 23 motif candidate discovery unit 24 motif Candidate clustering unit 25 motif determination unit

Claims (2)

In the indoor state estimation method that analyzes the wireless LAN radio waves transmitted and received between the wireless LAN transmitter and the wireless LAN receiver located indoors and estimates the motif by operating the indoor equipment.
A search phase in which channel state information (hereinafter referred to as CSI) of the wireless LAN radio wave is collected in advance to discover a motif by operating the indoor equipment, and an estimation phase in which the discovered motif is used to detect the operation of the indoor equipment. And with
The search phase is
Step 1 in which the Doppler effect is estimated from the CSI and the arrival direction of the radio wave is estimated from the phase difference of the CSI observed by the plurality of receiving antennas of the wireless LAN receiver.
Step 2 in which the Matrix profile is calculated from the time-series data of the Doppler effect and the time-series pattern of the Doppler effect that occurs repeatedly and has a high degree of similarity is used as a motif candidate.
Step 3 for clustering by combining the motif candidate and the information on the arrival direction of the radio wave, and
It has step 4 and step 4 in which a time-series pattern of a similar Doppler effect that repeatedly occurs in a certain direction of arrival of the radio wave is determined as a motif by operating the indoor equipment.
When the door is opened and closed by operating the indoor equipment,
In step 3, when the motif candidates belonging to different clusters are continuously observed, the motif cluster that occurs first is the cluster corresponding to the operation of opening the door, and the motif cluster that occurs later is the door. An indoor state estimation method characterized by forming a cluster corresponding to the closing operation of the door . In an indoor state estimation system that analyzes wireless LAN radio waves transmitted and received between a wireless LAN transmitter and a wireless LAN receiver located indoors and estimates a motif by operating indoor equipment.
A search means for collecting channel state information (hereinafter referred to as CSI) of the wireless LAN radio wave in advance to discover a motif by operating the indoor equipment, and an estimation means for detecting the operation of the indoor equipment using the discovered motif. And with
The search means
A CSI processing means that estimates the Doppler effect from the CSI and estimates the arrival direction of radio waves from the phase difference of the CSI observed by a plurality of receiving antennas of the wireless LAN receiver.
A motif candidate finding means that calculates a Matrix profile from the time-series data of the Doppler effect and uses a time-series pattern of the Doppler effect that occurs repeatedly and has a high degree of similarity as a motif candidate.
A motif candidate clustering means for clustering by combining the motif candidate and information on the arrival direction of the radio wave, and
It is equipped with a motif determining means for determining a time-series pattern of a similar Doppler effect that repeatedly occurs in a certain direction of arrival of the radio wave as a motif by operating the indoor equipment .
When the door is opened and closed by operating the indoor equipment,
In the motif candidate clustering means, when the motif candidates belonging to different clusters are continuously observed, the previously generated motif cluster is set as the cluster corresponding to the operation of opening the door, and the later generated motif cluster is used. It is a configuration to make a cluster corresponding to the operation of closing the door.
An indoor state estimation system characterized by that.

Images