KR20230031660A - Method and apparatus for performing wireless sensing in wireless LAN system - Google Patents

Abstract

Provided are a method for performing wireless sensing in a wireless LAN system based on wireless sensing and a device thereof. Specifically, a first wireless apparatus performs capability negotiation with second and third wireless apparatuses. The first wireless apparatus obtains capability information based on the capability negotiation. The first wireless apparatus determines a role switching mechanism of the first to third wireless apparatuses based on the capability information.

Description

Method and apparatus for performing wireless sensing in a wireless LAN system

The present specification relates to a method for identifying a user based on wireless sensing, and more particularly, to a method and apparatus for performing wireless sensing by switching Tx/Rx roles in a wireless device.

With advances in wireless technology and sensing methods, many studies have used wireless signals (e.g., WiFi) to detect human activity for intrusion detection, daily activity recognition, vital sign monitoring related to more detailed motion detection, and more. It has succeeded in realizing various application fields such as gesture recognition for user identification.

These applications can support a variety of domains for smart home and office environments, including safety protection, wellness monitoring/management, smart healthcare and smart appliance interaction.

Human body movement affects radio signal propagation (e.g., reflection, diffraction, and scattering), and analyzing the received radio signal provides a good opportunity to capture human movement. Researchers extract ready-to-use signal measurements or adopt frequency-modulated signals to determine frequency shift. Due to its low-cost and non-intrusive detection characteristics, radio-based human activity detection has attracted considerable attention and has become a prominent research field in the last decade.

This specification examines existing wireless sensing systems in terms of basic principles, techniques, and system architecture. Specifically, it describes how wireless signals can be leveraged to facilitate a variety of applications, including intrusion detection, meeting room occupancy monitoring, daily activity recognition, gesture recognition, vital sign monitoring, user identification, and indoor localization. Future research directions and limitations of using radio signals for human activity detection are also described.

The present specification proposes a method and apparatus for performing wireless sensing in a wireless LAN system.

An example of the present specification proposes a method in which a wireless device performs wireless sensing by switching Tx/Rx roles.

Conventionally, since a wireless device has a fixed role in transmitting and measuring a wireless signal, there is a problem in that only limited coverage is generated by applying a sensing area only to a pair of transmitting and receiving devices. This embodiment proposes a method in which a wireless device based on wireless sensing expands a sensing area by switching Tx/Rx roles based on exchanged capabilities information.

The first wireless device performs capabilities negotiation with the second and third wireless devices. The first wireless device performs device discovery before performing the capability negotiation, and the second and third wireless devices are discovered or detected based on the device discovery.

The first wireless device acquires capability information based on the capability negotiation. The first wireless device may acquire the capability information while exchanging capability negotiation request/response messages with the second and third wireless devices.

The capability information may include transmission or reception role information of the wireless device, wireless sensing information, RSSI (Received Signal Strength Indicator), CSI amplitude (Channel State Information Amplitude), location and distance information of the wireless device, or performance information of the wireless device. can The transmission or reception role information of the wireless device may indicate that the first wireless device supports both transmission and reception, the second wireless device supports only transmission, and the third wireless device supports only reception. .

The first wireless device determines a role switching mechanism of the first to third wireless devices based on the capability information.

According to the embodiment proposed in this specification, it is possible to solve the problem of the sensing sound area due to the location or arrangement of the wireless device, and to compare the existing coverage when multiple wireless devices coexist with a signal pattern suitable for the user's home environment. A wider coverage can be used, and a system that efficiently collects, learns, and predicts can be implemented, which has the effect of creating a new paradigm of IoT future smart home devices.

1 shows an example of a transmitting device and/or a receiving device of the present specification.
2 is a conceptual diagram showing the structure of a wireless LAN (WLAN).
3 is a diagram illustrating a general link setup process.
4 shows a procedure flow diagram of WiFi sensing.
5 shows a general procedure flow diagram of human activity sensing via wireless signals.
6 shows a CSI spectrogram according to human walking.
7 shows a deep learning architecture for user authentication.
8 shows an example of a problem according to the limited sensing coverage of a transmitting/receiving device.
9 shows another example of a problem according to the limited sensing coverage of a transmitting/receiving device.
10 shows a block diagram of a Tx/Rx Role Switching function for wireless sensing based coverage enhancement.
11 shows an example of a procedure in which a wireless sensing device performs Device Discovery and Capabilities Negotiation.
12 shows an example of a wireless sensing procedure when there are one Tx Device, one Tx/Rx Device, and one Rx Device.
13 shows an example of a wireless sensing procedure when there are two Tx/Rx devices and one Rx device.
14 shows an example of a wireless sensing procedure when there are three Tx / Rx devices.
15 shows an example of a wireless sensing scenario when there are one Tx Device, one Tx/Rx Device, and one Rx Device.
16 shows an example of a wireless sensing scenario when there are Tx/Rx Devices 1 and 2 and Rx Device 1.
17 shows an example of a wireless sensing scenario when there are Tx / Rx Devices 1 to 3.
18 is a flowchart illustrating a procedure of performing wireless sensing by switching Tx/Rx roles according to the present embodiment.
19 shows a modified example of the transmitter and/or receiver of the present specification.

In this specification, “A or B” may mean “only A”, “only B” or “both A and B”. In other words, “A or B (A or B)” in the present specification may be interpreted as “A and / or B (A and / or B)”. For example, “A, B or C (A, B or C)” herein means “only A”, “only B”, “only C” or “any combination of A, B and C (any combination of A, B and C)”.

A slash (/) or comma (comma) used in this specification may mean “and/or”. For example, “A/B” may mean “and/or B”. Accordingly, "A/B" can mean "only A", "only B", or "both A and B". For example, “A, B, C” may mean “A, B or C”.

In this specification, “at least one of A and B” may mean “only A”, “only B” or “both A and B”. In addition, in this specification, the expression “at least one of A or B” or “at least one of A and/or B” means “at least one of A and B (at least one of A and B)”.

In addition, in this specification, “at least one of A, B and C” means “only A”, “only B”, “only C” or “A, B and C It may mean “any combination of A, B and C”. Also, “at least one of A, B or C” or “at least one of A, B and/or C” means It can mean "at least one of A, B and C".

Also, parentheses used in this specification may mean “for example”. Specifically, when displayed as “control information (EHT-Signal)”, “EHT-Signal” may be suggested as an example of “control information”. In other words, “control information” in this specification is not limited to “EHT-Signal”, and “EHT-Signal” may be suggested as an example of “control information”. Also, even when displayed as “control information (ie, EHT-signal)”, “EHT-Signal” may be suggested as an example of “control information”.

Technical features that are individually described in one drawing in this specification may be implemented individually or simultaneously.

The following examples of this specification can be applied to various wireless communication systems. For example, the following example of the present specification may be applied to a wireless local area network (WLAN) system. For example, this specification may be applied to the IEEE 802.11a/g/n/ac standard or the IEEE 802.11ax standard. In addition, this specification may be applied to the newly proposed EHT standard or IEEE 802.11be standard. In addition, an example of the present specification may be applied to a new wireless LAN standard that enhances the EHT standard or IEEE 802.11be. In addition, an example of the present specification can be applied to a mobile communication system. For example, it can be applied to Long Term Evolution (LTE) based on the 3rd Generation Partnership Project (3GPP) standard and a mobile communication system based on its evolution. In addition, an example of the present specification may be applied to a communication system of the 5G NR standard based on the 3GPP standard.

Hereinafter, technical features to which the present specification can be applied will be described in order to describe the technical features of the present specification.

1 shows an example of a transmitting device and/or a receiving device of the present specification.

The example of FIG. 1 may perform various technical features described below. 1 relates to at least one STA (station). For example, the STAs 110 and 120 of the present specification include a mobile terminal, a wireless device, a wireless transmit/receive unit (WTRU), a user equipment (UE), It may also be called various names such as a mobile station (MS), a mobile subscriber unit, or simply a user. The STAs 110 and 120 of the present specification may be called various names such as a network, a base station, a Node-B, an access point (AP), a repeater, a router, and a relay. The STAs 110 and 120 of this specification may be called various names such as a receiving device, a transmitting device, a receiving STA, a transmitting STA, a receiving device, and a transmitting device.

For example, the STAs 110 and 120 may perform an access point (AP) role or a non-AP role. That is, the STAs 110 and 120 of the present specification may perform functions of an AP and/or a non-AP. In this specification, an AP may also be indicated as an AP STA.

The STAs 110 and 120 of the present specification may support various communication standards other than the IEEE 802.11 standard together. For example, communication standards (eg, LTE, LTE-A, 5G NR standards) according to 3GPP standards may be supported. In addition, the STA of the present specification may be implemented in various devices such as a mobile phone, a vehicle, and a personal computer. In addition, the STA of the present specification may support communication for various communication services such as voice call, video call, data communication, and autonomous driving (Self-Driving, Autonomous-Driving).

In this specification, the STAs 110 and 120 may include a medium access control (MAC) conforming to the IEEE 802.11 standard and a physical layer interface for a wireless medium.

The STAs 110 and 120 will be described based on sub-drawing (a) of FIG. 1 as follows.

The first STA 110 may include a processor 111 , a memory 112 and a transceiver 113 . The illustrated processor, memory, and transceiver may be implemented as separate chips, or at least two or more blocks/functions may be implemented through one chip.

The transceiver 113 of the first STA performs signal transmission and reception operations. Specifically, IEEE 802.11 packets (eg, IEEE 802.11a/b/g/n/ac/ax/be) may be transmitted and received.

For example, the first STA 110 may perform an intended operation of the AP. For example, the processor 111 of the AP may receive a signal through the transceiver 113, process the received signal, generate a transmission signal, and perform control for signal transmission. The memory 112 of the AP may store a signal received through the transceiver 113 (ie, a received signal) and may store a signal to be transmitted through the transceiver (ie, a transmission signal).

For example, the second STA 120 may perform an intended operation of a non-AP STA. For example, the non-AP transceiver 123 performs signal transmission and reception operations. Specifically, IEEE 802.11 packets (eg, IEEE 802.11a/b/g/n/ac/ax/be) may be transmitted and received.

For example, the processor 121 of the non-AP STA may receive a signal through the transceiver 123, process the received signal, generate a transmission signal, and perform control for signal transmission. The memory 122 of the non-AP STA may store a signal received through the transceiver 123 (ie, a received signal) and may store a signal to be transmitted through the transceiver (ie, a transmission signal).

For example, an operation of a device indicated as an AP in the following specification may be performed by the first STA 110 or the second STA 120. For example, when the first STA 110 is an AP, the operation of the device indicated by the AP is controlled by the processor 111 of the first STA 110, and by the processor 111 of the first STA 110 A related signal may be transmitted or received via the controlled transceiver 113 . In addition, control information related to the operation of the AP or transmission/reception signals of the AP may be stored in the memory 112 of the first STA 110 . In addition, when the second STA 110 is an AP, the operation of the device indicated by the AP is controlled by the processor 121 of the second STA 120, and is controlled by the processor 121 of the second STA 120 A related signal may be transmitted or received through the transceiver 123 that becomes. In addition, control information related to the operation of the AP or transmission/reception signals of the AP may be stored in the memory 122 of the second STA 110 .

For example, the operation of a device indicated as a non-AP (or User-STA) in the following specification may be performed by the 1st STA 110 or the 2nd STA 120. For example, when the second STA 120 is a non-AP, the operation of a device marked as non-AP is controlled by the processor 121 of the second STA 120, and the processor of the second STA 120 ( A related signal may be transmitted or received via the transceiver 123 controlled by 121 . In addition, control information related to non-AP operations or AP transmission/reception signals may be stored in the memory 122 of the second STA 120 . For example, when the first STA 110 is a non-AP, the operation of a device marked as non-AP is controlled by the processor 111 of the first STA 110, and the processor of the first STA 120 ( A related signal may be transmitted or received through the transceiver 113 controlled by 111). In addition, control information related to non-AP operations or AP transmission/reception signals may be stored in the memory 112 of the first STA 110 .

In the following specification, (transmitting / receiving) STA, 1st STA, 2nd STA, STA1, STA2, AP, 1st AP, 2nd AP, AP1, AP2, (transmitting / receiving) terminal, (transmitting / receiving) device , (transmitting / receiving) apparatus, a device called a network, etc. may mean the STAs 110 and 120 of FIG. 1 . For example, (transmitting/receiving) STA, 1st STA, 2nd STA, STA1, STA2, AP, 1st AP, 2nd AP, AP1, AP2, (transmitting/receiving) Terminal, (transmitting) without specific reference numerals. Devices indicated as /receive) device, (transmit/receive) apparatus, network, etc. may also mean the STAs 110 and 120 of FIG. 1 . For example, in the following example, an operation in which various STAs transmit and receive signals (eg, PPPDUs) may be performed by the transceivers 113 and 123 of FIG. 1 . Also, in the following example, an operation in which various STAs generate transmission/reception signals or perform data processing or calculation in advance for transmission/reception signals may be performed by the processors 111 and 121 of FIG. 1 . For example, an example of an operation of generating a transmission/reception signal or performing data processing or calculation in advance for the transmission/reception signal is: 1) Determining bit information of subfields (SIG, STF, LTF, Data) included in the PPDU /Acquisition/Configuration/Operation/Decoding/Encoding operations, 2) Time resources or frequency resources (eg, subcarrier resources) used for subfields (SIG, STF, LTF, Data) included in the PPDU, etc. 3) a specific sequence used for a subfield (SIG, STF, LTF, Data) field included in the PPDU (eg, pilot sequence, STF/LTF sequence, applied to SIG) extra sequence), 4) power control operation and/or power saving operation applied to the STA, 5) operation related to determination/acquisition/configuration/operation/decoding/encoding of an ACK signal, etc. can include In addition, in the following example, various information (eg, information related to fields / subfields / control fields / parameters / power, etc.) used by various STAs to determine / acquire / configure / calculate / decode / encode transmission and reception signals It may be stored in the memories 112 and 122 of FIG. 1 .

The above-described device/STA of FIG. 1 (a) may be modified as shown in FIG. 1 (b). Hereinafter, the STAs 110 and 120 of the present specification will be described based on the subfigure (b) of FIG. 1 .

For example, the transceivers 113 and 123 shown in sub-drawing (b) of FIG. 1 may perform the same function as the transceiver shown in sub-drawing (a) of FIG. 1 described above. For example, the processing chips 114 and 124 shown in sub-drawing (b) of FIG. 1 may include processors 111 and 121 and memories 112 and 122 . The processors 111 and 121 and the memories 112 and 122 shown in the sub-drawing (b) of FIG. 1 are the processors 111 and 121 and the memories 112 and 122 shown in the sub-drawing (a) of FIG. ) can perform the same function as

Mobile terminal, wireless device, wireless transmit/receive unit (WTRU), user equipment (UE), mobile station (MS), mobile, described below Mobile Subscriber Unit, user, user STA, network, base station, Node-B, AP (Access Point), repeater, router, relay, receiving device, transmitting device, receiving STA, transmission STA, Receiving Device, Transmitting Device, Receiving Apparatus, and/or Transmitting Apparatus refer to the STAs 110 and 120 shown in sub-drawings (a)/(b) of FIG. ) may mean the processing chips 114 and 124 shown in. That is, the technical features of the present specification may be performed in the STAs 110 and 120 shown in sub-drawings (a) / (b) of FIG. 1, and the processing chip shown in sub-drawing (b) of FIG. 1 ( 114, 124) may also be performed. For example, the technical feature of transmitting the control signal by the transmitting STA is that the control signal generated by the processors 111 and 121 shown in sub-drawings (a) and (b) of FIG. It can be understood as a technical feature transmitted through the transceivers 113 and 123 shown in )/(b). Alternatively, the technical feature of transmitting the control signal by the transmitting STA is the technical feature of generating a control signal to be transmitted to the transceivers 113 and 123 in the processing chips 114 and 124 shown in sub-drawing (b) of FIG. can be understood

For example, a technical feature in which a receiving STA receives a control signal may be understood as a technical feature in which a control signal is received by the transceivers 113 and 123 shown in sub-drawing (a) of FIG. 1 . Alternatively, the technical feature of receiving the control signal by the receiving STA is that the control signal received by the transceivers 113 and 123 shown in sub-drawing (a) of FIG. 111, 121) can be understood as a technical feature obtained. Alternatively, the technical feature of receiving the control signal by the receiving STA is that the control signal received by the transceivers 113 and 123 shown in sub-drawing (b) of FIG. 1 is the processing chip shown in sub-drawing (b) of FIG. It can be understood as a technical feature obtained by (114, 124).

Referring to sub-drawing (b) of FIG. 1 , software codes 115 and 125 may be included in memories 112 and 122 . The software codes 115 and 125 may include instructions for controlling the operation of the processors 111 and 121 . Software code 115, 125 may be included in a variety of programming languages.

The processors 111 and 121 or processing chips 114 and 124 shown in FIG. 1 may include application-specific integrated circuits (ASICs), other chipsets, logic circuits, and/or data processing devices. The processor may be an application processor (AP). For example, the processors 111 and 121 or processing chips 114 and 124 shown in FIG. 1 may include a digital signal processor (DSP), a central processing unit (CPU), a graphics processing unit (GPU), and a modulator (Modem). and demodulator). For example, the processors 111 and 121 or the processing chips 114 and 124 shown in FIG. 1 include a SNAPDRAGONTM series processor manufactured by Qualcomm®, an EXYNOSTM series processor manufactured by Samsung®, and an Apple® manufactured processor. It may be an A series processor, a HELIOTM series processor manufactured by MediaTek®, an ATOMTM series processor manufactured by INTEL®, or a processor that enhances them.

In this specification, uplink may mean a link for communication from a non-AP STA to an AP STA, and an uplink PPDU/packet/signal may be transmitted through the uplink. In addition, in this specification, downlink may mean a link for communication from an AP STA to a non-AP STA, and a downlink PPDU/packet/signal may be transmitted through the downlink.

2 is a conceptual diagram showing the structure of a wireless LAN (WLAN).

The upper part of FIG. 2 shows the structure of an infrastructure basic service set (BSS) of Institute of Electrical and Electronic Engineers (IEEE) 802.11.

Referring to the upper part of FIG. 2 , the WLAN system may include one or more infrastructure BSSs 200 and 205 (hereinafter referred to as BSSs). The BSSs 200 and 205 are a set of APs and STAs such as an access point (AP) 225 and a station (STA 200-1) that can successfully synchronize and communicate with each other, and do not point to a specific area. The BSS 205 may include one or more STAs 205-1 and 205-2 capable of being coupled to one AP 230.

The BSS may include at least one STA, APs 225 and 230 providing a distribution service, and a distribution system (DS, 210) connecting a plurality of APs.

The distributed system 210 may implement an extended service set (ESS) 240, which is an extended service set, by connecting several BSSs 200 and 205. The ESS 240 may be used as a term indicating one network formed by connecting one or several APs through the distributed system 210 . APs included in one ESS 240 may have the same service set identification (SSID).

The portal 220 may serve as a bridge connecting a wireless LAN network (IEEE 802.11) and another network (eg, 802.X).

In the BSS shown at the top of FIG. 2, a network between APs 225 and 230 and a network between APs 225 and 230 and STAs 200-1, 205-1 and 205-2 may be implemented. However, it may be possible to perform communication by configuring a network even between STAs without the APs 225 and 230. A network in which communication is performed by configuring a network even between STAs without APs 225 and 230 is defined as an ad-hoc network or an independent basic service set (IBSS).

The lower part of FIG. 2 is a conceptual diagram showing IBSS.

Referring to the bottom of FIG. 2 , the IBSS is a BSS operating in an ad-hoc mode. Since the IBSS does not include an AP, there is no centralized management entity. That is, in IBSS, STAs 250-1, 250-2, 250-3, 255-4, and 255-5 are managed in a distributed manner. In IBSS, all STAs (250-1, 250-2, 250-3, 255-4, 255-5) can be made up of mobile STAs, and access to the distributed system is not allowed, so a self-contained network network).

3 is a diagram illustrating a general link setup process.

In the illustrated step S310, the STA may perform a network discovery operation. The network discovery operation may include a scanning operation of the STA. That is, in order for the STA to access the network, it needs to find a network in which it can participate. The STA must identify a compatible network before participating in a wireless network, and the process of identifying a network existing in a specific area is called scanning. Scanning schemes include active scanning and passive scanning.

FIG. 3 exemplarily illustrates a network discovery operation including an active scanning process. In active scanning, an STA performing scanning transmits a probe request frame to discover which APs exist around it while moving channels and waits for a response thereto. A responder transmits a probe response frame as a response to the probe request frame to the STA that has transmitted the probe request frame. Here, the responder may be an STA that last transmitted a beacon frame in the BSS of the channel being scanned. In the BSS, since the AP transmits the beacon frame, the AP becomes a responder. In the IBSS, the STAs in the IBSS rotate to transmit the beacon frame, so the responder is not constant. For example, an STA that transmits a probe request frame on channel 1 and receives a probe response frame on channel 1 stores BSS-related information included in the received probe response frame and transmits the probe request frame on the next channel (e.g., channel 2). channel), and scanning (ie, probe request/response transmission/reception on channel 2) can be performed in the same way.

Although not shown in the example of FIG. 3 , the scanning operation may be performed in a passive scanning manner. An STA performing scanning based on passive scanning may wait for a beacon frame while moving channels. A beacon frame is one of the management frames in IEEE 802.11, and is periodically transmitted to notify the existence of a wireless network and to allow an STA performing scanning to find a wireless network and participate in the wireless network. In the BSS, the AP serves to transmit beacon frames periodically, and in the IBSS, STAs within the IBSS rotate to transmit beacon frames. When an STA performing scanning receives a beacon frame, it stores information about the BSS included in the beacon frame and records beacon frame information in each channel while moving to another channel. The STA receiving the beacon frame may store BSS-related information included in the received beacon frame, move to the next channel, and perform scanning in the next channel in the same way.

The STA discovering the network may perform an authentication process through step SS320. This authentication process may be referred to as a first authentication process in order to be clearly distinguished from the security setup operation of step S340 to be described later. The authentication process of S320 may include a process in which the STA transmits an authentication request frame to the AP, and in response to this, the AP transmits an authentication response frame to the STA. An authentication frame used for authentication request/response corresponds to a management frame.

The authentication frame includes authentication algorithm number, authentication transaction sequence number, status code, challenge text, RSN (Robust Security Network), finite cyclic group Group), etc.

The STA may transmit an authentication request frame to the AP. The AP may determine whether to allow authentication of the corresponding STA based on information included in the received authentication request frame. The AP may provide the result of the authentication process to the STA through an authentication response frame.

The successfully authenticated STA may perform a connection process based on step S330. The association process includes a process in which the STA transmits an association request frame to the AP, and in response, the AP transmits an association response frame to the STA. For example, the connection request frame includes information related to various capabilities, beacon listen interval, service set identifier (SSID), supported rates, supported channels, RSN, mobility domain , supported operating classes, TIM broadcast request (Traffic Indication Map Broadcast request), interworking service capability, and the like. For example, an association response frame may include information related to various capabilities, a status code, an Association ID (AID), an assisted rate, an Enhanced Distributed Channel Access (EDCA) parameter set, a Received Channel Power Indicator (RCPI), and Received Signal to Noise (RSNI). indicator), mobility domain, timeout interval (association comeback time), overlapping BSS scan parameter, TIM broadcast response, QoS map, and the like.

After that, in step S340, the STA may perform a security setup process. The security setup process of step S340 may include, for example, a process of setting up a private key through 4-way handshaking through an Extensible Authentication Protocol over LAN (EAPOL) frame. .

As the demand for wireless data traffic rises, WiFi networks grow very quickly as they offer high throughput and are easy to deploy. Recently, channel state information (CSI) measured in a WiFi network is widely used for various sensing purposes. To better understand existing WiFi sensing technologies and future WiFi sensing trends, this paper comprehensively reviews the signal processing techniques, algorithms, applications, and performance results of WiFi sensing using CSI. Different WiFi sensing algorithms and signal processing techniques have their own strengths and limitations, making them suitable for different WiFi sensing applications. This specification classifies CSI-based WiFi sensing applications into three categories: sensing, recognition, and estimation, depending on whether the output is binary/multi-class classification or numerical. With the development and deployment of new WiFi technologies, there will be more WiFi sensing opportunities for targets to cross over from humans to environments, animals and objects.

This specification highlights three challenges in WiFi sensing: robustness and generalization, privacy and security, and coexistence of WiFi sensing and networking. In addition, this specification proposes three future WiFi sensing trends such as inter-layer network information integration, multi-device cooperation, and convergence of different sensors to enhance existing WiFi sensing capabilities and enable new WiFi sensing opportunities.

WiFi is growing very fast as wireless devices become more popular. One of the key technologies for the success of WiFi is Multiple-Input Multiple-Output (MIMO), which provides high throughput to meet the growing demands of wireless data traffic. Together with Orthogonal Frequency-Division Multiplexing (OFDM), MIMO provides channel state information (CSI) for each transmit/receive antenna pair at each carrier frequency. CSI measurement of recent WiFi systems is used for various sensing purposes. WiFi sensing reuses the infrastructure used for wireless communications, making it easy to deploy and low cost. And unlike sensor-based and video-based solutions, WiFi sensing does not interfere with lighting conditions.

CSI describes how a radio path propagates from a transmitter to a receiver at a specific carrier frequency along multiple paths. For WiFi systems with MIMO-OFDM, CSI is a 3D matrix of complex values representing the amplitude attenuation and phase shift of a multipath WiFi channel.

The time series of CSI measurements can be used for other wireless sensing applications by capturing how radio signals travel through nearby objects and people in the time, frequency and spatial domains. For example, CSI amplitude fluctuations in the time domain have different patterns depending on human, activity, gesture, etc., which can be used for human presence detection, fall detection, motion detection, activity recognition, gesture recognition, and human identification/authentication. have

The CSI phase shift in the spatial and frequency domain, i.e. the transmit/receive antenna and carrier frequency, is related to signal transmission delay and direction, which can be used for human positioning and tracking. The CSI phase shift in the time domain may have other dominant frequency components that can be used to estimate the respiratory rate. Various WiFi sensing applications have specific requirements for signal processing techniques and classification/estimation algorithms.

This specification proposes signal processing techniques, algorithms, applications, performance results, challenges, and future trends in WiFi sensing through CSI to enhance understanding of existing WiFi sensing technologies and gain insights into future WiFi sensing directions.

4 shows a procedure flow diagram of WiFi sensing.

A WiFi signal (eg, CSI measurement value) including a mathematical model, measurement procedure, actual WiFi model, basic processing principle, and experimental platform is input in the input terminal 410. Raw CSI measurements are fed to the signal processing module for noise reduction, signal conversion and/or signal extraction as indicated by the signal processing stage 420.

The pre-processed CSI traces are fed into modeling-based, learning-based or hybrid algorithms such as Algorithm stage 430 to obtain outputs for various WiFi sensing purposes. Depending on the output type, WiFi sensing can be classified into three categories. In the Application stage 440, the detection/recognition application tries to solve a binary/multi-class classification problem and the estimation application tries to get the quantity value of another task.

5 shows a general procedure flow diagram of human activity sensing via wireless signals.

Specifically, the sensing system can detect human activity based on different sensing methods (e.g., Received Signal Strength Indicator (RSSI), Channel State Information (CSI), Frequency Modulated Carrier Wave (FMCW), and Doppler shift)). The relevant signal changes are first extracted. Next, a series of signal preprocessing procedures (eg, filtering, denoising, and correction) are employed to mitigate the effects of interference, ambient noise, and system offset. Finally, unique features are extracted and fed into machine learning models to perform human activity detection and recognition.

That is, the human activity sensing procedure of FIG. 5 is as follows.

1) Measurements: Measuring RSSI, CSI, Doppler shift, etc. with input values

2) Derived Metrics with Human movements: Signal strength variations, Channel condition variations, Frequency shift associated with human body depth, Frequency shift associated with human moving speed

3) Signal Pre-processing: Noise reduction, Signal Time-Frequency Transform, Signal Extraction

4) Feature Extraction: User ID feature extraction using gait cycle, body speed, and human activity

5) Prediction via Machine/Deep learning: algorithm

6) Application: User identification prediction model Detection, Recognition, Estimation (Intrusion detection, Room occupancy monitoring, Daily activity recognition, Gesture recognition, Vital signs monitoring, User identification, Indoor localization & tracking)

1. Wireless Sensing, Wi-Fi, Machine Learning

<Background of invention>

The IoT future smart home market is changing from device connection-oriented to service-oriented, and as a result, the need for artificial intelligence device-based personalization and automation services is increasing. The development of wireless sensing-based technology, which is one of the elemental technologies for IoT service of artificial intelligence devices, is actively being developed. Research on user identification by learning the pattern of this signal is actively underway.

<Background art and problems>

In order to install Wireless Sensing-based User Identification technology in commercial products, pre-learning (training and distributing a model for predicting collected data in machine learning in advance (for example, pre-learning a model that predicts dogs and cats) It is difficult to deploy and predict new images that are not used for learning) Wireless Signal is a commercial product because it is not possible to create and pre-deploy a general model as the signal pattern varies depending on the user's movement even for the same user depending on the environment. For this, it is necessary to create a model through learning tailored to each environment, but prior learning using supervised learning used in existing research requires user participation for collection of learning data and labeling (matching the correct answer of the data). The practicality of the commercialization point of view is poor.

Therefore, this specification proposes a post-learning automation method for wireless sensing-based user identification.

When learning the wireless sensing signal pattern suitable for each environment, the personal identification information of the user device (Personal Electronic Device - PED) is used to enable post-learning through collection of correct answers (for example, labels) for learning. Learning methods for post-learning can be applied to several methods, such as unsupervised learning, supervised learning, semi-supervised learning, and unsupervised/supervised fusion learning.

Through this embodiment, it is possible to implement a system that learns and predicts a signal pattern suitable for a user's home environment, thereby creating a new paradigm of IoT future smart home devices such as artificial intelligence devices that identify people.

<Example of Wi-Fi CSI-based User Identification Study>

An example of research on learning/prediction using Wi-Fi CSI, wireless signal refinement, feature extraction, and machine learning is as follows.

1) Signal Pre-processing

-> CSI measurement collection - Based on 20MHz bandwidth, CSI measurement values of 30 to 52 subcarriers are collected as many as the number of TX/RX antennas.

-> Denoising - Noise is removed from the signal using algorithms such as PCA (Principal Component Analysis), phase unwrapping, and band-pass butterworth filter.

-> Conversion to Time-Frequency domain - Spectrogram generation using STFT (Shot-Time Fourier Transform) (see Fig. 6) -> The denoising waveform is mixed with reflections of human body parts, which can be distinguished by frequency .

6 shows a CSI spectrogram according to human walking.

Referring to FIG. 6 , torso reflection and leg reflection are shown in a CSI spectogram in the time/frequency domain. At this time, the CSI spectogram has a certain cycle time.

2) Feature Extraction

-> The process of extracting features for user identification learning and prediction

-> Utilize Gait Cycle Time, Movement (or Torso) Speed, Human Activity, etc.

-> Based on the theory that the gait cycle is unique to each person, it is used as a feature of User Identification

-> Example of body velocity estimation method: using percentile method used in Doppler Radar

-> Example of human activity estimation method: using time domain features (max, min, mean, skewness, kurtiosis, std), which are low level features of CSI, human movement and contour, frequency domain features (spcetrogram energy, percentile frequency component, spectrogram energy difference) to predict the movement speed of the trunk and legs, and express walking or stationary activities using these features.

3) Machine/Deep Learning based training and prediction

-> Learning and prediction through various machine/deep learning-based algorithms

-> Representative Algorithm

i) Supervised Learning: Using machine learning and deep learning algorithms such as decision tree-based machine learning classifier, SVM (Support Vector Machine), and Softmax classifier

i)-1 The prediction model is created only by supervised learning, and the unsupervised learning algorithm is used to construct the layer of the supervised learning model (some studies)

-> Learning method

i) Data is collected under specific environmental conditions for each person and training/evaluation data is selected at a specific ratio (eg, Training data : Evaluation data = 8:2) -> Holdout verification

ii) Training data is trained by manually mapping the correct answer (e.g. Label) for each person and using it as an input for a machine/deep learning model.

iii) Some studies perform auto feature extraction, clustering, etc. using unsupervised learning to increase the degree of freedom of the data collection environment, and then perform user identification using a supervised learning model (eg Softmax classifier)

Unsupervised learning is a learning method that only studies problems without teaching answers (labels). According to unsupervised learning, based on the relationship between variables, clustering (a representative example of unsupervised learning) is performed to find the correct answer (eg, YouTube group recommendation, animal classification).

On the other hand, supervised learning is a learning method that teaches and studies answers. Supervised learning is divided into regression and classification. Regression is a learning method that predicts outcomes within a continuous range of data (for example, guessing ages 0-100). Classification is a learning method that predicts an outcome within a range of discretely separated data (e.g. whether a tumor is malignant or benign).

In addition, quasi-supervised learning is a method of learning data with answers and data without answers at the same time, and is a learning method that studies numerous unanswered data without discarding them.

7 shows a deep learning architecture for user authentication.

The deep learning architecture of FIG. 7 is an example of performing auto feature extraction using an autoencoder for each hidden layer and using softmax classification for classification.

Referring to FIG. 7 , a supervised learning model configures each hidden layer, and an unsupervised learning model is used only for configuring the corresponding layer. Activity Separation, Activity Recognition, and User Authentication in FIG. 7 are all characteristics acquired through auto feature extraction.

2. Wireless Sensing, Wi-Fi, Channel State Information, Machine Learning

<Background of invention>

The IoT (Internet of Things) future smart home market is changing from device connection-oriented to service-oriented, and as a result, the need for artificial intelligence device-based personalization and automation services is increasing. Wireless Sensing-based technology development, which is one of the elemental technologies for IoT service of artificial intelligence devices, is being actively developed. Actively in progress.

<Prior art and problems>

The role of wireless sensing-based devices to transmit and measure wireless signals such as Wi-Fi CSI (Channel State Information) is determined.

When the transmitting device and the receiving device are far away, the receiving device collects radio signals that cannot be measured or uncertain, such as Wi-Fi CSI.

If the sensing coverage of the transmitting device and the receiving device are different and there are multiple transmitting/receiving devices, a limited coverage is created as the sensing coverage is applied only to the pair of the transmitting and receiving devices.

Since the Tx / Rx Role is limited, the shadow area of the Wireless Sensing Coverage occurs, and to improve this, a Role Switching Mechanism that negotiates and defines the Tx / Rx Device Role is needed.

Therefore, this specification proposes a Tx / Rx Role Switching technique for wireless sensing based coverage enhancement. Specifically, when there are several wireless sensing devices, Capabilities information can be exchanged through mutual negotiation. The wireless sensing device can set the transmission device and reception device through Capabilities information, and can set the order of transmission devices. Transmitting and receiving devices can perform Tx / Rx Role Switching based on the exchanged Capabilities information. Through the embodiment proposed in this specification, it is possible to use a wider coverage than the existing coverage when several devices coexist with signal patterns suitable for the user's home environment, and it is possible to implement a system that efficiently collects, learns, and predicts new It has the effect of creating paradigm IoT future smart home devices.

The existing operation method (existing protocol) based on Wireless Sensing is as follows. 1) The transmitting device transmits a measurable signal such as Wi-Fi CSI (Channel State Information). 2) The receiving device measures the CSI radio signal sent from the transmitting device. 3) Transmitting and receiving devices perform wireless signal pre-processing to refine the collected signals. 4) Transmitting and receiving devices perform a process of extracting features for learning and prediction (Feature Extraction). 5) Transmitting/receiving devices learn and evaluate data sets that have gone through Wireless Signal Pre-processing and Feature Extraction. - Supervised Learning, Unsupervised Learning

However, the existing wireless sensing-based operation method has the following problems.

Wireless Sensing-based devices have a predetermined role for transmitting and receiving wireless signals such as Wi-Fi CSI (Channel State Information). Depending on the device, the Tx Device has only the Tx Role that transmits radio signals, and the Rx Device has only the Rx Role that receives radio signals.

If the sensing coverage of the transmitting device and the receiving device are different and there are multiple transmitting/receiving devices, a limited coverage is created as the sensing coverage is applied only to the pair of the transmitting and receiving devices. Since the Tx / Rx Role is limited, the shadow area of the Wireless Sensing Coverage occurs, and to improve this, a Role Switching Mechanism that negotiates and defines the Tx / Rx Device Role is needed.

However, if the transmitting device and the receiving device are far away, the wireless signal may be weakened, making Wi-Fi CSI measurement impossible or an uncertain wireless signal may be collected.

8 shows an example of a problem according to the limited sensing coverage of a transmitting/receiving device.

Referring to FIG. 8, 1) Tx Device transmits a radio signal such as Wi-Fi CSI. 2) Rx Device 1 and Rx Device 2 measure the radio signal. 3) Rx Device 1 accurately receives a radio signal such as CSI and can accurately predict it, but Rx Device 2 cannot collect radio signals due to a long distance or collects radio signals uncertainly, making accurate prediction impossible.

9 shows another example of a problem according to the limited sensing coverage of a transmitting/receiving device.

Referring to FIG. 9, 1) Tx Device transmits a radio signal such as Wi-Fi CSI. 2) Rx Device 1 and Rx Device 2 measure the radio signal. 3) Rx Device 1 and Rx Device 2 accurately receive radio signals such as CSI and can accurately predict them, but a shadow area in which sensing is difficult occurs.

10 shows a block diagram of a Tx/Rx Role Switching function for wireless sensing based coverage enhancement.

Specifically, FIG. 10 shows a functional block diagram for Wireless Sensing Architecture, Negotiation, and Signal exchange. The functional unit shown in FIG. 10 can be defined as follows.

First, Wireless Sensing devices include Wireless PHY / MAC Driver (10), Device Discovery (20), Capabilities Negotiation (30), Wireless Sensing (40), Signal Pre-Processing (50), Feature Selection / Extraction (60), and Deep /Machine Learning (70) is included. All information can be transmitted and received between the wireless sensing devices based on the information exchange network unit 80.

The Device Discovery 20 plays a role in discovering peripheral devices. The Capabilities Negotiation (30) mutually exchanges Tx / Rx Role information, Wireless Sensing information, RSSI (Received Signal Strength Indicator), CSI Amplitude, location/distance information of devices, and device performance of discovered peripheral devices to obtain Tx / Rx It plays a role in setting the role and wireless sensing data transmission method. The Wireless Sensing 40 serves to transmit and collect wireless signals such as Wi-Fi CSI (Channel State Information). The Signal Pre-Processing 50 serves to perform CSI Measurement, Phase Offset Calibration, De-Noising, and the like. The Feature Selection / Extraction (60) serves to select and extract features for learning and prediction. The Deep/Machine Learning 70 serves to perform training and prediction through various machine/deep learning-based algorithms.

11 shows an example of a procedure in which a wireless sensing device performs Device Discovery and Capabilities Negotiation.

Referring to FIG. 11, each device detects other devices while transmitting and receiving a Device Discovery Request/Response message based on the Device Discovery 20 and the information exchange network unit 80.

In addition, each device performs Capabilities Negotiation while transmitting Capabilities Negotiation Request/Response/Confirm based on Capabilities Negotiation (30) and information exchange network unit (80).

12 shows an example of a wireless sensing procedure when there are one Tx Device, one Tx/Rx Device, and one Rx Device.

Referring to FIG. 12, each device may perform one or more of 1) Wireless Sensing, 2) Signal Pre-Processing, 3) Feature Selection / Extraction, and 4) Deep / Machine Learning according to their capabilities.

In FIG. 12A, the Tx Device transmits or receives Wireless Sensing Data to the Tx/Rx Device and the Rx Device based on the Wireless Sensing 40 and the information exchange network unit 80. At this time, the Tx / Rx Device plays the role of Rx. The Tx / Rx Device and Rx Device pre-process measured CSI Data based on Signal Pre-Processing (50), select and extract features for learning and prediction based on Feature Selection / Extraction (60), and deep / Based on Machine Learning (70) and information exchange network unit (80), it learns and predicts and shares the prediction result to the Tx Device.

Tx / Rx Device and Rx Device process Sensing Data and transmit Result Message to Tx Device, and Tx Device can transmit Confirm Message for role switch after receiving Result Message from each device (Confirm message can be sent to Rx role If the device of the device does not receive, another role switch method may be applied).

In FIG. 12B, if the role switch conditions such as receiving a Confirm Message or after a certain period of time determined in Capabilities Negotiation are satisfied, the next Tx Device (Tx / Rx Device in FIG. 12) switches to the Tx Role and sends sensing data to the Rx Device. can transmit After the Tx/Rx Device is switched to the Tx Role, the Tx/Rx Device transmits or receives wireless sensing data to the Rx Device based on the wireless sensing unit 40 and the information exchange network unit 80. The Rx Device preprocesses CSI Data measured based on Signal Pre-Processing (50), selects and extracts features for learning and prediction based on Feature Selection / Extraction (60), and Deep / Machine Learning (70 ) and the information exchange network unit 80, learns and predicts, and shares the prediction result to the Tx / Rx Device.

Devices that have finished sensing can either end sensing or rotate according to the number of times set in Capabilities Negotiation.

13 shows an example of a wireless sensing procedure when there are two Tx/Rx devices and one Rx device.

Referring to FIG. 13, each device may perform one or more of 1) Wireless Sensing, 2) Signal Pre-Processing, 3) Feature Selection / Extraction, and 4) Deep / Machine Learning according to their capabilities.

In FIG. 13a, Tx/Rx Device 2 and Rx Device may process Sensing Data, transmit a Result Message to Tx/Rx Device 1, receive a Result Message from each device, and transmit a Confirm Message for a role switch (Confirm message If the device of the Rx role does not receive, another role switch method may be applied).

In FIG. 13B, if the role switch conditions such as receiving a Confirm Message or after a certain period of time determined in Capabilities Negotiation are satisfied, Tx/Rx Deivce 1 waits for Sensing Data after Rx Role Switching and then waits for the next Tx Device (Tx/Rx in FIG. 13). Device 2) can transmit sensing data to Tx / Rx Device1 and Rx Device after switching to Tx Role. That is, in FIG. 13, the Tx/Rx Device 1 is switched from the Tx Role to the Rx Role, and the Tx/Rx Device 2 is switched from the Rx Role to the Tx Role.

Devices that have finished sensing can either end sensing or rotate according to the number of times set in Capabilities Negotiation.

14 shows an example of a wireless sensing procedure when there are three Tx / Rx devices.

Referring to FIG. 14, each device may perform one or more of 1) Wireless Sensing, 2) Signal Pre-Processing, 3) Feature Selection / Extraction, and 4) Deep / Machine Learning according to their capabilities.

In FIG. 14a, Tx/Rx Device 2 and Tx/Rx Device 3 transmit Sensing Data and Result Message to Tx/Rx Device 1, receive the Result Message from each device, and then transmit a Confirm Message for Role Switch (Confirm If the message is not received by the Rx Role device, other role switch methods may be applied).

In FIG. 14B, if the role switch conditions such as receiving a Confirm Message or after a certain period of time determined in Capabilities Negotiation are satisfied, Tx/Rx Deivce 1 waits for sensing data after Rx Role Switching and then waits for the next Tx Device (Tx/Rx Device in FIG. 14). 2) can transmit sensing data to Tx / Rx Device 1 and Tx / Rx Device 3 after switching to Tx Role.

The next Tx Device (Tx/Rx Device 2 in FIG. 14) transmits Sensing Data, and Tx/Rx Device 1 and Tx/Rx Device 3 receive the Sensing Data and proceed in the same manner as above.

Then, the Tx Device (Tx/Rx Device 3 in FIG. 14) transmits Sensing Data, and the Tx/Rx Device 2 and Tx/Rx Device 3 receive the Sensing Data and proceed in the same manner as above. That is, in FIG. 14B, the Tx/Rx Device 1 switches from the Tx Role to the Rx Role, in FIG. 14B, the Tx/Rx Device 2 switches from the Rx Role to the Tx Role, and in FIG. 14C, the Tx/Rx Device 3 It is switched from Rx Role to Tx Role.

Devices that have finished sensing can either end sensing or rotate according to the number of times set in Capabilities Negotiation.

Hereinafter, Capabilities Negotiation between Wireless Sensing Devices will be described. Wireless Sensing Devices exchange Capabilities Information, which includes Tx / Rx Role information (information on whether the device supports only Tx, only Rx, or both Tx / Rx), Wireless Sensing information (depending on the device) Data processing capability information), RSSI, CSI Amplitude, location/distance information of devices, and device performance. In this case, RSSI is RSSI information between devices, CSI Amplitude is CSI Amplitude information between devices, and location/distance information of devices is location/distance information between devices. Distance information utilizes Wi-Fi location or an algorithm that can measure distance. In the case of location information, the location stored in the device or the location stored in the cloud is used. Device performance means the performance of devices such as CPU, RAM, and memory.

Next, Role Switching Operation between Wireless Sensing Devices will be described. Role Switching Operation can be called Tx / Rx Role Switching mechanism.

First, the transmission device order is determined. Devices with Device Discovery and Capabilities Negotiation can set roles related to Tx, Rx, and Tx/Rx roles through role negotiation within the group.

A device supporting only Tx may be selected as a transmitting device before a device supporting both Tx and Rx. In the case of a device that supports both Tx and Rx, the optimal device can be set as the first transmission device by checking RSSI, CSI Amplitude, location/distance information of devices, device performance, etc. For example, a device closest among devices in a group may be selected as the first transmission device. Alternatively, a device with the best performance may be selected as the first transmission device.

In the case of a device that supports only Rx, it cannot be set as a transmission device.

Role switching mechanism According to the role switching mechanism in information exchange, each device can prepare a role switching mechanism operation. Tx / Rx Role can prepare scheduling according to the order of Tx switching. Depending on the Capabilities Negotiation information, it is possible to send and receive information on the period and number of sensing data and role switching rotation, and devices can receive sensing data accordingly and execute role switching rotation.

15 shows an example of a wireless sensing scenario when there are one Tx Device, one Tx/Rx Device, and one Rx Device.

Referring to FIG. 15, each device discovers a nearby device (20) and performs Capabilities Negotiation (30) with the detected devices. During the Capabilities Negotiation, 1) Tx, Tx/Rx, Rx Role Information and 2) Wireless Sensing Information can be transmitted and received.

The Tx Device transmits a measurable signal (Wireless Sensing (40)) including CSI Information to the Tx / Rx Device and Rx Device that have completed Capabilities Negotiation (30).

The measurable signal including CSI information is measured in Tx / Rx Device and Rx Device (Wireless Sensing (40)).

The received Tx / Rx Device and Rx Device collect CSI information and deliver data according to the Negotiation result (Signal Pre-Processing (50), Feature Selection / Extraction (60), and Deep / Machine Learning (80)). The learning and prediction results are shared to the Tx Device (first Wireless Sensing Path).

Tx Device transmits Confirm Message to Tx / Rx Device and Rx Device.

After receiving the Confirm Message from the Tx/Rx Device, the Tx/Rx Device knows that the Sensing is finished and the Role Switched, and transmits a measurable signal including CSI Information to the Rx Device (Wireless Sensing (40)).

The measurable signal containing the CSI information is measured by the Rx Device (Wireless Sensing (40)).

The Rx Device collects CSI Information and transmits the learning and prediction results to the Tx through data transfer (Signal Pre-Processing (50), Feature Selection / Extraction (60) and Deep / Machine Learning (80)) according to the Negotiation result. / Share to Rx Device (Second Wireless Sensing Path).

16 shows an example of a wireless sensing scenario when there are Tx/Rx Devices 1 and 2 and Rx Device 1.

Referring to FIG. 16, each device discovers a nearby device (20) and performs Capabilities Negotiation (30) with the detected devices. During the Capabilities Negotiation, 1) Tx, Tx/Rx, Rx Role Information and 2) Wireless Sensing Information can be transmitted and received.

Tx/Rx Device 1 transmits a measurable signal including CSI information to Tx/Rx Device 2 and Rx Device that have completed Capabilities Negotiation (30) (Wireless Sensing (40)).

The measurable signal including CSI information is measured in Tx / Rx Device 2 and Rx Device (Wireless Sensing (40)).

The received Tx / Rx Device 2 and Rx Device collect CSI information and deliver data (Signal Pre-Processing (50), Feature Selection / Extraction (60) and Deep / Machine Learning (80)) according to the Negotiation result. Through this, the learning and prediction results are shared with Tx / Rx Device 1 (first Wireless Sensing Path).

Tx/Rx Device 1 sends Confirm Message to Tx/Rx Device 2 and Rx Device.

In Tx/Rx Device 2, after receiving a Confirm Message from Tx/Rx Device 1, Tx/Rx Deivce 1 knows that sensing is finished and that it has switched roles, and Tx/Rx Deivce 1 and Rx Device measure with CSI Information included. Transmit possible signals (Wireless Sensing (40)). That is, Tx/Rx Deivce 1 is switched from Tx Role to Rx Role, and Tx/Rx Deivce 2 is switched from Rx Role to Tx Role.

The measurable signal including CSI information is measured (Wireless Sensing (40)) in Tx / Rx Deivce 1 and Rx Device.

Tx / Rx Deivce 1 and Rx Device collect CSI information and learn through data delivery (Signal Pre-Processing (50), Feature Selection / Extraction (60), and Deep / Machine Learning (80)) according to the Negotiation result. And the prediction result is shared with the Tx/Rx Device 2 (second Wireless Sensing Path).

17 shows an example of a wireless sensing scenario when there are Tx / Rx Devices 1 to 3.

Referring to FIG. 17, each device discovers a nearby device (20) and performs Capabilities Negotiation (30) with the detected devices. During the Capabilities Negotiation, 1) Tx, Tx/Rx, Rx Role Information and 2) Wireless Sensing Information can be transmitted and received.

Tx/Rx Device 1 transmits a measurable signal including CSI information to Tx/Rx Device 2 and Tx/Rx Device 3 that have completed Capabilities Negotiation (30) (Wireless Sensing (40)).

The measurable signal including CSI information is measured in Tx / Rx Device 2 and Tx / Rx Device 3 (Wireless Sensing (40)).

The received Tx / Rx Device 2 and Tx / Rx Device 3 collect CSI information and deliver data according to the result of negotiation (Signal Pre-Processing (50), Feature Selection / Extraction (60)) and Deep / Machine Learning (80 )) to share learning and prediction results to Tx / Rx Device 1 (first Wireless Sensing Path).

Tx/Rx Device 1 transmits a Confirm Message to Tx/Rx Device 2 and Tx/Rx Device 3, and then performs Role Switching to Rx Role.

After Tx/Rx Device 2 receives the Confirm Message from Tx/Rx Device 1, it recognizes that Tx/Rx Device 1 has finished sensing and has switched roles, and sends CSI information to Tx/Rx Device 1 and Tx/Rx Device 3. The included measurable signal is transmitted (Wireless Sensing (40)). That is, Tx/Rx Deivce 1 is switched from Tx Role to Rx Role, and Tx/Rx Deivce 2 is switched from Rx Role to Tx Role.

Measure the measurable signal including CSI information in Tx / Rx Device 1 and Tx / Rx Device 3 (Wireless Sensing (40)).

Tx / Rx Deivce 1 and Tx / Rx Device 3 collect CSI information and deliver data according to the result of negotiation (Signal Pre-Processing (50), Feature Selection / Extraction (60) and Deep / Machine Learning (80)) The learning and prediction results are shared with the Tx/Rx Device 2 through (second Wireless Sensing Path).

Tx/Rx Device 2 transmits a Confirm Message to Tx/Rx Device 1 and Tx/Rx Device 3, and then performs role switching to the Rx role.

After Tx/Rx Device 3 receives the Confirm Message from Tx/Rx Device 1, it knows that Tx/Rx Device 2 has finished sensing and has switched roles, and sends CSI information to Tx/Rx Device 1 and Tx/Rx Device 2. The included measurable signal is transmitted (Wireless Sensing (40)). That is, Tx/Rx Deivce 2 is switched from Tx Role to Rx Role, and Tx/Rx Deivce 3 is switched from Rx Role to Tx Role.

The measurable signal including CSI information is measured in Tx / Rx Device 1 and Tx / Rx Device 2 (Wireless Sensing (40)).

Tx / Rx Device 1 and Tx / Rx Device 2 collect CSI information and deliver data (Signal Pre-Processing (50), Feature Selection / Extraction (60)) and Deep / Machine Learning (80) according to the Negotiation result. Through this, the learning and prediction results are shared with Tx / Rx Device 3 (third Wireless Sensing Path).

Hereinafter, the above-described embodiment will be described with reference to FIGS. 1 to 17 .

18 is a flowchart illustrating a procedure of performing wireless sensing by switching Tx/Rx roles according to the present embodiment.

Conventionally, since a wireless device has a fixed role in transmitting and measuring a wireless signal, there is a problem in that only limited coverage is generated by applying a sensing area only to a pair of transmitting and receiving devices. This embodiment proposes a method in which a wireless device based on wireless sensing expands a sensing area by switching Tx/Rx roles based on exchanged capabilities information.

In step S1810, the first wireless device performs capabilities negotiation with the second and third wireless devices. The first wireless device performs device discovery before performing the capability negotiation, and the second and third wireless devices are discovered or detected based on the device discovery.

In step S1820, the first wireless device obtains capability information based on the capability negotiation. The first wireless device may acquire the capability information while exchanging capability negotiation request/response messages with the second and third wireless devices.

The capability information may include transmission or reception role information of the wireless device, wireless sensing information, RSSI (Received Signal Strength Indicator), CSI amplitude (Channel State Information Amplitude), location and distance information of the wireless device, or performance information of the wireless device. can The transmission or reception role information of the wireless device may indicate that the first wireless device supports both transmission and reception, the second wireless device supports only transmission, and the third wireless device supports only reception. .

In step S1830, the first wireless device determines a role switching mechanism of the first to third wireless devices based on the capability information.

Based on the role switching mechanism, an order of transmission roles of wireless devices, a scheduling method according to the switching order, and a rotation period and number of role switching may be determined.

Based on the order of transmission roles of the wireless devices, the second wireless device is set as the first transmitting device, and the first wireless device may perform a receiving role. The second wireless device may be a device closest to an object in the device group or a device with the best performance. This may be a basis for setting the second wireless device as the first transmission device.

The first wireless device may receive first sensing data from the second wireless device. The first wireless device may transmit a result of the first sensing data to the second wireless device. The first wireless device may receive a first message confirming a result of the first sensing data from the second wireless device.

The first wireless device may pre-process the measured first CSI data. The measured first CSI data may be included in the first sensing data. The first wireless device may select and extract features of the preprocessed first CSI data. The first wireless device may obtain a predicted result by performing learning and prediction on the selected and extracted features.

The first wireless device may transmit the predicted result to the second wireless device. The first wireless device may receive a second message confirming the predicted result from the second wireless device. The first wireless device may be switched to perform a transmitting role (receiving role -> transmitting role) when the second message is received or after a time specified in the scheduling method according to the switching order. From this time, the first wireless device can perform a transmission role of transmitting a radio signal (or sensing data) to the third wireless device.

The first wireless device may transmit second sensing data to the third wireless device. The first wireless device may receive a result of the second sensing data from the third wireless device. The first wireless device may transmit a third message confirming a result of the second sensing data to the third wireless device.

The second sensing data may include measured second CSI data, and the measured second CSI data may be preprocessed, and a predicted result may be obtained by performing learning and prediction after selecting and extracting a feature. The pre-processing, selection and extraction, learning and prediction procedures may all be performed by the third wireless device.

The first wireless device may receive the predicted result from the third wireless device. The first wireless device may transmit a fourth message confirming the predicted result to the third wireless device. The wireless device may identify the user, the user's location, or the user's gesture based on the predicted result.

In this embodiment, the result learned and predicted may be obtained based on machine learning or deep learning as a pre-learning model.

Through this embodiment, it is possible to solve the problem that the sensing sound area occurs according to the location or arrangement of the wireless device, and when several wireless devices coexist with a signal pattern suitable for the user's home environment, a wider coverage than the existing coverage can be used. In addition, it is possible to implement a system that efficiently collects, learns, and predicts, which has the effect of creating a new paradigm of IoT future smart home devices.

3. Device Configuration

19 shows a modified example of the transmitter and/or receiver of the present specification.

Each device/STA in the sub-drawings (a)/(b) of FIG. 1 may be modified as shown in FIG. 19 . The transceiver 630 of FIG. 19 may be the same as the transceivers 113 and 123 of FIG. 1 . The transceiver 630 of FIG. 19 may include a receiver and a transmitter.

The processor 610 of FIG. 19 may be the same as the processors 111 and 121 of FIG. 1 . Alternatively, the processor 610 of FIG. 19 may be the same as the processing chips 114 and 124 of FIG. 1 .

The memory 150 of FIG. 19 may be the same as the memories 112 and 122 of FIG. 1 . Alternatively, the memory 150 of FIG. 19 may be a separate external memory different from the memories 112 and 122 of FIG. 1 .

Referring to FIG. 19 , the power management module 611 manages power to the processor 610 and/or the transceiver 630 . The battery 612 supplies power to the power management module 611 . The display 613 outputs the result processed by the processor 610 . Keypad 614 receives input to be used by processor 610 . A keypad 614 may be displayed on the display 613 . The SIM card 615 may be an integrated circuit used to securely store international mobile subscriber identities (IMSIs) used to identify and authenticate subscribers in mobile phone devices such as mobile phones and computers, and keys associated therewith. .

Referring to FIG. 19 , the speaker 640 may output sound-related results processed by the processor 610 . The microphone 641 may receive sound-related input to be used by the processor 610 .

The technical features of the present specification described above may be applied to various devices and methods. For example, the technical features of the present specification described above may be performed/supported through the device of FIGS. 1 and/or 19 . For example, the technical features of the present specification described above may be applied only to a part of FIGS. 1 and/or 19 . For example, the technical features of the present specification described above are implemented based on the processing chips 114 and 124 of FIG. 1, or implemented based on the processors 111 and 121 and the memories 112 and 122 of FIG. , may be implemented based on the processor 610 and the memory 620 of FIG. 19 . For example, the device of the present specification is a wireless device based on wireless sensing, and the device includes a memory and a processor operably coupled to the memory, the processor including second and third wireless devices and conduct capabilities negotiation; obtain capability information based on the capability negotiation; and determining a role switching mechanism between the first wireless device and the second and third wireless devices based on the capability information.

Technical features of the present specification may be implemented based on a computer readable medium (CRM). For example, the CRM proposed by this specification is at least one computer readable medium containing instructions based on being executed by at least one processor.

The CRM performs capabilities negotiation with second and third wireless devices; obtaining capability information based on the capability negotiation; and determining a role switching mechanism of the first wireless device and the second and third wireless devices based on the capability information. Instructions stored in the CRM of the present specification may be executed by at least one processor. At least one processor related to the CRM of the present specification may be the processors 111 and 121 or the processing chips 114 and 124 of FIG. 1 or the processor 610 of FIG. 19 . Meanwhile, the CRM of this specification may be the memories 112 and 122 of FIG. 1, the memory 620 of FIG. 19, or a separate external memory/storage medium/disk.

The technical features of the present specification described above are applicable to various applications or business models. For example, the technical features described above may be applied to wireless communication in a device supporting artificial intelligence (AI).

Artificial intelligence refers to the field of studying artificial intelligence or methodology to create it, and machine learning (Machine Learning) refers to the field of defining various problems dealt with in the field of artificial intelligence and studying methodologies to solve them. do. Machine learning is also defined as an algorithm that improves the performance of a certain task through constant experience.

An Artificial Neural Network (ANN) is a model used in machine learning, and may refer to an overall model that has problem-solving capabilities and is composed of artificial neurons (nodes) that form a network by combining synapses. An artificial neural network can be defined by a connection pattern between neurons in different layers, a learning process for updating model parameters, and an activation function for generating output values.

An artificial neural network may include an input layer, an output layer, and optionally one or more hidden layers. Each layer may include one or more neurons, and the artificial neural network may include neurons and synapses connecting the neurons. In an artificial neural network, each neuron may output a function value of an activation function for input signals, weights, and biases input through a synapse.

Model parameters refer to parameters determined through learning, and include weights of synaptic connections and biases of neurons. In addition, hyperparameters mean parameters that must be set before learning in a machine learning algorithm, and include a learning rate, number of iterations, mini-batch size, initialization function, and the like.

The purpose of learning an artificial neural network can be seen as determining model parameters that minimize the loss function. The loss function may be used as an index for determining optimal model parameters in the learning process of an artificial neural network.

Machine learning can be classified into supervised learning, unsupervised learning, reinforcement learning, and semi-supervised learning according to the learning method.

Supervised learning refers to a method of training an artificial neural network given a label for training data, and a label is the correct answer (or result value) that the artificial neural network must infer when learning data is input to the artificial neural network. can mean Unsupervised learning may refer to a method of training an artificial neural network in a state in which a label for training data is not given. Reinforcement learning may refer to a learning method in which an agent defined in an environment learns to select an action or action sequence that maximizes a cumulative reward in each state.

Among artificial neural networks, machine learning implemented as a deep neural network (DNN) including a plurality of hidden layers is also called deep learning, and deep learning is a part of machine learning. Hereinafter, machine learning is used to include deep learning.

In addition, the technical features described above can be applied to wireless communication of robots.

A robot may refer to a machine that automatically processes or operates a given task based on its own abilities. In particular, a robot having a function of recognizing an environment and performing an operation based on self-determination may be referred to as an intelligent robot.

Robots can be classified into industrial, medical, household, military, etc. according to the purpose or field of use. The robot may perform various physical operations such as moving a robot joint by having a driving unit including an actuator or a motor. In addition, the movable robot includes wheels, brakes, propellers, and the like in the driving unit, and can run on the ground or fly in the air through the driving unit.

In addition, the above-described technical features may be applied to devices supporting augmented reality.

Extended reality is a generic term for virtual reality (VR), augmented reality (AR), and mixed reality (MR). VR technology provides only CG images of objects or backgrounds in the real world, AR technology provides CG images created virtually on top of images of real objects, and MR technology provides a computer that mixes and combines virtual objects in the real world. It is a graphic technique.

MR technology is similar to AR technology in that it shows real and virtual objects together. However, there is a difference in that virtual objects are used to supplement real objects in AR technology, whereas virtual objects and real objects are used with equal characteristics in MR technology.

XR technology can be applied to HMD (Head-Mount Display), HUD (Head-Up Display), mobile phones, tablet PCs, laptops, desktops, TVs, digital signage, etc. can be called

The claims set forth herein can be combined in a variety of ways. For example, the technical features of the method claims of this specification may be combined to be implemented as a device, and the technical features of the device claims of this specification may be combined to be implemented as a method. In addition, the technical features of the method claims of the present specification and the technical features of the device claims may be combined to be implemented as a device, and the technical features of the method claims of the present specification and the technical features of the device claims may be combined to be implemented as a method.

Claims (20)

In a wireless LAN system based on wireless sensing,
performing, by a first wireless device, capabilities negotiation with second and third wireless devices;
obtaining, by the first wireless device, capability information based on the capability negotiation; and
Determining, by the first wireless device, a role switching mechanism of the first to third wireless devices based on the capability information
method.
According to claim 1,
The capability information includes transmission or reception role information of the wireless device, wireless sensing information, RSSI (Received Signal Strength Indicator), CSI amplitude (Channel State Information Amplitude), location and distance information of the wireless device, or performance information of the wireless device, ,
The transmission or reception role information of the wireless device indicates that the first wireless device supports both transmission and reception, the second wireless device supports only transmission, and the third wireless device supports only reception.
method.
According to claim 1,
Based on the role switching mechanism, the order of the transmission role of the wireless device, the scheduling method according to the switching order, and the rotation period and number of role switching are determined,
Based on the order of transmission roles of the wireless devices, the second wireless device is set as the first transmitting device, the first wireless device performs a receiving role, and the second wireless device is an object in the device group. The device closest to or with the best performance
method.
According to claim 3,
receiving, by the first wireless device, first sensing data from the second wireless device;
transmitting, by the first wireless device, a result of the first sensing data to the second wireless device; and
Receiving, by the first wireless device, a first message confirming a result of the first sensing data from the second wireless device
method.
According to claim 4,
Preprocessing, by the first wireless device, the measured first CSI data;
Selecting and extracting, by the first wireless device, a feature of the preprocessed first CSI data; and
Further comprising, by the first wireless device, obtaining a predicted result by performing learning and prediction on the selected and extracted feature,
The measured first CSI data is included in the first sensing data
method.
According to claim 5,
transmitting, by the first wireless device, the predicted result to the second wireless device; and
Receiving, by the first wireless device, a second message confirming the predicted result from the second wireless device;
The first wireless device is switched to perform a transmission role after the second message is received or after a time specified in a scheduling method according to the switching order
method.
According to claim 6,
transmitting, by the first wireless device, second sensing data to the third wireless device;
receiving, by the first wireless device, a result of the second sensing data from the third wireless device; and
Transmitting, by the first wireless device, a third message confirming a result of the second sensing data to the third wireless device
method.
According to claim 7,
The second sensing data includes measured second CSI data, and the measured second CSI data is preprocessed, and after selecting and extracting features, learning and prediction are performed to obtain a predicted result.
method.
According to claim 8,
receiving, by the first wireless device, the predicted result from the third wireless device; and
Transmitting, by the first wireless device, a fourth message confirming the predicted result to the third wireless device
method.
In a first wireless device in a wireless LAN system based on wireless sensing,
Memory;
transceiver; and
a processor operably coupled with the memory and the transceiver, the processor comprising:
performing capabilities negotiation with the second and third wireless devices;
obtain capability information based on the capability negotiation; and
Determining a role switching mechanism of the first to third wireless devices based on the capability information
First wireless device.
According to claim 10,
The capability information includes transmission or reception role information of the wireless device, wireless sensing information, RSSI (Received Signal Strength Indicator), CSI amplitude (Channel State Information Amplitude), location and distance information of the wireless device, or performance information of the wireless device, ,
The transmission or reception role information of the wireless device indicates that the first wireless device supports both transmission and reception, the second wireless device supports only transmission, and the third wireless device supports only reception.
First wireless device.
According to claim 10,
Based on the role switching mechanism, the order of the transmission role of the wireless device, the scheduling method according to the switching order, and the rotation period and number of role switching are determined,
Based on the order of transmission roles of the wireless devices, the second wireless device is set as the first transmitting device, the first wireless device performs a receiving role, and the second wireless device is an object in the device group. The device closest to or with the best performance
First wireless device.
According to claim 12,
The processor:
Receiving first sensing data from the second wireless device;
Transmitting a result of the first sensing data to the second wireless device; and
Receiving a first message confirming a result of the first sensing data from the second wireless device
First wireless device.
According to claim 13,
The processor:
pre-processing the measured first CSI data;
selecting and extracting features for the preprocessed first CSI data; and
Obtain a predicted result by performing learning and prediction on the selected and extracted features,
The measured first CSI data is included in the first sensing data
First wireless device.
According to claim 14,
The processor:
transmitting the predicted result to the second wireless device; and
Receiving a second message confirming the predicted result from the second wireless device,
The first wireless device is switched to perform a transmission role after the second message is received or a time specified in the scheduling method according to the switching order
First wireless device.
According to claim 15,
The processor:
transmit second sensing data to the third wireless device;
receiving a result of the second sensing data from the third wireless device; and
Transmitting a third message confirming a result of the second sensing data to the third wireless device
First wireless device.
According to claim 16,
The second sensing data includes measured second CSI data, and the measured second CSI data is preprocessed, and after selecting and extracting features, learning and prediction are performed to obtain a predicted result.
First wireless device.
According to claim 17,
The processor:
receiving the predicted result from the third wireless device; and
Transmitting a fourth message confirming the predicted result to the third wireless device
First wireless device.
In at least one computer readable medium containing instructions based on being executed by at least one processor,
performing capabilities negotiation with second and third wireless devices;
obtaining capability information based on the capability negotiation; and
Determining a role switching mechanism of the first wireless device and the second and third wireless devices based on the capability information
recording medium.
In a device in a wireless LAN system based on wireless sensing,
Memory; and
a processor operatively coupled to the memory, the processor comprising:
performing capabilities negotiation with the second and third wireless devices;
obtain capability information based on the capability negotiation; and
Determining a role switching mechanism of the first wireless device and the second and third wireless devices based on the capability information
Device.

Images