KR102493471B1 - Method for tracking user’s angular position of smart electric fans by leveraging smartphones, recording medium and device for performing the method - Google Patents

Abstract

A method of automatically tracking a user's direction of a smart fan using a smart phone includes, when the smart fan is operated, operating in a rotation mode for a predetermined time while changing a wind direction; Receiving a sound signal emitted by the wind using a speaker and a microphone built into a smart phone near the user; detecting deflection caused by wind by analyzing propagation characteristics of the received sound signal; estimating a user's angular position by detecting a wind blowing time when wind refraction is detected; and setting the angular position of the user estimated at the time of blowing of the smart fan to the wind direction of the smart fan. Accordingly, the user's location can be automatically tracked without a user's manipulation or separate hardware.

Description

Method for automatically tracking the user's direction of a smart electric fan using a smartphone, recording medium and device for performing the same

The present invention relates to a method for automatically tracking a user's direction of a smart electric fan using a smart phone, a recording medium and an apparatus for performing the same, and more particularly, by cooperatively utilizing a smart phone without installing a separate sensor in the electric fan It relates to mobile computing and IoT technology that automatically changes to the direction of the wind after recognizing the user's direction.

Electric fans, one of the most used cooling devices, are getting smarter thanks to advances in IoT (Internet of Things) technology. A recent trend in electric fans is to support smart functions based on an internet connection, such as remote control.

For example, imagine a person working out at the gym. After training on a treadmill, if they want to cool off in a chair, they can remotely control a nearby fan. However, in this case, it is inconvenient for the user because control is required whenever moving to another location.

A key technology for developing smart fans that automatically change the direction of airflow is tracking the user's angular position. There have been many attempts to identify the angular position (or location) in the fields of mobile computing and IoT.

The prior art is not perfect for implementing a smart fan, mainly because it requires special hardware separately, such as a light sensor, an infrared sensor, and a speaker/microphone array that are not usually built-in. Other studies have relaxed the hardware requirements by using easily accessible devices such as “smartphones” and “WiFi” access points.

However, in places where fans are commonly used (e.g., a home where only a single access point is available), accuracy is poor and a user's specific action (e.g., training data collection) is required.

KR 10-1686838 B1 KR 10-2020-0080392 A KR 10-2151105 B1

Accordingly, the technical problem of the present invention is conceived in this respect, and an object of the present invention is to provide a method for automatically tracking a user's direction of a smart electric fan using a smart phone.

Another object of the present invention is to provide a recording medium on which a computer program for performing a method for automatically tracking a user's direction of a smart electric fan using the smart phone is recorded.

Another object of the present invention is to provide a device for performing a method for automatically tracking a user's direction of a smart electric fan using the smart phone.

A method for automatically tracking a user's direction of a smart fan using a smart phone according to an embodiment for realizing the object of the present invention described above includes, when the smart fan is operated, operating in a rotation mode for a predetermined time while changing a wind direction; Receiving a sound signal emitted by the wind using a speaker and a microphone built into a smart phone near the user; detecting deflection caused by wind by analyzing propagation characteristics of the received sound signal; estimating a user's angular position by detecting a wind blowing time when wind refraction is detected; and setting the angular position of the user estimated at the time of blowing of the smart fan to the wind direction of the smart fan.

In an embodiment of the present invention, the method for automatically tracking the user's direction of the smart electric fan using the smart phone may further include re-estimating the user's location by automatically repeating all steps when a change in the user's location is detected. can

In an embodiment of the present invention, the step of re-estimating the user's location may be determined as a change in the user's location when the acceleration is greater than or equal to a threshold value for a preset time period.

In an embodiment of the present invention, the step of operating in rotation mode for a predetermined time while changing the wind direction is repeated for at least one rotation period, and has a frequency of 17 KHz or more and 24 KHz or less during each rotation period and has a length of 40 ms. A chirp signal can be transmitted every 70 ms.

In an embodiment of the present invention, the step of receiving the sound signal emitted by the wind includes dividing the received sound signal into subframes; extracting propagation uniqueness and characteristics of low-frequency energy in each subframe; and combining the propagation characteristics and the characteristics of the low-frequency energy with weight.

In an embodiment of the present invention, the step of estimating the user's angular position may be calculated using a rotation parameter of the smart electric fan and a wind blowing time.

A computer program for performing a method for automatically tracking a user's direction of a smart electric fan using the smart phone is recorded in a computer-readable storage medium according to an embodiment for realizing another object of the present invention described above.

An apparatus for automatically tracking the user's direction of a smart fan using a smart phone according to an embodiment for realizing another object of the present invention described above, when the smart fan is operated, operates in a rotation mode for a certain period of time while changing the wind direction. Oscillator unit; a scanning unit for receiving a sound signal emitted by the wind using a speaker and a microphone built into a smart phone near the user; an analyzer configured to detect refraction caused by wind by analyzing propagation characteristics of the received sound signal; an angle estimator for estimating an angular position of the user by detecting a wind blowing time when detecting wind refraction; and a wind direction setting unit configured to set the angular position of the user estimated at the time of blowing of the smart fan as the wind direction of the smart fan.

In an embodiment of the present invention, the device for automatically tracking the user's direction of a smart electric fan using a smart phone may further include a re-estimation unit that re-estimates the user's location when a change in the user's location is detected.

In an embodiment of the present invention, the re-estimation unit may determine that the user's position change when the acceleration is greater than or equal to a threshold value for a preset time period.

In an embodiment of the present invention, the scanning unit divides the received sound signal into sub-frames, and extracts propagation characteristics and low-frequency energy characteristics from each sub-frame and performs weight combining.

In an embodiment of the present invention, the angle estimator may calculate using a rotation parameter of the smart electric fan and a wind blowing time.

According to the method of automatically tracking the user's direction of a smart fan using a smart phone, we propose 　WindTrack, which can accurately and conveniently track the user's angular position by increasing the possibility of distribution, usefulness, and accuracy.

Specifically, the WindTrack proposed in the present invention can be easily deployed to an electric fan by minimally modifying the hardware and software of the fan, and does not depend on a pre-installed infrastructure.

In addition, it is easy to set and use because the angular position can be specified without a user's manipulation. Furthermore, the WindTrack according to the present invention accurately measures the user's angular position in the fan in order to accurately deliver cool air to the target, and can operate regardless of the environment.

1 is a conceptual diagram showing usage scenarios and GUIs of WindTrack according to the present invention.
2 is a block diagram of a device for automatically tracking a user's direction of a smart electric fan using a smart phone according to an embodiment of the present invention.
3 is a diagram for explaining a phenomenon in which a sound wave is refracted due to a difference in sound speed.
4 is a diagram showing the frequency spectrum of a sound signal measured when there is wind and when there is no wind.
5 is a diagram showing spatial variability of wind speed when a fan blows forward.
6 is a diagram showing a correlation between frequency spectra of sound signals detected according to a wind direction (θ ^W ) of a fan.
7 is a schematic diagram showing the system architecture of WindTrack proposed in the present invention.
8 is a diagram showing an acoustic spectrogram observed for a fan in a rotational mode of the present invention.
Figure 9 is a diagram showing wind characteristics combined to identify a set of downwind times of the present invention.
10 is a diagram showing angular position measurement based on the x-axis in the fan of the present invention.
11 is a diagram for explaining the rotation of the fan of the present invention.
12 is a diagram showing the pseudo code of the algorithm for determining the angular position of the present invention.
13 is a flowchart of a method for automatically tracking a user's direction of a smart electric fan using a smart phone according to an embodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The detailed description of the present invention which follows refers to the accompanying drawings which illustrate, by way of illustration, specific embodiments in which the present invention may be practiced. These embodiments are described in sufficient detail to enable one skilled in the art to practice the present invention. It should be understood that the various embodiments of the present invention are different from each other but are not necessarily mutually exclusive. For example, specific shapes, structures, and characteristics described herein may be implemented in one embodiment in another embodiment without departing from the spirit and scope of the invention. Additionally, it should be understood that the location or arrangement of individual components within each disclosed embodiment may be changed without departing from the spirit and scope of the invention. Accordingly, the detailed description set forth below is not to be taken in a limiting sense, and the scope of the present invention, if properly described, is limited only by the appended claims, along with all equivalents as claimed by those claims. Like reference numbers in the drawings indicate the same or similar function throughout the various aspects.

Hereinafter, preferred embodiments of the present invention will be described in more detail with reference to the drawings.

A recent trend in electric fans is to support smart functions based on an internet connection, such as remote control. The present invention can more intelligently support smart electric fans (or referred to as fans) by proposing 'WindTrack', which is a new angular positioning technology for electric fans.

The present invention may help develop new technology smart fans that use spatial information for automatic wind direction control. Although angular positioning has been extensively studied in the prior art, there are limitations in realizing smart fans in terms of deployability (due to hardware requirements), accuracy and usability.

The WindTrack technology according to the present invention overcomes these limitations by utilizing the acoustic phenomenon of refraction caused by wind. More specifically, sound signals are transmitted and received using a smart phone located near the user while operating the fan in a rotation mode. Therefore, the WindTrack of the present invention requires only a fan connected to the Internet to collaborate with a smartphone.

The angular position of the user is then identified based on the relationship between the propagation characteristics of the received signal and the rotating fan. The robustness of WindTrack can be further improved by using multiple speakers and microphones built into the smartphone. For example, receive wind noise that is picked up as the wind passes through the microphone.

1 is a conceptual diagram showing usage scenarios and GUIs of WindTrack according to the present invention. 2 is a block diagram of a device for automatically tracking a user's direction of a smart electric fan using a smart phone according to an embodiment of the present invention.

Referring to FIG. 1, the present invention combines a commercially available smart phone and an IoT (or Internet-connected) electric fan to measure the user's angular position while the smart phone is running.

The present invention proposes "WindTrack", a new system capable of tracking the exact angular position of an electric fan through collaboration with a smart phone. The only requirement for deploying WindTrack is that the fan allows remote control over the Internet.

That is, the WindTrack of the present invention can be applied to most commercial Internet-connected fans without additional hardware. The core concept of WindTrack is to utilize the effect of wind on sound propagation. Wind, which is essentially air in motion, affects the speed at which sound propagates through the air. Furthermore, since the strength and direction of wind vary from place to place, the change in sound speed can vary from place to place.

Thus, wind causes sound waves to travel from one area to another at different speeds, which in turn leads to sound wave deflection. This phenomenon is also caused by wind blowing from a fan. For example, when a fan blows air into a user, all sound waves traveling around the user experience deflection.

Referring to Figure 1, WindTrack shows the process of identifying the user's angular position (denoted by θ ^U ) by utilizing acoustic phenomena. First, WindTrack can be triggered by a user on a smartphone, such as turning on a fan's automatic direction control mode. The WindTrack rotates, first changing the direction of the wind, while the fan blows it.

Then, it transmits and receives sound signals using the speaker and microphone built into the smartphone. WindTrack analyzes the propagation characteristics of the received signal. In particular, when the characteristics change significantly, that is, when deflection occurs due to direct wind, θ ^U is determined by calculating the wind direction of the fan at the expected wind time. Finally, the direction of the fan is set to the angle θ ^U. This angular positioning process is automatically repeated when a change in user position is detected.

Referring to FIG. 2 , the device for automatically tracking the user's direction of a smart electric fan using a smart phone according to the present invention (hereinafter referred to as device 10) includes an oscillator unit 110, a scanning unit 130, an analysis unit 150, and an angle tracking device. It includes a government 170 and a wind direction setting unit 190.

In the device 10 of the present invention, software (application) for performing automatic tracking of a user's direction of a smart electric fan using a smart phone may be installed and executed, and the oscillator unit 110, the scanning unit 130, The analysis unit 150, the angle estimation unit 170, and the wind direction setting unit 190 are configured to automatically track the user's direction of the smart electric fan using the smart phone executed in the device 10. Can be controlled by software.

The device 10 may be a separate terminal or a part of a module of the terminal. In addition, the configuration of the oscillator unit 110, the scanning unit 130, the analysis unit 150, the angle estimation unit 170, and the wind direction setting unit 190 is formed as an integrated module, or one or more modules. can be made with However, on the contrary, each component may be composed of a separate module.

The device 10 may be mobile or stationary. The apparatus 10 may be in the form of a server or engine, and may be a device, an apparatus, a terminal, a user equipment (UE), a mobile station (MS), or a wireless device. It can be called by other terms such as wireless device, handheld device, etc.

The device 10 may execute or manufacture various software based on an operating system (OS), that is, a system. The operating system is a system program for enabling software to use the hardware of the device, and is a mobile computer operating system such as Android OS, iOS, Windows mobile OS, Bada OS, Symbian OS, Blackberry OS, and Windows-based, Linux-based, Unix-based, It can include all computer operating systems such as MAC, AIX, and HP-UX.

When the smart electric fan is operated, the oscillator unit 110 operates in a rotation mode for a certain period of time while changing the wind direction. The scanning unit 130 receives a sound signal emitted by the wind using a speaker and a microphone built into a smart phone near the user.

When the fan blows air directly onto the smartphone (and user), it can capture the unique acoustic features due to the refraction phenomenon. WindTrack of the present invention uses this sound as one of the basic clues of angular position tracking through observation and analysis.

The present invention proposes an active sound sensing method that transmits and receives an inaudible signal using a plurality of speakers and microphones in most 'smart phones'. This increases the probability of clearly observing the refraction phenomenon in various real environments.

In particular, we propose two new functions of “low-frequency” energy designed based on the uniqueness of radio waves that detect when a received signal is refracted and the phenomenon of recording low-frequency noise when wind passes through a microphone. These functions compensate for each other's weaknesses in wind time estimation and enable more accurate measurement.

The present invention accurately estimates the user's angular position without modifying the firmware of commercially available, Internet-connected, fans. To this end, the rotational characteristics of the fan are utilized to the fullest.

The analyzer 150 analyzes propagation characteristics of the received sound signal to detect deflection caused by wind.

The design of 　WindTrack for 　deployability, usability and accurate angular position tracking for the electric fan　 of the present invention uses the phenomenon that 　wind refracts sound waves and affects radio waves.

Wind and Sound Refraction In theory, sound is a wave that travels through a medium such as air, and changes in the medium cause the wave to propagate differently. For example, the movement of air or wind affects the propagation of sound in the air, especially the speed of propagation. As sound travels in wind or wind, its velocity increases or decreases, respectively. More specifically, assuming that the wind blows at a speed u in the direction in which sound propagates, the sound speed c is defined as in Equation 1 below.

[Equation 1]

where c ^* is the speed of sound in air at rest.

The wind speed u may vary depending on the region. The most common example is the vertical gradient of wind speed caused by a decrease in temperature with height. As sound passes between regions of different wind speeds, it propagates at different velocities and eventually refracts according to Snell's law for a moving medium.

When c ₁ and c ₂ represent sound speeds in regions 1 and 2, respectively, when sound moves from region 1 to region 2 as shown in FIG. 3, the angle of incidence (θ ₁ ) and the angle of refraction (θ ₂ ) are have a relationship

[Equation 2]

The sound wave is refracted to a region where the speed of sound is low, and the amount of refraction θ is determined by Equation 3 below by the ratio of c ₁ and c ₂ .

[Equation 3]

Specifically, in the presence of wind, θ of Δ is mainly affected by the wind speed difference between the two regions, that is, u ₁ and u ₂ , as shown in Equation 4 below.

[Equation 4]

That is, the greater the change in wind speed, the more the sound wave is refracted.

This refraction affects the sound wave propagation path from the sound source (eg loudspeaker) to the receiver (eg microphone) and the characteristics of the received signal. It is assumed that the acoustic signal emitted from the sound source reaches the receiver through several n ^P paths including straight lines and reflections from surrounding objects.

Then, the signal received on the ith path denoted by y _i (t) is modeled as a _i x(t-τ _i ). Here, a _i and τ _i are the amplitude and time delay of the signal in the ith path, and x(t) is the transmitted signal. The parameters _ai and _τi depend primarily on the propagation path, such as the distance traveled by the path and the reflection ratio of the reflector. For example, as the distance of a path increases, the signal on the path arrives with a larger delay (τ _i ) and a smaller amplitude (a _i ).

These multipath signals are superimposed to form the resulting received signal y(t) as shown in Equation 5 below.

[Equation 5]

At this time, multi-path interference occurs and the frequency characteristics of the received signal are changed. In particular, depending on a _i and τ _i of the signal from each path, y(t) has a different phase and magnitude at a specific frequency. For example, if two signals arrive at frequency f with a time difference of 1/f (or 1/2f seconds), generating (or destroying) interference occurs and the amplitude of the resulting signal increases (or decrease).

Therefore, due to a change in propagation path such as deflection, received signals have different time delays and amplitudes in time and frequency domains, and eventually have different propagation characteristics. That is, y(t) 　 and Y(f), where Y(f) is the Fourier transform (or frequency spectrum) of y(t) calculated by Equation 6 below.

[Equation 6]

In the present invention, one of the main techniques of "WindTrack" for specifying the angular position is to use Y(f), which is affected by the refraction phenomenon. To observe the feasibility of refraction-based angular positioning in a fan, a preliminary experiment was conducted using a Google Pixel 4 smartphone and a Xiaomi Smartmi Fan 2S in a small, quiet office.

During the experiment, the smartphone was placed on a table in front of a fan 1.5 m away. A 1 second long linear chirp was recorded at a sampling rate of 48 kHz while the smartphone was running and the fan was blowing air at full speed. The signal covers the frequency range of 12-24 KHz and typical noise represents a negligible sound level.

4 is a diagram showing the frequency spectrum of a sound signal measured when there is wind and when there is no wind. Due to the refraction of sound by the wind, the two signals have different frequency characteristics with a correlation coefficient of 0.91.

5 is a diagram showing spatial variability of wind speed when a fan blows forward. The dotted circle indicates the position of the smartphone, and the speed was measured with a Benetech GM816 anemometer.

Referring to FIG. 4, it is shown that the wind blowing from the fan affects the propagation of sound waves. Fans are usually small, so they blow air over a small area rather than the entire room. This causes differences in wind speed depending on the region.

For example, as shown in FIG. 5 , the wind blows only in an area close to the wind direction of the fan, and the speed greatly decreases in other areas. Due to the spatial variability of wind speed, sound waves emitted from the loudspeaker of the smart phone are refracted according to Equation 2 and transmitted to the built-in microphone through the refracted path. This change in the propagation path then compares the unique characteristic (Y(f)) of the received signal with the signal in the absence of wind (see Fig. 4).

6 is a diagram showing a correlation between frequency spectra of sound signals detected according to a wind direction (θ ^W ) of a fan. The spectrum was measured 5 times for each direction, 0 degrees means the fan blows to the front, i.e. directly to the smartphone.

In addition, as the wind direction of the fan, represented by θ ^W , changes, the characteristics of sound waves change (see FIG. 6). Depending on ^θW , the spatial variation patterns of sound waves of wind speed change are refracted differently. In particular, when the wind blows directly toward the smart phone (ie, θ ^W is 0 degrees), the similarity is further reduced.

As can be seen in FIG. 5, the wind blowing directly brings a significant spatial change in the wind speed near the “smart phone”. That is, according to Equation 4, a sound wave emitted from a smartphone causes severe refraction, and in particular, a intensive change in radio wave characteristics.

7 is a schematic diagram showing the system architecture of WindTrack proposed in the present invention.

Since the present invention utilizes the phenomenon of refraction when designing the WindTrack, the WindTrack works in two stages: scan and analyze together with the fan and the smartphone.

During the scanning phase, WindTrack emits and records acoustic signals using a smartphone near the user while operating the fan in rotation mode. This cooperative sensing may be repeated for at least one rotation cycle of the fan (eg, about 30 seconds). In other words, WindTrack collects acoustic data for all possible wind directions.

In the subsequent analysis step, WindTrack compares the propagation characteristics of the received signal and detects an anomaly caused by wind to calculate the wind blowing time. Finally, the user's angular position θ ^U is estimated as the fan's wind direction at the time of blowing, and the fan's direction is changed to the angle θ ^U .

Additionally, WindTrack can be triggered automatically when a user's location changes significantly. Displacement can be detected when the acceleration of the smartphone fluctuates significantly for more than 3 seconds (eg standard deviation greater than 0.5 m/s ² ). Then, when the acceleration stabilizes again, the user's angular position is re-estimated.

The WindTrack of the present invention can be deployed and set an available angular position while assuming only the following four conditions. 1) The operation of the fan (e.g. rotation) can be controlled via the Internet, 2) There is a speaker and microphone equipped with a smartphone, 3) The user stays in one place during the scanning phase, 4) The rotation of the fan equal to the rotation period Parameters are provided in advance.

Since one of the recent major trends in electric fans is to be able to connect to the Internet and support remote control services including rotation control, conditions 1) and 2) can be easily satisfied. Also, most smartphones already have several built-in speakers and microphones. Thus, 　WindTrack can be easily deployed in a variety of environments with minimal hardware requirements (e.g. commercial internet-connected fans and 　smart phones).

Condition 3) is a relatively common scenario when using a fan. For example, users typically use the fan while sitting on a sofa or chair and doing something (watching TV or playing games on a smart phone).

Condition 4) can be satisfied in various ways. Some vendors provide detailed information about their fans, such as rotation angle and period. Even if not, users can easily measure mechanical properties with the help of 　WindTrack. This is explained in detail below. After you get this information, you can save it locally for future use and even share it with others.

The 'WindTrack' of the present invention exchanges inaudible sound signals by using several speakers and microphones installed in commercially available smart phones. This allows more refracted sound paths to be observed.

In addition, 　WindTrack of the present invention uses a function called propagation uniqueness to detect the wind blowing time based on the refraction phenomenon. To further increase environmental robustness, an additional feature called low-frequency sound was designed based on the phenomenon that turbulence is recorded at low frequencies when wind passes through the microphone.

When detecting wind refraction, the angle estimating unit 170 detects the wind blowing time and estimates the user's angular position. The wind direction setting unit 190 sets the angular position of the user estimated at the time of blowing of the smart fan as the wind direction of the smart fan.

Even if the support of the fan is lacking, the 　WindTrack of the present invention can accurately estimate the user's angular position by utilizing the fan's rotation parameter without modifying hardware and software.

The present invention's WindTrack uses a linear chirp signal widely used in acoustic sensing to measure the effect of wind on sound propagation. The parameters of the chirp are determined in a way that is robust, but 1) unobtrusive and 2) enables high-speed detection. Therefore, 　WindTrack　 first selects a frequency that is inaudible to humans.

According to previous studies, it is known that humans can hear frequencies up to 20 KHz. However, using a limited range of frequencies (e.g., above 20 kHz) may reduce the signal-to-noise ratio (SNR). Thus, the lowest frequency used to mitigate the SNR degradation is slightly extended to 17 KHz, which is still barely audible to most adult humans. The highest frequency can be set to 24KHz, as many commercial Android　smartphones support sampling rates up to 48KHz.

Second, the length of the signal is kept short enough to observe changes in propagation characteristics at high detection rates. However, there is a trade-off between SNR and detection speed. For example, longer signals can support higher SNRs but lower detection rates. For balance, the length can be determined based on the observation that commercial fans typically rotate at about 10 degrees per second in rotational mode. This means that support for sensing rates of at least 10 Hz is required to achieve accuracy of a few degrees or less. Therefore, the length of the chirp signal can be empirically set to, for example, 40 ms.

WindTrack according to the present invention continuously transmits a chirp signal for a rotation period of an electric fan, that is, about 30 seconds. More specifically, the signal is repeatedly reproduced every 70 ms, i.e., with a time interval of 30 ms between two consecutive signals. This interval is necessary to support independent measurements for each signal. A signal emitted from the speaker is transmitted to the microphone either directly or via reflection.

The intensity of the reflected wave gradually decreases over time. That is, when traveling over long distances, especially in homes and small offices, the reflections almost disappear about 30 ms after their direct arrival. In other words, at 30ms intervals, WindTrack can minimize interference between repeated signals while supporting a high detection rate of 14.2Hz. Before each chirp is reproduced, it can be preprocessed by removing burst noise caused by rapid fluctuations at the beginning and end of the signal.

WindTrack of the present invention can further improve its function by using multiple speakers and microphones for signal transmission. Commercially available smartphones are equipped with multiple speakers and microphones to improve playback and recording quality, respectively. Using these multiple audio devices allows WindTrack to gather a wealth of information about sound propagation, including refraction.

The main reason for this difference is that the audio input/output devices are in different locations. In other words, the path of the sound wave depends on the combination of transceivers. Once captured, WindTrack plays the same chirp signal from both speakers while taking audio samples from multiple microphones.

이고 l^D는 신호 전송 간격 즉,　70ms이다.　따라서, 기록된 샘플은 프레임 크기가 l^D이고　연속된 두 프레임 사이에 겹치지　않는 n^T개의 서브 프레임으로 분리된다.　When collecting acoustic data, WindTrack first divides the audio sample into frames. Each frame contains a single received signal. l ^O is assumed to be the duration of the scanning phase equal to the rotation period of the fan in use. The signal is transmitted n ^T times during the rotation period, where n ^T is

and l ^D is the signal transmission interval, that is, 70 ms. Therefore, the recorded sample is divided into n ^T subframes with a frame size of ^lD and non-overlapping between two consecutive frames.

Then, two acoustic features, propagation uniqueness and low-frequency energy, are extracted from each frame, which are used to estimate wind time.

First of all, if the extraction of propagation characteristics is described, the received signal has unique propagation characteristics when wind blows. To exploit this relationship in angular position, WindTrack extracts the propagation uniqueness of the _ith received signal, denoted by pi, which represents the difference from other signals in terms of propagation characteristics.

First, after filtering out the frequency range of interest that is not included in the chirp signal, the frequency spectrum of the filtered sample is calculated using FFT (Fast Fourier Transform) with n ^F coefficients, and the size of the frequency range of 17-24KHz ( magnitudes) only. Finally, p _i is obtained as shown in Equation 7 below.

[Equation 7]

Here, s _i,j is a Pearson correlation coefficient between the extracted magnitudes of the i-th signal and the j-th signal. That is, _pi increases as the signal has a unique propagation characteristic compared to other signals.

Low-frequency energy estimation is described below. You can also listen to the sound produced by the wind passing through the microphone, called wind noise, to identify windy times.

8 is a diagram showing an acoustic spectrogram observed for a fan in a rotating mode of the present invention. Here, the brighter the spectrum, the louder the sound is captured at that frequency.

As can be seen in Fig. 8, a high-amplitude, low-frequency (e.g., greater than 200 Hz) sound is only observed when wind blows across the microphone for about 3 seconds. In particular, there is a synergistic effect of using both propagation characteristics and wind noise when estimating the wind blowing time.

For example, if there is a moving object, wind noise can play an important role in the measurement because it is hardly affected by the displacement of the surrounding object (see Fig. 8(b)). Based on this observation, WindTrack extracts an additional feature called low-frequency energy (e _i ) from the ith frame. First, a low-pass Butterworth filter of order 4 and a cut-off frequency of 200 Hz can be applied. WindTrack can calculate e _i as shown in Equation 8 below.

[Equation 8]

는 시간 t에서 i 번째 저역 통과 필터링된 프레임에서 관찰된 진폭이다.here,

is the observed amplitude in the ith low-pass filtered frame at time t.

와

를　각각　k 번째 마이크의 녹음에서 얻은 i 번째 프레임의 전파 고유성과 저주파 에너지라고 정의한다. 본 발명에 따른 WindTrack은 기능을 m_i로 표시되는 단일 값으로 결합하여(수학식 9), 바람이 불어 오는 시간을 예측할 때 견고성을 향상시킨다.The feature extraction process is repeated for audio recordings captured from multiple microphones to combine features.

and

is defined as the propagation uniqueness and low-frequency energy of the i-th frame obtained from the recording of the k-th microphone, respectively. WindTrack according to the present invention combines the functions into a single value represented by m _i (Equation 9), improving robustness when predicting wind time.

[Equation 9]

와

에 대한 가중치이다.　WindTrack은 일련의

와

를 병합하기 전에 최소-최대 스케일링(0~1 범위)을 통해 개별적으로 정규화할 수 있다.where n ^M is the number of microphones, and α ^k and β ^k are respectively

and

is the weight for WindTrack is a series of

and

can be individually normalized through minimum-maximum scaling (range 0 to 1) before merging.

In the present invention, the weights α ^k and β ^k can be individually and adaptively determined. Basically, we can assign more weight to feature sets with more pronounced outliers, i.e. more clearly capturing wind-driven phenomena.

가 k 번째 마이크에 의해 획득된 전파 고유성 값의 집합, 즉

라고 가정한다.　

는 각각 정상-J 최고 값과 나머지로 구성되는, 두 클러스터,

및

로 분할된다. 그런 다음 α^k 다음의 수학식 10과 같이 계산된 두 군집 간의 최대 차이로 결정한다.

is the set of propagation uniqueness values obtained by the k-th microphone, i.e.

Assume that

are two clusters, each consisting of the top-J peak value and the remainder,

and

is divided into Then, α ^k is determined as the maximum difference between the two clusters calculated as in Equation 10 below.

[Equation 10]

및　

는 각각

및

세트의 평균을 나타낸다.　특히, j의 범위는 0.05×n^T 부터 0.2× n^T,　즉, 이상 값 클러스터

의 크기를 제한한다. 이는 팬의 회전 주기 동안 사용자에게 바람이 짧은 시간 동안 불기 때문이다. 유사하게, β^k는

세트에 대해 클러스터링 기반 방법을 수행하여 얻을 수 있다.　일 실시예에서, α^k 및 β^k는 스마트 폰에 터치 입력이 이루어진 경우에 0으로 설정되고, 저주파수 음의 최대 진폭은 각각 α^k 보다 낮다.here,

and

are respectively

and

represents the average of the set. In particular, j ranges from 0.05×n ^T to 0.2× n ^T , i.e. clusters of outliers.

limit the size of This is because wind blows to the user for a short period of time during the rotation period of the fan. Similarly, β ^k is

It can be obtained by performing a clustering-based method on the set. In one embodiment, α ^k and β ^k are set to 0 when a touch input is made to the smart phone, and the maximum amplitude of a low-frequency sound is lower than α ^k , respectively.

When estimating wind time, it is important to detect outliers among the extracted features (m _i ). Figure 9 shows that m _i shows significant fluctuations when the wind blows. In particular, this change is guaranteed to be observed once or twice in a series of m _i . This is because during one rotation cycle, the fan delivers airflow to the target user when it rotates up to two times, i.e. left to right and right to left. One exceptional situation is that the user is on the corner of a rotating arc. In this case, m _i fluctuates only once.

를　추정한다.　첫째, 명확한 피크 값을 얻기 위해 일련의 m_i에 Gaussian smoothing을 적용한다.　스무딩을 위한 창(window)의 크기는 3 초로 설정될 수 있다.　팬과　스마트 폰은 서로 독립적으로 작동하기 때문에　스마트 폰은 녹음 시작시 중요한 정보를 놓칠 수 있다(도 9 참조). Based on these fluctuations, WindTrack is a set of windward times,

to estimate First, Gaussian smoothing is applied to a series of m _i to obtain clear peak values. The size of a window for smoothing may be set to 3 seconds. Since the fan and smartphone operate independently of each other, the smartphone may miss important information at the start of recording (see Fig. 9).

　WindTrack's scan phase, however, is accurately maintained during one rotation cycle of the fan. This means that the missed pattern is repeated and captured at the end of the recording. Considering these periodic characteristics, WindTrack uses a circular smoothing method and normalizes the smoothed features with a min-max scale (in the range [0, 1]).

를 아래의 수학식 11과 같이 시간 순간 집합　t^W로 계산한다. 여기서, 각 순간의 해당 특성은 다음과 같이 0.2보다 큰 로컬 최대 값을 갖는다.at last

is calculated as a set of time instants t ^W as shown in Equation 11 below. Here, the corresponding characteristic at each instant has a local maximum greater than 0.2 as

[Equation 11]

If more than three times are selected, the two with the highest peak value can be selected.

가 각각　

로 표시된다고 가정한다.　θ^U를 결정하는 한 가지 순수한 접근 방식은　

에서 팬의 풍향을 계산하는 것이다.　그러나, 상기 언급된 바와 같이 상용 팬의 제한된 지원으로 인해 특정 시간에 풍향을 정확하게 식별하는 것은 매우 어렵다.Referring to FIG. 9(a), given a set of times t ^W during which the wind blows, WindTrack estimates θ ^U , the angular position of the target user relative to the x-axis of the fan. n ^W to include wind blow time

are respectively

Assume that it is represented by One naive approach to determining θ ^U is

to calculate the wind direction of the fan. However, as mentioned above, it is very difficult to accurately identify the wind direction at any given time due to the limited support of commercial fans.

The WindTrack of the present invention addresses this limitation by utilizing the rotational characteristics of the fan. One of the key functions of a fan is to rotate left and right to direct airflow over a larger area. Assume that the fan supports a rotational angle of θ ⁰ and the head initially faces the left edge of the arc as shown in FIG. 11 .

기간 동안 회전한다.　그 후, 헤드는 일정 시간(l^S) 동안 호의 오른쪽 모서리에 머물고 반대 방향(오른쪽에서 왼쪽으로)으로 회전하고 왼쪽 모서리에 머물러 있다.　이 네 가지 상태는

의 회전 주기로 정기적으로 반복된다.　The periodic movement of the fan can be divided into four states. First, the fan's head moves from left to right with an angular velocity of v ^R , that is,

rotate during the period After that, the head stays on the right edge of the arc for some time l ^S , then rotates in the opposite direction (right to left) and stays on the left edge. these four states

is repeated regularly with a rotation period of

과　

　사이에 팬의 풍향이 θ^U에서 θ^O/2로 선형으로 변경되고 일정 기간 동안 안정된 후 다시 θ^U로 회전 함을 보여준다.　여기서, 다음의 수학식 12의 관계를 도출할 수 있다.These parameters can be provided by the hardware vendor or measured by WindTrack. 10 (b) shows two windy times

class

It shows that the wind direction of the fan changes linearly from θ ^U to θ ^O /2 in the interval and then rotates back to θ ^U after being stable for a period of time. Here, the relationship of Equation 12 below can be derived.

[Equation 12]

That is, the user's angular position θ ^U can be obtained from the measured windy time and predetermined parameters as shown in Equation 13 below.

[Equation 13]

,　

]의 회전 방향은 예기치 않게 달라질 수 있다.　그 주된 이유는 팬의 초기 풍향과 회전 방향이 이전 작동 상태에 따라 다르며 상용 팬은 이러한 유형의 정보를 제공하거나 보관하지 않기 때문이다.　이러한 예측 불가능으로 인해　다음의 수학식 14와 같이　팬이　

이후 왼쪽 또는 오른쪽으로 회전하는 경우를 포함하여 가능한 각 경우에 대해 θ^U를 계산해야 한다.however, [

,

] may change unexpectedly. The main reason for this is that the fan's initial wind direction and direction of rotation are dependent on its previous operating conditions, and commercial fans do not provide or retain this type of information. Due to this unpredictability, the fan as shown in Equation 14 below

Then, θ ^U must be calculated for each possible case, including left or right turns.

[Equation 14]

When the user is at the corner of the rotating arc, the time the wind blows is observed only once. Based on this, if n ^W is 1, WindTrack determines θ ^U as ±θ ^O /2.

과　

　사이에 어떻게 변하는지 정확히 알 수　있고,　

이후의 변화를 예측할 수 있기　때문이다.　따라서, 특정 시간의 풍향에 대한 정보를　얻으면 WindTrack은　사용성을 더 이상 손상시키지 않고 사용자의 각도 위치를 자동으로 추적한다.To make an accurate decision, WindTrack can ask the user to choose between two candidates. This user cooperation is only required once while using the fan. This allows WindTrack to adjust the fan's wind direction with the help of the user.

class

You can see exactly how it changes between

This is because future changes can be predicted. Thus, upon obtaining information about the wind direction at a particular time, WindTrack automatically tracks the user's angular position without further compromising usability.

13 is a flowchart of a method for automatically tracking a user's direction of a smart electric fan using a smart phone according to an embodiment of the present invention.

The method for automatically tracking a user's direction of a smart electric fan using a smart phone according to the present embodiment may be performed in substantially the same configuration as the device 10 of FIG. 1 . Accordingly, components identical to those of the apparatus 10 of FIG. 1 are given the same reference numerals, and repeated descriptions are omitted.

In addition, the method for automatically tracking the user's direction of a smart electric fan using a smart phone according to the present embodiment may be executed by software (application) for automatically tracking the user's direction of a smart electric fan using a smart phone.

The present invention is a technology that more intelligently supports a smart electric fan (or referred to as fans) by proposing a 'WindTrack', which is a new angular positioning technology for an electric fan.

The WindTrack technology according to the present invention utilizes the acoustic phenomenon of refraction caused by wind to automatically control the wind direction. More specifically, sound signals are transmitted and received using a smart phone located near the user while operating the fan in a rotation mode. The angular position of the user is then identified based on the relationship between the propagation characteristics of the received signal and the rotating fan. The robustness of WindTrack can be further enhanced by using multiple speakers and microphones built into the smartphone, for example, to receive wind noise that is picked up as the wind passes through the microphone.

Referring to FIG. 13, in the method of automatically tracking the user's direction of a smart fan using a smart phone according to the present embodiment, when the smart fan is operated, it operates in a rotation mode for a predetermined time while changing the wind direction (step S10).

For example, a linear chirp signal having a length of 40 ms and having a frequency of 17 KHz or more and 24 KHz or less may be transmitted every 70 ms while being repeated for at least one rotation period.

A sound signal emitted by the wind is received using a speaker and a microphone built into a smart phone near the user (step S20).

In this case, the received sound signal may be divided into subframes, and propagation uniqueness and low frequency energy characteristics may be extracted from each subframe and combined with weights.

Refraction caused by wind is detected by analyzing the propagation characteristics of the received sound signal (step S30).

When detecting the refraction of the wind, the user's angular position is estimated by detecting the time when the wind blows (step S40).

It is calculated using the rotation parameter of the smart fan and the wind blowing time. Rotational parameters can be provided by the fan used or calculated through data accumulation.

The angular position of the user estimated at the time of blowing the smart fan is set as the wind direction of the smart fan (step S50).

When a change in the user's location is detected, the user's location is re-estimated by automatically repeating all steps. For example, if the acceleration exceeds a threshold value for a preset time (eg, 3 seconds), it may be determined as a change in the user's location.

In summary, in the scanning step, the present invention emits and records sound signals using a smart phone near the user while operating the fan in a rotation mode. This cooperative sensing may be repeated for at least one rotation cycle of the fan (eg, about 30 seconds). In other words, WindTrack collects acoustic data for all possible wind directions.

After that, in the analysis step, the propagation characteristics of the received signal are compared, and the wind blow time is calculated by detecting anomalies caused by the wind. Finally, the user's angular position θ ^U is estimated as the fan's wind direction at the time of blowing, and the fan's direction is changed to the angle θ ^U .

Accordingly, the WindTrack proposed in the present invention can be easily deployed to an electric fan by minimally modifying the hardware and software of the fan, and does not depend on a pre-installed infrastructure.

In addition, it is easy to set and use because the angular position can be specified without a user's manipulation. Furthermore, the WindTrack according to the present invention accurately measures the user's angular position in the fan in order to accurately deliver cool air to the target, and can operate regardless of the environment.

Such a method for automatically tracking a user's direction of a smart electric fan using a smart phone may be implemented as an application or implemented in the form of program instructions that can be executed through various computer components and recorded on a computer-readable recording medium. The computer readable recording medium may include program instructions, data files, data structures, etc. alone or in combination.

Program instructions recorded on the computer-readable recording medium may be those specially designed and configured for the present invention, or those known and usable to those skilled in the art of computer software.

Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical recording media such as CD-ROMs and DVDs, and magneto-optical media such as floptical disks. media), and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like.

Examples of program instructions include high-level language codes that can be executed by a computer using an interpreter or the like as well as machine language codes such as those produced by a compiler. The hardware device may be configured to act as one or more software modules to perform processing according to the present invention and vice versa.

Although the above has been described with reference to embodiments, those skilled in the art can variously modify and change the present invention without departing from the spirit and scope of the present invention described in the claims below. You will understand.

The performance of WindTrack proposed in the present invention was evaluated through prototype implementation distributed to commercially available “smart phones” and internet-connected fans. According to the evaluation results, the WindTrack according to the present invention was able to very accurately determine the user's angular position with an average error of 1.27 degrees in various angular positions. In addition, the WindTrack proposed in the present invention has successfully solved various practical problems. It showed high performance even in the presence of low speed wind, moving objects and ambient noise. That is, since the WindTrack proposed in the present invention does not require separate hardware for a smart electric fan, it is highly deployable, can be used conveniently, and can support accurate angular positioning.

10: Automatic tracking device for user direction of smart fan
110: oscillator unit
130: scanning unit
150: analysis unit
170: angle estimation unit
190: wind direction setting unit

Claims (12)

When the smart fan is operated, operating in a rotation mode for a predetermined time while changing the wind direction;
Receiving a sound signal emitted by the wind using a speaker and a microphone built into a smart phone near the user;
detecting deflection caused by wind by analyzing propagation characteristics of the received sound signal;
estimating a user's angular position by detecting a wind blowing time when wind refraction is detected; and
A method of automatically tracking the user's direction of a smart electric fan using a smart phone, including the step of setting the user's angular position estimated at the time of blowing of the smart electric fan as the wind direction of the smart electric fan.
According to claim 1,
When a change in the user's location is detected, automatically repeating all the steps to re-estimate the user's location; further comprising, a method for automatically tracking the user's direction of a smart electric fan using a smart phone.
The method of claim 2, wherein the step of re-estimating the location of the user comprises:
A method for automatically tracking a user's direction of a smart electric fan using a smart phone, which is determined by a change in the user's position when the acceleration exceeds a threshold value for a preset time.
The method of claim 1, wherein the step of operating in a rotation mode for a predetermined time while changing the wind direction comprises:
A method for automatically tracking the user's direction of a smart electric fan using a smart phone, which repeats for at least one rotation cycle and transmits a linear chirp signal every 70 ms with a frequency of 17 KHz or more and 24 KHz or less and a length of 40 ms during each rotation cycle.
The method of claim 1, wherein receiving the sound signal emitted by the wind comprises:
Dividing the received sound signal into subframes;
extracting propagation uniqueness and characteristics of low-frequency energy in each subframe; and
A method for automatically tracking a user's direction of a smart electric fan using a smart phone, including the step of weighting the characteristics of radio wave uniqueness and low-frequency energy.
The method of claim 1, wherein the estimating the angular position of the user comprises:
A method for automatically tracking the user's direction of a smart fan using a smart phone, which is calculated using the rotation parameter of the smart fan and the wind blowing time.
A computer-readable storage medium having a computer program recorded thereon for performing the method for automatically tracking a user's direction of a smart electric fan using the smart phone according to any one of claims 1 to 6.
an oscillator unit operating in a rotation mode for a certain period of time while changing a wind direction when the smart fan is operated;
a scanning unit for receiving a sound signal emitted by the wind using a speaker and a microphone built into a smart phone near the user;
an analyzer configured to detect refraction caused by wind by analyzing propagation characteristics of the received sound signal;
an angle estimator for estimating an angular position of the user by detecting a wind blowing time when detecting wind refraction; and
A wind direction setting unit for setting the user's angular position estimated at the time of blowing of the smart fan as the wind direction of the smart fan; including, a device for automatically tracking the user's direction of a smart fan using a smart phone.
According to claim 8,
When a change in the user's location is detected, a re-estimation unit for re-estimating the user's location; further comprising, a device for automatically tracking the user's direction of a smart electric fan using a smart phone.
The method of claim 9, wherein the reestimation unit,
An automatic tracking device for a user's direction of a smart electric fan using a smart phone, which is determined by a change in the user's position when the acceleration exceeds a threshold value for a preset time.
The method of claim 8, wherein the scanning unit,
A device for automatically tracking a user's direction of a smart electric fan using a smart phone, which divides a received sound signal into subframes, extracts characteristics of radio wave eigenvalues and low frequency energy from each subframe, and combines them with weights.
The method of claim 8, wherein the angle estimation unit,
A device for automatically tracking the user's direction of a smart fan using a smart phone, which is calculated using the rotation parameter of the smart fan and the wind blowing time.

Images