KR20220088259A - System and methods for locating mobile devices using wireless headsets - Google Patents

Abstract

A method and system for locating a mobile device using a wireless headset are disclosed. A location search method performed in a location search system includes the steps of: receiving sound data generated from another electronic device for which location data is to be searched; determining relative position information with the other electronic device based on the electronic device using the received sound data; and reproducing the stereophonic sound information generated based on the determined relative position information to the electronic device.

Description

SYSTEM AND METHODS FOR LOCATING MOBILE DEVICES USING WIRELESS HEADSETS

The description below relates to a technique for determining the location of a mobile device using a wireless headset.

With the development of wireless technology, the frequency of using a wireless headset for the purpose of listening to music or an online meeting has rapidly increased. Users frequently use two or three electronic devices, including smartphones and tablets, at the same time. If the location of the corresponding electronic device is not known, there is a need to propose a technology that enables a user to easily find the corresponding electronic device by playing an alarm sound to which a stereoscopic sound technology capable of determining the location of the corresponding electronic device is applied to a wireless headset.

Even without a wireless headset, a sound can be generated directly from an electronic device so that the user can immediately recognize the location. Furthermore, there is a need for a technology for generating an inaudible sound from a portable electronic device so that the user can determine the location with the wireless headset he is wearing so as not to disturb the user's privacy and others around him.

By determining the relative location information between the user wearing the wireless headset and the electronic device (mobile device), and applying the stereophonic sound technology corresponding to the determined relative location information, the user can easily find the location of the electronic device in 3D space. A method and system for doing so can be provided.

A location search method performed in a location search system includes the steps of: receiving sound data generated from another electronic device for which location data is to be searched; determining relative position information with the other electronic device based on the electronic device using the received sound data; and reproducing the stereophonic sound information generated based on the determined relative position information to the electronic device.

The receiving may include receiving sound wave data in a sound range above an audible frequency generated from the other electronic device, and determining the relative location information may include analyzing the received sound wave data and the other electronic device. It may include the step of determining the relative position information of the

The determining of the relative position information may include: based on an Interaural Level Difference (ILD) of sound wave data arriving at both ears of a user wearing the electronic device in a three-dimensional space, relative to the other electronic device. It may include calculating a difference value of the volume corresponding to the location information.

The determining of the relative location information may include: based on an Interaural Time Difference (ITD) of sound wave data arriving at both ears of a user wearing the electronic device in a three-dimensional space, the relative position with the other electronic device It may include calculating a time difference value corresponding to the location information.

The determining of the relative position information includes determining the relative position information with the other electronic device using a Head-Related Transfer Function (HRTF) for sound wave data generated from another electronic device in a three-dimensional space. may include the step of

The reproducing may include applying a 3D stereophonic sound technology corresponding to the determined relative position information to the reproducing sound output to the electronic device.

The reproducing may include generating sound body sound information by applying a volume difference value obtained by calculating relative position information with respect to the determined other electronic device to a predetermined alarm sound.

The reproducing may include generating sound body sound information by applying a time difference value obtained by calculating relative position information with the determined other electronic device to a predetermined alarm sound.

The electronic device includes a wireless earphone or a wireless headset capable of receiving sound data generated from the other electronic device, and the electronic device for which the location data is to be searched is a portable device that generates sound wave data in a sound range greater than or equal to an audible frequency. may include devices.

The reproducing may include providing location information of the electronic device through voice guidance (eg, at a point of 3 meters in the 10 o'clock direction).

The location search system includes: a sound receiver configured to receive sound data generated from another electronic device for which location data is to be searched; a location determining unit that determines relative location information with respect to the electronic device based on the electronic device using the received sound data; and a sound reproducing unit that reproduces the stereoscopic sound information generated based on the determined relative position information to the electronic device.

A wireless headset user can easily determine the location of an electronic device by using stereoscopic sound information containing spatial information while reproducing the listening sound as it is (music or the other person's voice).

Since an acoustic signal outside the audible frequency band is used between the wireless headset and the electronic device, there is an advantage in that a desired purpose can be performed without disturbing the people around the wireless headset.

1 is an example for explaining a path through which sound data generated from an object in a three-dimensional space reaches both ears of a listener, according to an embodiment.
2 is a block diagram illustrating a configuration of a location search system according to an embodiment.
3 is a flowchart illustrating a location search method in a location search system according to an embodiment.
4 is an example for explaining a method of determining relative position information according to an embodiment.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings.

1 is an example for explaining a path through which sound data generated from an object in a three-dimensional space arrives to both ears of a listener, according to an embodiment.

In the embodiment, when a user wears an electronic device (eg, a wireless headset such as a Bluetooth earphone or headphones), an operation of notifying the user of the location of another electronic device (eg, a smartphone) in the vicinity will be described. When the user needs to determine the location of another electronic device when wearing the electronic device, the user can play the sound by overlaying the sound to the playback sound to determine the location of the other electronic device through the alarm sound. have.

The location search system may be operated in an electronic device and other electronic devices for which location data is to be searched. For example, the location search system may be driven in the form of a program or an app in the electronic device and the other electronic device.

The electronic device and the other electronic device may be a fixed terminal implemented as a computer system or a mobile terminal. Electronic devices and other electronic devices include, for example, headsets, earphones, smart phones, mobile phones, navigation devices, computers, laptops, digital broadcasting terminals, personal digital assistants (PDAs), portable multimedia players (PMPs), and tablets. There are PCs, game consoles, wearable devices, Internet of things (IoT) devices, virtual reality (VR) devices, augmented reality (AR) devices, and the like. In embodiments, the electronic device and the other electronic device may refer to one of various physical computer systems capable of communicating with other electronic devices and/or servers through a network using a wireless or wired communication method. have. In the embodiment, it is assumed that the electronic device is a wireless headset and the other electronic device is a portable device (eg, a smart phone).

In detail, the location search system may audibly inform a user (a wireless headset worn by the user) of location data of another electronic device whose location data is unclear in a three-dimensional space. In other words, when the user intends to search for a location of another electronic device, after determining the location data of the other electronic device, an alarm sound may be heard at the corresponding location through a preprocessing process of an alarm sound generated by the electronic device. This is basically by determining the difference in time (Interaural Time Difference; ITD) and the difference in volume (ILD) for sound data heard from both ears, and comprehensively, by determining the relative position of the object in the three-dimensional space with respect to the user. The purpose of this is to enable the user to easily locate the electronic device aurally by using a corresponding head-related transfer function (HRTF).

As shown in FIG. 1 , when an object (eg, another electronic device) exists in a three-dimensional space, sound data generated from the object arrives at both ears with a difference in volume and a time difference between the sounds. This becomes very important information for the user to determine the location data of the object. In addition, the acoustic characteristics of the sound arriving at both ears may vary depending on the anatomical structure of the head and the relative position of the object, which is called a head transfer function.

로 정의한다고 하면 해당하는 소리가 사용자의 양쪽 귀에 도달하는 경우의 신호는 다음과 같이 표현이 가능하다.If the sound signal generated by the electronic device is

If defined as , the signal when the corresponding sound reaches both ears of the user can be expressed as follows.

(1a)

(1a)

(1b)

(1b)

은 3차원 공간 내에서 전자 기기의 위치로부터 각각 왼쪽, 오른쪽 귀에 해당하는 위치로 3차원 공간 내의 음파가 전달되는 Head-Related Impulse Response(HRIR),

는 컨볼루션,

과

은 양쪽 귀에 도달하는 배경 잡음을 의미한다. HRIR 은 공간 내의 위치 정보를 판별하게 하는 음향 정보이며 이것의 푸리에 변환을 한 것이 머리전달함수(HRTF) 이다. 사용자는 발생한 소리의 신호

가 3차원 공간을 통해서 전달 된 후 양쪽 귀에 도달하는

과

의 차이를 통해 해당 물체의 3차원 공간에서의 위치를 판단할 수 있다.From here

is Head-Related Impulse Response (HRIR), in which sound waves in 3D space are transmitted from the position of the electronic device in the 3D space to the positions corresponding to the left and right ears, respectively,

is the convolution,

class

is the background noise reaching both ears. HRIR is acoustic information that allows to discriminate position information in space, and the head transfer function (HRTF) is a Fourier transform of this. The user is the signal of the sound

is transmitted through the three-dimensional space and reaches both ears.

class

It is possible to determine the position of the object in 3D space through the difference in .

양쪽 귀에 도달하는 신호가 음량 감쇄 및 시간차로만 구성된다면 HRIR 은 각각 다음과 같이 표현 가능하다.Assuming there is no background noise

If the signal reaching both ears consists only of volume attenuation and time difference, HRIR can be expressed as follows, respectively.

(2a)

(2a)

(2b)

(2b)

은 각각 왼쪽 귀에 도달하는 음량 감쇄 및 도달 시간을 의미하며

은 각각 오른쪽 귀에 도달하는 음량 감쇄 및 도달 시간을 의미한다. 이 경우 각 HRIR 의 푸리에 변환을 하게 되면 다음과 같이 HRTF 를 구할 수 있다.From here

is the volume decay and arrival time to the left ear, respectively.

denotes the volume decay and arrival time to the right ear, respectively. In this case, if Fourier transform of each HRIR is performed, HRTF can be obtained as follows.

(3a)

(3a)

(3b)

(3b)

는 푸리에변환의 주파수에 해당된다.From here

is the frequency of the Fourier transform.

및 음량의 차이(ILD)

(데시벨 기준으로는

(dB)) 를 통해서 사용자가 판별 가능하다. 주어진 두 개의 신호를 기반으로 ITD 및 ILD 를 판별하는 기술은 다수 존재한다.Therefore, the location of the device is the difference in time for sound information to reach both ears (ITD), i.e.

and difference in volume (ILD)

(in decibels)

(dB)) through which the user can determine. There are many techniques for discriminating ITD and ILD based on two given signals.

As such, the head transfer function is defined as a three-dimensional function by generating the same sound data from all directions and measuring the frequency response according to the direction. When sound is generated from a sound source and transmitted to the listener's ear, the sound passes through various paths. For example, when it is assumed that a sound source generated in a room is listened to, there is a direct sound transmitted from the sound source to the listener's ear in the shortest distance. In addition, there is an early reflected sound that is primarily reflected from the floor, wall, or ceiling, and a late reflected sound that is reflected, absorbed, diffracted, or diffused in the second order. A transmission path for this may be obtained, for example, through an impact sound or a specific recording according to a method for measuring room acoustics. However, in the case of a microphone, it is assumed that it is a sound pickup point and only a transmission path by an indoor object needs to be defined. This is the head transfer function. The head transfer function may vary from person to person because it includes influences on the shape of the body such as chest, shoulders, outer ear, head, and nose. In the embodiment, a standard head transfer function obtained by dataizing the results of the head transfer function of many people may be used.

As shown in FIG. 1 , it is an example showing a path through which sound data generated from an object in a three-dimensional space reaches both ears of a listener. In an embodiment, location data of other electronic devices of unknown location may be provided to the user through stereophonic sound techniques by utilizing a time difference, a volume difference, a head transfer function, etc. for sound data for determining the location data of an object. have.

로 정의하고 HRTF 는 ITD

와 ILD

로 구성된다면 무선헤드셋의 왼쪽, 오른쪽 에서 각기 재생하게 될 신호

과

는 다음과 같다.If the beep signal to be reproduced to determine the location of the mobile device is

, and HRTF is ITD

and ILD

signal to be played on the left and right sides of the wireless headset, respectively.

class

is as follows

(4a)

(4a)

(4b)

(4b)

로 표현한다면 무선헤드셋에서 재생하게 될 신호

과

는 다음과 같다.If the HRTF includes information other than ITD and ILD, the HRIR corresponding to the inverse Fourier transform of the corresponding HRTF is calculated.

If expressed as , the signal to be reproduced in the wireless headset

class

is as follows

(5a)

(5a)

(5b)

(5b)

2 is a block diagram illustrating a configuration of a location search system according to an embodiment, and FIG. 3 is a flowchart illustrating a location search method in the location search system according to an embodiment.

The processor of the location search system 100 may include a sound receiver 210 , a location determiner 220 , and a sound reproducer 230 . These processor components may be representations of different functions performed by the processor in accordance with control instructions provided by program code stored in the location search system. The processor and components of the processor may control the location search system to perform steps 310 to 330 included in the location search method of FIG. 3 . In this case, the processor and the components of the processor may be implemented to execute instructions according to the code of the operating system included in the memory and the code of at least one program.

The processor may load the program code stored in the file of the program for the location search method into the memory. For example, when a program is executed in the location search system, the processor may control the location search system to load a program code from a file of the program into a memory according to the control of the operating system. At this time, each of the sound receiving unit 210 , the location determining unit 220 , and the sound reproducing unit 230 executes a command of a corresponding part of the program code loaded in the memory to execute subsequent steps 310 to 330 . They may be different functional representations of the processor.

In step 310 , the sound receiver 210 may receive sound data generated from another electronic device for which location data is to be searched. The sound receiving unit 210 may receive sound wave data in a sound range greater than or equal to an audible frequency generated from another electronic device. The sound receiver 210 may store the received sound wave data.

In step 320 , the location determining unit 220 may determine relative location information with respect to another electronic device for which location data is to be searched based on the electronic device. The location determining unit 220 may analyze the received sound wave data to determine relative location information with respect to other electronic devices. For example, the location determination unit 220 may compare the received sound wave data with a preset value to determine relative location information with respect to other electronic devices.

Referring to FIG. 4 , it is a flowchart for explaining a method of determining relative location information. At 400 , sound wave data in a sound range greater than or equal to an audible frequency generated from another electronic device may be received. In this case, request information for location search may be transmitted from the electronic device to another electronic device. Accordingly, sound wave data may be generated in response to request information for an electronic device from another electronic device. For example, the location determining unit 220 may select any one method for determining relative location information with respect to another electronic device. For example, a method for determining the relative position information selected from the user may be received, and the relative position information may be calculated using the received method. Alternatively, a method for randomly determining relative position information may be selected by a computer/program.

At 410 , the location determining unit 220 may calculate an Interaural Level Difference (ILD) of sound wave data reaching both ears of a user wearing an electronic device in a three-dimensional space. For example, the difference in volume means the difference in the intensity (strength) of sound pressure between the two ears. It is caused by the difference in the path to the two ears according to the location of the sound source and the difference in the amount of decrease due to the difference in the path. The location determining unit 220 may calculate a difference in volume corresponding to relative location information with another electronic device based on a difference in volume of sound wave data reaching both ears of a user wearing the electronic device.

In step 420 , the location determining unit 220 may calculate an Interaural Time Difference (ITD) for sound wave data reaching both ears of a user wearing the electronic device in a three-dimensional space. The time difference means the time difference between the two ears. The location determining unit 220 may calculate a time difference value corresponding to relative location information with another electronic device based on a time difference for sound wave data reaching both ears of a user wearing the electronic device.

In 430, the position determining unit 220 may determine relative position information with other electronic devices by using a Head-Related Transfer Function (HRTF) for sound wave data generated from other electronic devices in a three-dimensional space. have. The head transfer function refers to the characteristics of the acoustic signal received from the ear according to the shape of the head and ears. Using a dummy head microphone modeled on a human auditory organ, a value obtained by obtaining the impulse responses of the left and right sides for each position at a constant interval between an azimuth angle of 360 degrees and an elevation angle of 180 degrees can be derived. The head transfer function can perceive stereophonic sound based on the phenomenon in which complex path characteristics such as diffraction from the surface of the head and reflections on the curves of the pinna change depending on the direction of the sound rather than the simple path difference of the difference in volume and time. can When the head transfer function in each direction has a unique characteristic, it is possible to create a stereophonic sound by using it.

In operation 330, the sound reproducing unit 230 may reproduce the stereoscopic sound information generated based on the determined relative position information to the electronic device. The sound reproducing unit 230 may apply a 3D stereophonic sound technology corresponding to the determined relative position information to the reproduced sound output to the electronic device. The sound reproducing unit 230 may generate stereophonic sound information by applying a volume difference value obtained by calculating relative position information with respect to another electronic device determined to a predetermined alarm sound. A description will be made on the assumption that the predetermined alarm sound is a beep sound. A beep sound is a "beep" sound, usually a short, high pitched signal tone produced by a computer or other machine. Also, the predetermined alarm sound may be set differently according to each electronic device. For example, the sound reproducing unit 230 may generate 3D sound information by applying a volume difference value to the beep sound. For example, a larger difference in volume may mean that the distance between the electronic device and another electronic device is greater, and a smaller difference in volume may mean a shorter distance between the electronic device and other electronic devices. In this case, a pre-processing process for applying the difference in volume to the beep sound may be performed. The pre-processing process may include noise removal, a process of changing a format for applying a volume difference value to a beep sound, and the like. As the preprocessing process is performed, 3D sound information may be generated by applying a difference in volume to the beep sound. In this case, the attribute information (size, frequency, and speed) of the stereoscopic sound information may be adjusted according to the difference value of the volume.

The sound reproducing unit 230 may generate sound body sound information by applying a time difference value obtained by calculating relative position information with respect to another electronic device determined in advance to the alarm sound. For example, the sound reproducing unit 230 may generate 3D sound information by applying a time difference value to the beep sound. For example, a larger time difference value may mean that the distance between the electronic device and another electronic device is greater, and a smaller time difference value may mean a shorter distance between the electronic device and another electronic device. In this case, a pre-processing process for applying the time difference value to the beep sound may be performed. The pre-processing process may include noise removal, a process of changing a format for applying a time difference value to a beep sound, and the like. As the pre-processing process is performed, stereophonic information may be generated based on the time difference between the beeps. In this case, the attribute information (size, frequency, and speed) of the stereophonic sound information may be adjusted according to the time difference value.

Also, the sound reproducing unit 230 may provide location information of the electronic device through voice guidance. For example, the presence of location information of the electronic device at a point 3 meters in the 10 o'clock direction may be informed by voice.

According to an embodiment, a user wearing an electronic device receives 3D sound information including spatial information (eg, left, right, top, bottom, etc.) while playing a sound (eg, call, music) being listened to as it is. can be used to determine the location data of other electronic devices. Also, by periodically or aperiodically receiving sound wave data generated from another electronic device, location data of the other electronic device may be tracked, and another electronic device may be found according to the tracked location data.

In addition, when a predetermined alarm sound is set differently for each electronic device, a plurality of 3D sound information related to a plurality of other electronic devices may be output as a reproduced sound while the sound being listened to by a user wearing the electronic device is reproduced as it is. have. Accordingly, a plurality of different electronic devices can be recognized at the same time.

The device described above may be implemented as a hardware component, a software component, and/or a combination of the hardware component and the software component. For example, devices and components described in the embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA). , a programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions, may be implemented using one or more general purpose or special purpose computers. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. A processing device may also access, store, manipulate, process, and generate data in response to execution of the software. For convenience of understanding, although one processing device is sometimes described as being used, one of ordinary skill in the art will recognize that the processing device includes a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that can include For example, the processing device may include a plurality of processors or one processor and one controller. Other processing configurations are also possible, such as parallel processors.

Software may comprise a computer program, code, instructions, or a combination of one or more thereof, which configures a processing device to operate as desired or is independently or collectively processed You can command the device. The software and/or data may be any kind of machine, component, physical device, virtual equipment, computer storage medium or apparatus, to be interpreted by or to provide instructions or data to the processing device. may be embodied in The software may be distributed over networked computer systems and stored or executed in a distributed manner. Software and data may be stored in one or more computer-readable recording media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the medium may be specially designed and configured for the embodiment, or may be known and available to those skilled in the art of computer software. Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic such as floppy disks. - includes magneto-optical media, and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like.

As described above, although the embodiments have been described with reference to the limited embodiments and drawings, various modifications and variations are possible from the above description by those skilled in the art. For example, the described techniques are performed in an order different from the described method, and/or the described components of the system, structure, apparatus, circuit, etc. are combined or combined in a different form than the described method, or other components Or substituted or substituted by equivalents may achieve an appropriate result.

Therefore, other implementations, other embodiments, and equivalents to the claims are also within the scope of the following claims.

Claims (11)

In the location search method performed in the location search system,
Receiving sound data generated from another electronic device to search for location data;
determining relative location information with the other electronic device based on the electronic device using the received sound data; and
Reproducing the stereophonic sound information generated based on the determined relative position information in the electronic device
A method of locating a location that includes. According to claim 1,
The receiving step is
Receiving sound wave data in a sound range higher than an audible frequency generated from the other electronic device
including,
The step of determining the relative position information,
Analyzing the received sound wave data to determine relative position information with the other electronic device
A method of locating a location that includes. 3. The method of claim 2,
The step of determining the relative position information,
Based on an Interaural Level Difference (ILD) of sound wave data reaching both ears of a user wearing the electronic device in a three-dimensional space, a difference in volume corresponding to relative position information with the other electronic device is calculated. steps to count
A method of locating a location that includes. 3. The method of claim 2,
The step of determining the relative position information,
Based on an Interaural Time Difference (ITD) of sound wave data reaching both ears of a user wearing the electronic device in a three-dimensional space, a time difference value corresponding to relative location information with the other electronic device is calculated. steps to count
A method of locating a location that includes. 3. The method of claim 2,
The step of determining the relative position information,
Using a Head-Related Transfer Function (HRTF) for sound wave data generated from another electronic device in a three-dimensional space, determining relative position information with the other electronic device
A method of locating a location that includes. According to claim 1,
The regenerating step is
applying a 3D stereophonic sound technology corresponding to the determined relative position information to the reproduced sound output to the electronic device
A method of locating a location that includes. 7. The method of claim 6,
The regenerating step is
generating sound body sound information by applying a volume difference value obtained by calculating relative position information with respect to the determined other electronic device to a predetermined alarm sound
A method of locating a location that includes. 7. The method of claim 6,
The regenerating step is
generating sound body sound information by applying a time difference value obtained by calculating relative position information with the determined other electronic device to a predetermined alarm sound
A method of locating a location that includes. According to claim 1,
The electronic device includes a wireless earphone or a wireless headset capable of receiving sound data generated from the other electronic device,
The electronic device for which the location data is to be searched includes a portable device that generates sound waves in a sound range above an audible frequency.
A location search method, characterized in that. According to claim 1,
The regenerating step is
Step of providing location information of the electronic device through voice guidance
A method of locating a location that includes. In the location search system,
a sound receiver configured to receive sound data generated from another electronic device for which location data is to be searched;
a location determining unit that determines relative location information with respect to the electronic device based on the electronic device using the received sound data; and
A sound reproducing unit that reproduces the stereoscopic sound information generated based on the determined relative position information to the electronic device
A location navigation system comprising a.

Images