KR102519156B1 - System and methods for locating mobile devices using wireless headsets - Google Patents

Abstract

A method and system for locating a mobile device using a wireless headset are disclosed. A location search method performed in a location search system includes receiving sound data generated from another electronic device for which location data is to be searched; determining relative location information with respect to the other electronic device based on the electronic device using the received sound data; and reproducing stereophonic sound information generated based on the determined relative location information to the electronic device.

Description

Method and system for notifying mobile device location using wireless headset {SYSTEM AND METHODS FOR LOCATING MOBILE DEVICES USING WIRELESS HEADSETS}

The description below relates to techniques for determining the location of a mobile device using a wireless headset.

With the development of wireless technology, the frequency of users using wireless headsets for the purpose of listening to music or online meetings has rapidly increased. Users frequently use two or three electronic devices including a smartphone, a tablet, and the like at the same time. If the location of the corresponding electronic device is not known, it is necessary to propose a technology that allows the user to easily find the corresponding electronic device by playing an alarm sound applied to a wireless headset using stereophonic technology capable of determining the location of the corresponding electronic device.

By generating sound directly from an electronic device without a wireless headset, the user can instantly recognize the location. Furthermore, there is a need for a technology for generating an inaudible sound in a portable electronic device so that a user can determine a location using a wireless headset worn by the user so as not to disturb the user's privacy and others around him.

By discriminating relative location information between a user wearing a wireless headset and an electronic device (mobile device), and applying stereophonic technology corresponding to the determined relative location information, the user can easily locate the electronic device in a 3D space. It is possible to provide a method and system to enable it.

A location search method performed in a location search system includes receiving sound data generated from another electronic device for which location data is to be searched; determining relative location information with respect to the other electronic device based on the electronic device using the received sound data; and reproducing stereophonic sound information generated based on the determined relative location information to the electronic device.

The receiving may include receiving sound wave data of an audible frequency or higher range generated from the other electronic device, and the determining of the relative location information may include analyzing the received sound wave data to obtain the sound wave data from the other electronic device. It may include determining relative positional information of and.

Determining the relative location information may include determining the relative positional information relative to other electronic devices based on a difference in volume of sound wave data reaching both ears of a user wearing the electronic device in a three-dimensional space (Interaural Level Difference; ILD). A step of calculating a difference value of volume corresponding to the location information may be included.

The determining of the relative location information may include determining the relative location information of the electronic device relative to other electronic devices based on an interaural time difference (ITD) of sound wave data reaching both ears of a user wearing the electronic device in a three-dimensional space. A step of calculating a time difference value corresponding to the location information may be included.

The determining of the relative location information may include determining relative location information with the other electronic device using a Head-Related Transfer Function (HRTF) with respect to sound wave data generated from the other electronic device in a 3D space. steps may be included.

The reproducing may include applying a 3D stereoscopic sound technology corresponding to the determined relative location information to the reproduced sound output to the electronic device.

The reproducing may include generating sound information by applying a volume difference value obtained by calculating relative positional information of the identified other electronic device to a predetermined alarm sound.

The reproducing may include generating sound information by applying a time difference value obtained by calculating relative position information with another electronic device determined above to a predetermined alarm sound.

The electronic device includes a wireless earphone or wireless headset capable of receiving sound data generated from the other electronic device, and the electronic device to search for the location data is a portable device that generates sound wave data in a sound range higher than an audible frequency. device may be included.

The reproducing may include providing location information of the electronic device through voice guidance (eg, 3 meters in the 10 o'clock direction).

The location search system includes a sound receiver for receiving sound data generated from other electronic devices to search for location data; a location determination unit determining relative location information with respect to the other electronic device based on the electronic device using the received sound data; and a sound reproducing unit for reproducing stereophonic sound information generated based on the determined relative position information to the electronic device.

Users of wireless headsets can easily determine the location of the electronic device by using stereoscopic sound information containing spatial information while reproducing the sound they are listening to (music or the voice of the other party) as it is.

Since a sound signal outside the audible frequency band is used between the wireless headset and the electronic device, there is an advantage in that a desired purpose can be performed without disturbing neighbors.

1 is an example for explaining a path in which sound data generated from an object in a 3D space reaches listener's two ears in one embodiment.
2 is a block diagram for explaining the configuration of a location search system according to an embodiment.
3 is a flowchart illustrating a location search method in a location search system according to an exemplary embodiment.
4 is an example for explaining a method of determining relative location information according to an embodiment.

Hereinafter, an embodiment will be described in detail with reference to the accompanying drawings.

1 is an example for explaining a path in which sound data generated from an object in a 3D space reaches listener's two ears in one embodiment.

In the embodiment, when a user wears an electronic device (eg, a wireless headset such as a Bluetooth earphone or a headphone), an operation of audibly notifying the location of other nearby electronic devices (eg, a smartphone) will be described. When a user needs to determine the location of another electronic device while wearing an electronic device, it can be reproduced by overlaying the sound the user hears with 3D sound technology so that the location of the other electronic device can be determined through an alarm sound. there is.

The location search system can be operated in electronic devices and other electronic devices that seek location data. For example, the location search system may be driven in the form of a program or app in an electronic device and other electronic devices.

The electronic device and the other electronic device may be a fixed terminal implemented as a computer system or a mobile terminal. Electronic devices and other electronic devices, for example, headsets, earphones, smart phones, mobile phones, navigation devices, computers, laptop computers, digital broadcasting terminals, personal digital assistants (PDAs), portable multimedia players (PMPs), and tablets There are PCs, game consoles, wearable devices, internet of things (IoT) devices, virtual reality (VR) devices, augmented reality (AR) devices, and the like. In embodiments, an electronic device and the other electronic device may refer to one of a variety of physical computer systems capable of communicating with other electronic devices and/or servers over a network using a substantially wireless or wired communication method. there is. In the embodiment, it is assumed that the electronic device is a wireless headset and the other electronic device is a portable device (eg, a smart phone).

In detail, the location search system can audibly inform a user (a wireless headset worn by the user) of location data of other electronic devices whose location data is unclear in the 3D space. In other words, when the user wants to search for the location of another electronic device, after determining the location data of the other electronic device, the alarm sound generated by the electronic device can be pre-processed so that the alarm sound can be heard at the corresponding location. This basically determines the time difference (Interaural Time Difference; ITD) and the volume difference (Interaural Level Difference; ILD) for the sound data heard from both ears, and the relative position of objects in the 3D space relative to the user. The purpose of this is to enable the user to easily find the position of the electronic device aurally by using the corresponding Head-Related Transfer Function (HRTF).

As shown in FIG. 1, when an object (eg, another electronic device) exists in a 3D space, sound data generated from the object arrives at both ears with a difference in volume and a time difference between the sounds. This becomes very important information for the user to determine the location data of the object. In addition, the relative position of the object in the acoustic characteristics of the sound arriving at both ears may vary depending on the anatomical structure of the head, which is called a head transfer function.

로 정의한다고 하면 해당하는 소리가 사용자의 양쪽 귀에 도달하는 경우의 신호는 다음과 같이 표현이 가능하다.If the signal of the sound generated by the electronic device

If it is defined as , the signal when the corresponding sound reaches both ears of the user can be expressed as follows.

(1a)

(1a)

(1b)

(1b)

은 3차원 공간 내에서 전자 기기의 위치로부터 각각 왼쪽, 오른쪽 귀에 해당하는 위치로 3차원 공간 내의 음파가 전달되는 Head-Related Impulse Response(HRIR),

는 컨볼루션,

과

은 양쪽 귀에 도달하는 배경 잡음을 의미한다. HRIR 은 공간 내의 위치 정보를 판별하게 하는 음향 정보이며 이것의 푸리에 변환을 한 것이 머리전달함수(HRTF) 이다. 사용자는 발생한 소리의 신호

가 3차원 공간을 통해서 전달 된 후 양쪽 귀에 도달하는

과

의 차이를 통해 해당 물체의 3차원 공간에서의 위치를 판단할 수 있다.From here

is a Head-Related Impulse Response (HRIR) in which sound waves in a 3-dimensional space are transmitted from the position of the electronic device to the positions corresponding to the left and right ears, respectively, in the 3-dimensional space;

is the convolution,

class

denotes the background noise reaching both ears. HRIR is acoustic information that determines positional information in space, and its Fourier transform is the Head Transfer Function (HRTF). The user receives the signal of the generated sound

reaches both ears after being transmitted through the three-dimensional space

class

The position of the corresponding object in the 3D space can be determined through the difference in .

양쪽 귀에 도달하는 신호가 음량 감쇄 및 시간차로만 구성된다면 HRIR 은 각각 다음과 같이 표현 가능하다.Assuming there is no background noise

If the signal reaching both ears consists only of volume attenuation and time difference, HRIR can be expressed as follows.

(2a)

(2a)

(2b)

(2b)

은 각각 왼쪽 귀에 도달하는 음량 감쇄 및 도달 시간을 의미하며

은 각각 오른쪽 귀에 도달하는 음량 감쇄 및 도달 시간을 의미한다. 이 경우 각 HRIR 의 푸리에 변환을 하게 되면 다음과 같이 HRTF 를 구할 수 있다.From here

denotes the volume attenuation and arrival time reaching the left ear, respectively.

denotes the volume attenuation and arrival time reaching the right ear, respectively. In this case, if the Fourier transform of each HRIR is performed, the HRTF can be obtained as follows.

(3a)

(3a)

(3b)

(3b)

는 푸리에변환의 주파수에 해당된다.From here

corresponds to the frequency of the Fourier transform.

및 음량의 차이(ILD)

(데시벨 기준으로는

(dB)) 를 통해서 사용자가 판별 가능하다. 주어진 두 개의 신호를 기반으로 ITD 및 ILD 를 판별하는 기술은 다수 존재한다.Therefore, the position of the device is determined by the difference in time it takes for sound information to reach both ears (ITD), i.e.

and difference in loudness (ILD)

(in terms of decibels)

(dB)) can be determined by the user. There are many techniques for determining ITD and ILD based on two given signals.

In this way, the head transfer function means that the same sound data is generated in all directions and the frequency response according to the direction is measured and defined as a three-dimensional function. When sound is generated from a sound source and transmitted to a listener's ear, the sound passes through various paths. For example, assuming that a sound source generated indoors is heard, there is a direct sound transmitted from the sound source to the listener's ear in the shortest distance. In addition, there is an early reflected sound that is primarily reflected from a floor, a wall or a ceiling, and a later reflected sound that is reflected, absorbed, diffracted, or diffused. A transmission path for this may be obtained, for example, through an impact sound or a specific recording according to a method for measuring room acoustics. However, in the case of a microphone, it is assumed to be a sound pickup point and only the transmission path by an indoor object needs to be defined, but in the case of a real listener, since there is a body and an external ear, the influence or transmission path thereof must also be considered. This is the head transfer function. The head transfer function may vary from person to person because it includes effects on the shape of the body, such as the chest, shoulders, outer ear, head, and nose. In the embodiment, a standard head transfer function obtained by obtaining and converting result values of head transfer functions of numerous people into data may be used.

As shown in FIG. 1, this is an example of a path through which sound data generated from an object in a 3D space reaches the listener's ears. In the embodiment, it is possible to provide the location data of another electronic device whose location is unknown to the user through a stereophonic sound technique by utilizing a difference in time, a difference in volume, a head transfer function, and the like for sound data to determine the location data of the object. there is.

로 정의하고 HRTF 는 ITD

와 ILD

로 구성된다면 무선헤드셋의 왼쪽, 오른쪽 에서 각기 재생하게 될 신호

과

는 다음과 같다.If the signal of the beep sound to be reproduced to determine the location of the mobile device

, and HRTF is ITD

and ILD

If it consists of , the signals to be reproduced on the left and right sides of the wireless headset respectively

class

is as follows

(4a)

(4a)

(4b)

(4b)

로 표현한다면 무선헤드셋에서 재생하게 될 신호

과

는 다음과 같다.If the HRTF includes information other than ITD and ILD, HRIR corresponding to the inverse Fourier transform of the corresponding HRTF is

If expressed as , the signal that will be reproduced by the wireless headset

class

is as follows

(5a)

(5a)

(5b)

(5b)

Figure 2 is a block diagram for explaining the configuration of a location search system according to an embodiment, Figure 3 is a flow chart for explaining a location search method in the location search system according to an embodiment.

The processor of the location search system 100 may include a sound receiver 210 , a location determiner 220 and a sound player 230 . Components of such a processor may be representations of different functions performed by the processor according to control instructions provided by program codes stored in the positioning system. The processor and components of the processor may control the location search system to perform steps 310 to 330 included in the location search method of FIG. 3 . In this case, the processor and components of the processor may be implemented to execute instructions according to the code of an operating system included in the memory and the code of at least one program.

The processor may load a program code stored in a file of a program for a location search method into a memory. For example, when a program is executed in the location search system, the processor may control the location search system to load a program code from a file of the program into a memory under the control of an operating system. At this time, each of the sound receiving unit 210, the location determining unit 220, and the sound reproducing unit 230 executes a command of a corresponding part of the program code loaded into the memory to execute subsequent steps 310 to 330. There may be different functional representations of the processor.

In step 310, the sound receiving unit 210 may receive sound data generated from another electronic device to search for location data. The sound receiving unit 210 may receive sound wave data of a sound range higher than an audible frequency generated from other electronic devices. The sound receiver 210 may store the received sound wave data.

In step 320, the location determination unit 220 may determine relative location information with another electronic device for which location data is to be searched based on the electronic device. The location determining unit 220 may determine relative location information with other electronic devices by analyzing the received sound wave data. For example, the location determining unit 220 may determine relative location information with other electronic devices by comparing the received sound wave data with a preset value.

Referring to FIG. 4, it is a flowchart for explaining a method of determining relative location information. In 400, sound wave data of a sound range higher than the audible frequency generated from other electronic devices may be received. At this time, request information for location search may be transmitted from the electronic device to another electronic device. Accordingly, sound wave data may be generated in response to request information about the electronic device from another electronic device. For example, the location determining unit 220 may select one method for determining relative location information with other electronic devices. For example, a method for determining relative location information selected by a user may be received, and the relative location information may be calculated using the received method. Alternatively, a method for randomly determining relative location information may be selected by a computer/program.

In operation 410 , the location determining unit 220 may calculate an Interaural Level Difference (ILD) of sound wave data reaching both ears of the user wearing the electronic device in a 3D space. For example, the difference in volume means a difference in intensity (intensity) of sound pressure between the two ears. It is caused by the difference in the path to the two ears according to the location of the sound source and the difference in the amount of reduction due to the path difference. The location determining unit 220 may calculate a volume difference corresponding to relative location information with other electronic devices based on a difference in volume of sound wave data reaching both ears of the user wearing the electronic device.

In operation 420, the location determination unit 220 may calculate an Interaural Time Difference (ITD) of sound wave data reaching both ears of the user wearing the electronic device in a 3D space. The difference in time means the difference in time between the two ears. The location determination unit 220 may calculate a time difference value corresponding to relative location information with other electronic devices based on a time difference between sound wave data reaching both ears of the user wearing the electronic device.

In 430, the location determining unit 220 may determine relative location information with other electronic devices using a Head-Related Transfer Function (HRTF) for sound wave data generated from other electronic devices in a 3D space. there is. The head transfer function refers to the characteristics of an acoustic signal received from the ear according to the shape of the head and ears. A value obtained by obtaining left and right impulse responses for each position at regular intervals between an azimuth angle of 360 degrees and an altitude angle of 180 degrees using a dummy head microphone modeling the human auditory organ can be derived. The head transfer function can perceive stereophonic sound based on the phenomenon in which the characteristics of complex paths, such as diffraction on the surface of the head and reflection on the curves of the auricle, change according to the direction of sound arrival, rather than simple path differences of difference in volume and time. can When the head transfer function in each direction has a unique characteristic, it is possible to generate stereophonic sound using this.

In step 330, the sound reproducing unit 230 may reproduce stereophonic sound information generated based on the determined relative location information to the electronic device. The sound reproducing unit 230 may apply a 3D stereo sound technology corresponding to the determined relative location information to the reproduced sound output to the electronic device. The sound reproducing unit 230 may generate stereophonic sound information by applying a volume difference value obtained by calculating relative location information with other electronic devices determined in advance to a predetermined alarm sound. The description will be made on the assumption that the predetermined alarm sound is a beep sound. A beep sound is a sound called "beep" and is generally a short signal tone with a high pitch, produced by a computer or other machine. In addition, a predetermined alarm sound may be set differently according to each electronic device. For example, the sound reproducing unit 230 may generate stereophonic sound information by applying a volume difference value to a beep sound. For example, the greater the difference in volume, the greater the distance between the electronic device and other electronic devices, and the smaller the difference in volume, the shorter the distance between the electronic device and other electronic devices. At this time, a pre-processing process for applying the difference value of volume to the beep sound may be performed. The preprocessing process may include noise removal, a process of changing a format for applying a difference value of volume to a beep sound, and the like. As the pre-processing process is performed, stereo sound information may be generated by applying a volume difference value to the beep sound. At this time, attribute information (size, frequency, speed) of the stereophonic sound information may be adjusted according to the volume difference.

The sound reproducing unit 230 may generate sound information by applying a time difference value obtained by calculating relative position information with other electronic devices determined in advance to a predetermined alarm sound. For example, the sound reproducing unit 230 may generate stereophonic sound information by applying a time difference value to a beep sound. For example, a larger time difference value may mean a longer distance between an electronic device and another electronic device, and a smaller time difference value may mean a shorter distance between an electronic device and other electronic devices. At this time, a preprocessing process for applying the time difference value to the beep sound may be performed. The preprocessing process may include noise removal, a process of changing a format for applying a time difference value to a beep sound, and the like. As the preprocessing process is performed, stereophonic sound information may be generated by using a time difference value between the beep sound and the beep sound. At this time, attribute information (size, frequency, speed) of the stereophonic sound information may be adjusted according to the time difference value.

Also, the sound reproducing unit 230 may provide location information of the electronic device through voice guidance. For example, it is possible to inform by voice that there is location information of the electronic device at 3 meters in the 10 o'clock direction.

According to the embodiment, a user wearing an electronic device reproduces stereophonic sound information containing spatial information (eg, left, right, top, bottom, etc.) while reproducing sound (eg, call, music) being heard as it is It can be used to determine the location data of other electronic devices. In addition, the location data of other electronic devices may be tracked by periodically or non-periodically receiving sound wave data generated from other electronic devices, and other electronic devices may be found according to the tracked location data.

In addition, when a predetermined alarm sound is set differently for each electronic device, a plurality of stereoscopic sound information related to a plurality of other electronic devices may be output as playback sound while reproducing the sound that the user wearing the electronic device is listening to. there is. Accordingly, it is possible to recognize a plurality of different electronic devices at the same time.

The devices described above may be implemented as hardware components, software components, and/or a combination of hardware components and software components. For example, devices and components described in the embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA) , a programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions. A processing device may run an operating system (OS) and one or more software applications running on the operating system. A processing device may also access, store, manipulate, process, and generate data in response to execution of software. For convenience of understanding, there are cases in which one processing device is used, but those skilled in the art will understand that the processing device includes a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that it can include. For example, a processing device may include a plurality of processors or a processor and a controller. Other processing configurations are also possible, such as parallel processors.

Software may include a computer program, code, instructions, or a combination of one or more of the foregoing, which configures a processing device to operate as desired or processes independently or collectively. You can command the device. Software and/or data may be any tangible machine, component, physical device, virtual equipment, computer storage medium or device, intended to be interpreted by or provide instructions or data to a processing device. can be embodied in Software may be distributed on networked computer systems and stored or executed in a distributed manner. Software and data may be stored on one or more computer readable media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer readable medium. The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination. Program commands recorded on the medium may be specially designed and configured for the embodiment or may be known and usable to those skilled in computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. - includes hardware devices specially configured to store and execute program instructions, such as magneto-optical media, and ROM, RAM, flash memory, and the like. Examples of program instructions include high-level language codes that can be executed by a computer using an interpreter, as well as machine language codes such as those produced by a compiler.

As described above, although the embodiments have been described with limited examples and drawings, those skilled in the art can make various modifications and variations from the above description. For example, the described techniques may be performed in an order different from the method described, and/or components of the described system, structure, device, circuit, etc. may be combined or combined in a different form than the method described, or other components may be used. Or even if it is replaced or substituted by equivalents, appropriate results can be achieved.

Therefore, other implementations, other embodiments, and equivalents of the claims are within the scope of the following claims.

Claims (11)

In the location search method performed in the location search system,
receiving sound data generated from another electronic device to search for location data;
determining relative location information with respect to the other electronic device based on the electronic device using the received sound data; and
Reproducing stereophonic sound information generated based on the determined relative positional information on the electronic device
Location search method comprising a. According to claim 1,
In the receiving step,
Receiving sound wave data of a sound range higher than the audible frequency generated from the other electronic device
including,
Determining the relative location information,
Analyzing the received sound wave data to determine relative positional information with the other electronic device
Location search method comprising a. According to claim 2,
Determining the relative location information,
Based on the difference in volume (Interaural Level Difference; ILD) of sound wave data reaching both ears of the user wearing the electronic device in a three-dimensional space, a volume difference value corresponding to relative positional information with the other electronic device steps to calculate
Location search method comprising a. According to claim 2,
Determining the relative location information,
Based on the interaural time difference (ITD) of sound wave data reaching both ears of the user wearing the electronic device in a three-dimensional space, a time difference value corresponding to relative positional information with the other electronic device steps to calculate
Location search method comprising a. According to claim 2,
Determining the relative location information,
Determining relative position information with other electronic devices using a Head-Related Transfer Function (HRTF) for sound wave data generated from other electronic devices in a three-dimensional space.
Location search method comprising a. According to claim 1,
In the regeneration step,
Applying a 3D stereoscopic sound technology corresponding to the determined relative location information to the playback sound output to the electronic device.
Location search method comprising a. According to claim 6,
In the regeneration step,
Generating sound information by applying a difference in volume obtained by calculating relative positional information of the determined other electronic device to a predetermined alarm sound
Location search method comprising a. According to claim 6,
In the regeneration step,
Generating sound information by applying a time difference value obtained by calculating relative positional information with other determined electronic devices to a predetermined alarm sound
Location search method comprising a. According to claim 1,
The electronic device includes a wireless earphone or wireless headset capable of receiving sound data generated from the other electronic device,
The electronic device to search for the location data includes a portable device that generates sound waves of an audible frequency or higher.
Location search method characterized in that. According to claim 1,
The regeneration step is
Providing location information of an electronic device through voice guidance
Location search method comprising a. In the positioning system,
a sound receiver for receiving sound data generated from other electronic devices to search for location data;
a location determination unit determining relative location information with respect to the other electronic device based on the electronic device using the received sound data; and
A sound reproducing unit for reproducing 3D sound information generated based on the determined relative position information to the electronic device.
Positioning system including a.

Images