WO2015056996A1 - Location tracking system using sensors provided in smartphone and so forth - Google Patents

Abstract

The present invention relates to a location tracking system and, more particularly, to a location tracking system for measuring a sound delay in the air to determine the location of a computer, which is another electric device, and using the location as an input value. A location tracking system of the present invention comprises a first electric communication device, the first electric communication device comprising: a first speaker for emitting or outputting sound; a communication unit for performing communication with a second electric communication device; a storage unit for storing sound source data regarding sound and at least one piece of software; and a control unit for controlling the first speaker to output the sound source data as sound and for receiving the location information regarding the second electric communication device, which is associated with the delay of the output sound, from the second electric communication device via the communication unit, thereby using the received location information as an input value of the software.

Description

Location tracking system using sensors in smartphones

The present invention relates to a location tracking system using sensors provided in a smartphone, and more particularly, by using a sensor provided in a smartphone or the like used as an input value by determining a position of another telecommunication device by calculating a time of a sound delay in the air. A location tracking system.

The most common pointing devices for desktop and notebook PCs today are the mouse and trackpad. The mouse is connected to the PC by wire or wireless, but mechanically separated, the user can input the pointing device by dragging by hand on a large table.

The trackpad is also implemented with dedicated hardware such as capacitive touch sensors, and is commercialized as a software application that controls the mouse cursor of a wirelessly connected PC by receiving the movement of a user's finger pressing the touch screen of a general-purpose smartphone. There is also. However, such a device can only input a finger into a narrow touch screen space of a smartphone like a trackpad, and is inconvenient compared to a mouse that is held by a hand and drags on a large surface.

In addition to trackpads, dedicated hardware such as 3d mice and game controllers that input many degrees of freedom in spatial movement are also commercialized. The three-dimensional object has six degrees of freedom: displacement of three independent axis components and rotation of three independent axis components. For example, a 3d connexion mouse, as shown in U.S. Patent 7215323, can hold a user's hand and make a linear displacement in the three-dimensional direction, or by providing a knob that can be rotated in a three-dimensional axis. You can input the movement of at the same time. Along with the 3D mouse, there are products such as Wii remote and Xbox Kinect that can be used to reflect human movement into the game. Despite their excellent performance in motion recognition, these game interfaces require complex sensor and circuit configurations, need for power supply, and use light to operate when the object to be sensed is hidden from the line of sight with the sensor. It does not have the disadvantage. In addition, it can be used only in a space with dedicated hardware. Especially, the movement in close space is practically impossible to use because the sensing object is well hidden from the camera or IR sensor, and a large enough space of several meters is required for input. .

Attempts have been made to solve these three-dimensional inputs using software alone, using various acceleration sensors installed in existing smartphones. In the project named Phone Point Pen at Duke University in the US, the traces of letters are drawn on the smartphone in the air, and the smartphone's gyroscope and accelerometer recognize it and turn it into a software stroke. InvenSense's “Motion Processing” report also includes sensor fusion that removes noise and performs appropriate calculus calculations by referring to the accelerometer's gyroscope, accelerometer, and magnetic field sensor (e-compass). It claims that not only changes in velocity but also angles and displacements in six degrees of freedom are possible. However, the absolute displacement obtained by integrating the acceleration twice in an environment where it is difficult to accurately distinguish between the rotational acceleration and the linear acceleration, and a lot of noise due to small movements caused by tilting or shaking is so large that the error is large enough to be used as the input of the pointing device. It is impossible to achieve accuracy.

When using a small sensor mounted on a current smartphone, the absolute value of the direction angle can be accurately measured using an accelerometer and a magnetic field sensor in a constant gravity and magnetic field generated by the earth when the phone is stationary. Even when the smartphone is moving, the absolute value of the three-dimensional direction angle pointed by the phone can be accurately measured by filtering the error and the cumulative noise by further using the value measured by the phone's gyroscope. That is, the angle information about the direction can be measured accurately enough by the sensor built into the phone, while the information about the linear distance cannot be measured accurately.

Meanwhile, ultrasonic pens are commercially available as a pointing device for inputting a position by sound waves. In other words, Pentel Co., Ltd. manufactures and sells a stylus pen for recording contents written on an arbitrary plane on a computer. When the pen tip is pressed, the pen generates ultrasonic waves and infrared rays. Sensing the infrared and ultrasonic waves generated by the pen through the optical sensor and two microphones installed on the surface of the sensor module mounted on the plane, and measuring the distance when the ultrasonic waves reach the microphones at the two locations from the time of the infrared radiation. From this, the pen tip finds out where the paper was pressed and records it so that it can be viewed on a computer. Although the pointing device is convenient to use, it is popularized due to complicated management such as a separate device such as a pen and sensor module other than a PC or a smartphone, carrying it separately, and a battery replacement. It wasn't. Moreover, the inside of the pen uses a film speaker to generate ultrasonic pulses without clicking noise, and the microphone and peripheral circuits are also composed of dedicated hardware for detecting high frequency ultrasonic waves, so this technique cannot be applied to smartphones. .

The conventional method of measuring the distance between a smartphone and a speaker installed in a room is to measure the time at which the computer generates sound through a sound source and reaches the acoustic sensor that is spatially separated from the sound at this time. Write a way to multiply the rate of spread in air. In order to apply this method, the clock must be perfectly synchronized between the sound source and the acoustic sensor so that the elapsed time of the sound can be accurately measured.The number of dimensions is needed to know the components of the 2D or 3D axis, not just the one-dimensional distance. Sound sources or sensors should be installed away from each other. If the sound-generating side and the sound-sensing side are not synchronized in the three-dimensional space, TDOA (time difference of arrival) is obtained by using four mutually fixed sound sources or sensors, one more than the number of dimensions. A method of calculating the position in the dimensional space is known. For example, Patrick Lazik of Carnegie Mellon University (CMU) of the United States can learn how to track the location of a smartphone using ultrasonic wave parallax (TDOA) from surrounding speakers without using additional hardware in a mobile device such as a smartphone. In his paper, “Indoor Pseudo-ranging of Mobile Devices using Ultrasonic Chirps”. The method based on this paper uses 4KHz (= 24KHz-20KHz) ultrasound to solve the problem of clicking noise that is heard in the human ear when the pulse starts due to the physical constraints of most speakers when a simple single frequency pulse is generated. A method of loading the chirp with increasing frequency on the signal band is taken. The accuracy of the location tracking limited by the 4KHz bandwidth used as the chirp is shown to be 4.25cm by the well-known signal processing formula. Although this method may reduce the size of clicking noise depending on the speaker, compared to the conventional method of sending and tracking simple ultrasonic pulses, most of the speakers commonly used in computers and mobile devices still hear the clicking noise clearly. Because of the large error of 4.25 cm, it is impossible to use it as a three-dimensional or planar mouse.

As described above, it is possible to measure the direction angle and acceleration of a smartphone through sensors related to movement such as accelerometer, magnetic field sensor, gyroscope, etc. built in a mobile device such as a conventional smartphone, but linear position change (displacement) is necessary accuracy Cannot be secured. As described in Patrick Lazik's paper at Carnegie Mellon University, we can measure the linear position of a mobile device by measuring the time difference of arrival of chirp sound generated by several speakers, but 1) most generally Chirp signal is still too high clicking noise in the used speaker, and 2) the speaker has to be located in four independent places that are sufficiently separated in three dimensions, and 3) the resolution of the measurement (conventional smartphone) An error of 4.25 cm occurs.

The present invention generates sound waves without clicking noise in the first telecommunication device, and measures the sound waves generated by the first telecommunication device by using the microphone of the limited performance of the second telecommunication device, but the position of the first telecommunication device is more accurate than the conventional method. It is an object to provide a location tracking system that allows measurement.

In addition, the present invention uses the second telecommunication device as a pointing device, such as moving the software cursor of the first telecommunication device according to the position of the movement of the second telecommunication device through accurate position measurement. An object of the present invention is to provide a location tracking system for easily manipulating a state of software operating in a first telecommunication device according to a movement of an electric device.

In addition, the present invention generates a sound through a limited number of signal sources, such as the left and right two speakers provided in the first telecommunication device, the limited number of sounds, such as the up / down microphone provided in the second telecommunication device An object of the present invention is to provide a position tracking system capable of measuring a motion in a high degree of freedom by measuring with a sensor.

In addition, the present invention is that the first telecommunication device such as a conventional smart phone, tablet, notebook PC, desktop PC, etc. already has a speaker or microphone for music or voice input and output, and has a wired and wireless communication module, so that additional measurement It is an object of the present invention to provide a location tracking system that can accurately and accurately locate a second telecommunication device simply by installing software without hardware.

It is possible to input or output sound such as speaker or microphone with CPU, memory for signal processing such as notebook, PC, tablet, smart TV, smart phone, smart glasses, smart watch, etc. Commonly referred to as 'telecommunications equipment'.

In order for the first telecommunication device to measure the position of the second telecommunication computer without any dedicated hardware, the first telecommunication device generates a sound through the speaker, which is music, background sound, and sound effect for the user. And so on. The second telecommunication device acquires ambient sound, including the sound output by the first telecommunication device through the microphone. In addition to this, the second telecommunication device receives original data (reference data) without noise regarding the sound output from the speaker of the first telecommunication device from the first telecommunication device through wired or wireless communication means such as wifi. The original data may be sent from the first telecommunication device, or may be sent by a subject having sufficient information on the sound output by the first telecommunication device among members constituting the present invention such as a cloud. The second telecommunication device uses a type of TDE (Time Delay Estimation) that compares the waveform of the sound input through its microphone with the original data about the sound transmitted through the communication means along the time axis. Measure when the microphone detects sound that matches the sound signal generated by the device. Multiplying this time delay by the speed of sound calculates the distance between the speaker of the first telecommunication device and the microphone of the second telecommunication device. When the first telecommunication device or the cloud needs information related to a location such as the distance of the second telecommunication device, the second telecommunication device uses the wired / wireless communication means to determine the TDE information measured by the subject or the owned information. Location information may also be transmitted. The first telecommunication device may generate an ultrasonic pulse, but may continuously output a sufficiently continuous sound that is not in the form of a pulse to avoid clicking noise, which is a problem in the conventional ultrasonic pulse method. In addition, when performing ultrasonic measurement using a microphone commonly used in general mobile devices, the frequency bandwidth is limited to about 4 KHz, and the resolution of position measurement of the second telecommunication device is greatly reduced. The first telecommunication device outputs sound in the sound wave area, and the second telecommunication device solves this by using all of the frequency bands of the mobile device by measuring the delay time with the TDE algorithm. That is, the first telecommunication device outputs a background sound, an effect sound, etc. used in a music, voice, game, etc. that can be heard by the user through a speaker, and the second telecommunication device obtains it through a microphone, The TDE algorithm is applied to determine the delay time due to the distance between the speaker and the microphone against the original data of the generated sound. This greatly increases the frequency bandwidth of sound waves to 24KHz (= 24KHz-0KHz), which is greater than 4KHz (= 24KHz-20KHz), so that the positioning accuracy of the second telecommunication device is better than that of Patrick Lazik's method, which used only conventional ultrasonic bands. It is much more precise. In the silent section where no sound should enter the human ear, the first telecommunication device generates sound in the ultrasonic band and the second telecommunication device measures it, which reduces the accuracy to about 4.25 cm, but the continuity of the position measurement. Can be maintained. In this case, when a discrete pulse of the sound wave of the ultrasonic band is interrupted for a predetermined period and then generates an ultrasonic wave, clicking noise is generated in most computers. Therefore, it is preferable to use an ultrasonic wave whose frequency is continuously changed.

In this case, the first and second telecommunication devices are one telecommunication device, and the state of the software is measured by measuring the relative position between the speaker and the microphone by receiving the content outputted by one telecommunication device to the wired / wireless speaker of the device. Can be reflected in (control). In addition, the first telecommunication device generates the same sound signal at a period known to the second telecommunication device without separately transmitting and receiving data on the sound generated and detected by the first and second telecommunication devices through the communication means, When the second telecommunication device outputs the sound data known to the second telecommunication device through the speaker at a predetermined time, the second computer performs a TDE that compares the sound data known by the second computer to a signal detected by the microphone to obtain information regarding a position such as a distance. It may be.

That is, whether the same telecommunication device or a separate telecommunication device, the first and second telecommunication devices send and receive the actual sound through the air through the speaker and the microphone, and the original data of the sound generated through the data communication means, etc. Separately, by performing a TDE that compares the sound detected by the acoustic sensor and the original data entered through the communication means, and measures the delay time of the sound entered into the microphone after passing through the air, based on the It is an important aspect to calculate information relating to the location, such as the distance of the second telecommunication device. In addition to the delay time of the sound, it is obvious that distance or position information can be calculated by referring to various information inputted through the acoustic sensor such as the degree of sound attenuation. At this time, since the correct data, which does not mix noise or reverberation of the sound generated by the first telecommunication device in the second telecommunication device, is transmitted through the communication means, the second telecommunication device detects between several conventional microphones. It is possible to measure the propagation delay of sound in the air more accurately than the TDE method of contrasting signals. In this case, the measurement can be more accurately performed by referring to the characteristics of the speaker of the first telecommunication device, the characteristics of the sound source of the second telecommunication device, the characteristics of the acoustic sensor, the temperature, and the pressure of the transmission medium.

The time from the generation of sound in the first telecommunication device to the detection of sound in the second telecommunication device only if the time or clock of the first and second telecommunication device or the clock (time of the built-in timer) is synchronized with each other sufficiently precisely. You can take Synchronization of the clock may be performed by placing the second telecommunication device at a position where the distance is known and fixed to the first telecommunication device such as in front of the speaker of the first telecommunication device and listening to the sound generated by the first telecommunication device. Can be done. In addition, the first and second telecommunication devices may be synchronized over a data network using a precision time protocol, or the like, and the transmission time and reception time of the light may be substantially the same as they transmit and receive light such as infrared rays with little delay due to the medium. It may be synchronized using.

When two speakers are mounted at a sufficiently long distance from each other in the first telecommunication device and generate sounds that are distinguished from each other, as described above, the second telecommunication device listens to the sounds of one micro-two speaker. The distance between the two speakers and the microphone is obtained by performing a multi-source TDE on the second source, and if the clocks of the first and second telecommunication devices are synchronized in the same manner as described above, the second telecommunication is more detailed than the simple distance. You can get information about the location of the device. A microphone, which can be described as a point in space, is a three-degree of freedom movement in three-dimensional space, so if the distance between this microphone and two speakers is known and defined, the microphone is one-dimensional in space. (= 3 degrees of freedom-2 limitations) You can see that it is located at any position on a figure, that is, a curve or a straight line. If two speakers are at a similar height from the floor, and people assume that the height from the floor where they are carrying the second telecommunication device is usually the same, from this information the three-dimensional image of where the microphone of the second telecommunication device is located It can be limited to an approximate value.

In addition, when the three speakers are installed in a manner such that the three speakers are not far enough apart from each other on one plane, and all the speakers generate sound, the position of the microphone of the second telecommunication device is changed to three. It can be limited to one point in the dimension.

Sounds generated from a plurality of speakers (sound sources) must be sufficiently distinguished even if they are superimposed on a microphone so that the accurate distance can be measured by effectively executing the multiple sound source TDE algorithm. In the case of stereo speaker output of general sound, the left and right speaker outputs are similar to each other, making it difficult to apply a multi-source TDE algorithm. In this case, the user may intentionally apply different sounds to the left and right speakers, or periodically change the left and right speakers, or output the sound by superimposing consecutive ultrasonic waves on both (or one) speaker outputs. You can have each speaker output a sound with a sufficiently different waveform without disturbing your ears.

If it is not desirable to synchronize the clock between the first and second telecommunications devices, for each of the above cases, one more speaker of the first telecommunications device is used and the distance measurement by the time difference of arrival (TDOA) or the The method of limiting the position in space allows the position specification of the second telecommunication device to the same degree as when synchronized. In particular, by measuring the information relating to the distance between the pair consisting of each of the speaker (sound source) of the first computer and each of the microphone (sound sensor) of the second telecommunication device, the number of distances that can be measured from the limited speaker and the microphone as much as possible Secure. For example, ultrasonic waves are generated from the L, R (left, right) speakers of the notebook PC, and the U, D (up, down) microphones of the smartphone measure this sound, and thus the four distances between the LU, RU, LD, and RD pairs. Measure

If the second telecommunication device is equipped with a plurality of microphones, the second telecommunication device does not have to provide the second telecommunication device with a data source for sound generated by the speakers of the first telecommunication device as described above. By comparing and analyzing only signals obtained by a plurality of microphones, the difference in delay time at which sound arrives between the microphones may be obtained. The signal originating from the L speaker of the first telecommunications device arrives at a time difference between the L and U microphones of the second telecommunication device in most cases, providing the second telecommunication device with original data information about the sound from the L speaker. If not, the second telecommunication device can compare the waveforms detected from the U and D microphones with each other to calculate how the same sound waveform from the L speaker is reached at each microphone at a time difference. This is done using various algorithms studied by the existing SSL method. Even with respect to the sound output by the R speaker of the first telecommunication device, the parallax of when the same sound arrives at the L and U microphones can be measured using only a microphone input using a conventional TDE technique. Of course, since echo and noise exist in most environments, the second computer receives information on the sound generated from the speaker of the first computer through the wired / wireless data communication network such as wifi and the like through the L and U microphones. Compared with the signal, the delay time can be measured more accurately.

In addition, when the measurement of the direction angle of the second telecommunication device using the sensor provided in the second telecommunication device such as a smart phone, during the high degree of freedom (three-dimensional position + three-dimensional angle) movement of the second computer Since the information about the angle can be known, the number of distances that need to be measured by sound to limit the position can be reduced. For example, the measurement of the direction, roll, yaw, and pitch of a three-dimensional smartphone using accelerometers, gyroscopes, and magnetic field sensors mounted on a second telecommunication device can be performed using sound, x, y, z Because only three linear degrees of freedom are measured, the distance between two sound sources and two sensor pairs is known without the need for clock synchronization, so that the exact position and angle of the second telecommunication device in three-dimensional space are known.

In addition, in the case of a two-dimensional plane mouse, which is commonly used in a PC or a notebook, the position of the measurement target under the limited assumption that the first and second telecommunication devices are all placed on one plane (or other known two-dimensional curved surface). The degree of freedom (the number of dimensions) can be greatly reduced. In other words, if you move randomly in space, it becomes a problem of 6 dimensions that must measure the 3D position and the 3D direction angle, so the number of microphones (sound sources) or speakers (sound sensors) should be quite large, Assuming that the problem translates into a three-dimensional problem measuring only the plane coordinates x and y and, if necessary, the angle of rotation r on the plane, the problem can be solved by measuring four sound source-sensor pairs without the need for clock synchronization or sensors measuring angles inside the telecommunications equipment. Can be.

Further, either one of the first and second telecommunication devices has a plurality of speakers (sound sources) or a plurality of microphones (acoustic sensors), and further information can be obtained by reciprocating sounds between them. For example, when a first telecommunication device such as a notebook PC uses a flat mouse with a second telecommunication device such as a smartphone, sound generated by two speakers of the first telecommunication device is generated by the first telecommunication device 1 of the second telecommunication device. Acquired from the two microphones, the second telecommunication device can generate sound to its single speaker immediately upon obtaining the sound so that the microphone of the first telecommunication device can hear it. When the second telecommunication device transmits the difference between the time when the sound generated by the two speakers of the first telecommunication device reaches the microphone of the second computer to the first computer through a communication means such as wifi, the first telecommunication device 2 With the parallax when two speakers reach the microphone of the second telecommunication device, two actual values related to the sound, such as the time when one of the sounds is returned from the second telecommunication device, can be known without clock synchronization. If the rotation angle r on the plane of the second computer is grasped using a gyroscope, a magnetic field sensor, or the like for measuring the inside of the second telecommunication device, it is limited to the position on the plane x, y and the rotation angle r on the plane. It is possible to accurately measure the movement (movement path, etc.) of the second telecommunication device having three degrees of freedom.

The present invention not only receives linear movement in space of a telecommunication device held in a user's hand, but also simultaneously performs a single or multi-touch gesture in which a user presses and drags various positions of a touch screen already provided in the telecommunication device with a finger. Receiving the input, software installation only allows more convenient and intuitive input than conventional mechanical mouse positioning. In addition, according to the software status, the vibration function of the telecommunication device is driven to provide force feedback to the user who is holding the smartphone according to the situation.

The present invention generates a general sound instead of the conventional distance measuring method using a non-continuous ultrasonic pulse and calculates the arrival time of the sound using the TDE technique so that there is no clicking noise of the speaker while using a general speaker and a mouse, only ultrasonic Precisely measure position or distance by allowing the use of much louder bands than writing. This enables general-purpose mobile devices such as smartphones to be used as pointing devices such as precise mice or 3D game controllers.

The present invention simultaneously receives various gestures that a user presses and drags a finger with a finger on various positions of a touch screen already provided in a telecommunication device held in a user's hand, thereby enabling more convenient and intuitive input only by installing software. For example, the shape and type of an operation device such as a button displayed on a touch screen of a telecommunication device are changed according to the state of the application software. In addition, the vibration function of the telecommunication device is turned on to provide force feedback to the user holding the telecommunication device according to the situation.

1 is a schematic configuration diagram of a location tracking system according to the present invention.

2 is a first embodiment of the position tracking system of FIG.

3A and 3B are signal graphs processed in the first embodiment of FIG.

4 is a second embodiment of the position tracking system of FIG.

5A to 5C are signal graphs processed in the second embodiment of FIG.

6 is a third embodiment of the position tracking system of FIG. 1.

7 are signal graphs processed in the third embodiment of FIG. 6.

In the following, the present invention is described in detail through embodiments and drawings.

1 is a schematic configuration diagram of a location tracking system according to the present invention. The location tracking system includes a first telecommunication device 10 for generating sound and a second telecommunication device 20 for obtaining a sound from the first telecommunication device 10 and tracking its location. The first telecommunication device 10 may acquire the location information of the second telecommunication device 20 and control the state and operation of the currently operating software using the location information as an input value.

In the present invention, the telecommunication device may generate or acquire a signal such as a signal processing, a speaker or a microphone, such as a notebook, a PC, a tablet, a smart TV, a smart phone, a smart glasses, a smart watch, a mobile device, and the like, and communicate It refers to any device having a function.

The first telecommunication device 10 obtains an input from a user, an input unit 11 including a button type and a touch module (touch screen), a display unit 12 for displaying various information, and sound source data (original) A storage unit 13 storing data and various software, and first and second speakers 14a and 14b (eg, left speaker and right speaker) that emit or output sound and are spaced apart by a predetermined distance or more. ), A communication unit 16 for performing communication with the microphone 15 for acquiring an external sound, another communication device including the second communication device 20, and the first communication device 10. It consists of a control unit 19 that performs unique functions, operates various software, and controls the above-mentioned components. However, although the power supply unit is not described, such components are well-known techniques and description thereof has been omitted, and the input unit 11, the display unit 12, the first and second speakers 14a and 14b, and the microphone 15 are described. ) And the communication unit 16 and the like are also omitted.

The storage unit 13 stores sound source data that is sufficiently continuous sound for position tracking or includes a pulsed sound such as ultrasonic waves, sound source data for general music, background sounds, effect sounds, and the like. The storage unit 13 also stores various software operated by the control unit 19, location information received from the second telecommunication device 20, and the like.

The controller 19 performs a unique function (calling, image reproducing, data communication, operation, photographing, image processing, etc.) of the first telecommunication device 10, and tracks location information of the second telecommunication device 20. A location tracking process is performed, and by using the location information identified by the location tracking process, it is applied as a new input value of the currently operating software to perform a state change and control of the software. The location tracking process is described in detail below.

The second telecommunication device 20 obtains an input from a user, an input unit 21 consisting of a button type and a touch module (touch screen), a display unit 22 for displaying various information, and sound source data (original) Data) and a storage unit 23 for storing various software, a speaker 24 for emitting or outputting sound, and first and second microphones 25a and 25b spaced apart by a predetermined distance to obtain external sound. (For example, an up / down microphone), a magnetic field sensor 26a for measuring a magnetic field, a gyroscope 26b, an accelerometer 26c, and a vibrator 27 for vibrating the second telecommunication device 20. And perform a unique function of the communication unit 28 for communicating with other communicable electric devices including the first telecommunication device 10 and the second telecommunication device 20, and operate various software, It is composed of a control unit 30 for controlling the above-described components. However, although the power supply unit is not described, these components correspond to a well-known technology and description thereof has been omitted, and the input unit 21, the display unit 22, the speaker 24, the first and second microphones 25a and 25b are described. ), The magnetic field sensor 26a, the gyroscope 26b, the accelerometer 26c, the vibrator 27, the communication unit 28 and the like are also omitted.

The storage unit 23 stores the first and second input waveform data obtained from the first and second microphones 25a and 25b and stores sound source data for position tracking. The storage unit 23 also stores software executed in the second telecommunication device 20.

The controller 30 performs unique functions (calling, image reproducing, data communication, calculation, photographing, image processing, etc.) of the first telecommunication device 10, and controls the first or second input waveform data and the sound source data. In comparison, the first position information (x, y, z) is calculated, and the second position information (pitch, yaw, roll) is obtained using the magnetic field sensor 26a, the gyroscope 26b, the accelerometer 26c, or the like. Calculate The control unit 30 transmits the first and second location information to the first telecommunication device 10 through the communication unit 28.

FIG. 2 is a first embodiment of the position tracking system of FIG. 1, and FIGS. 3A and 3B are signal graphs processed in the first embodiment of FIG. 2.

The first embodiment corresponds to an example of a location tracking process. The first embodiment calculates the distance between the second telecommunication device 20, which is a smart phone, and the first telecommunication device 10, which is a tablet computer, without a separate hardware. It transmits to the device 10, and the first telecommunication device 10 drives software such as a game in which the display unit 12 and the first speaker 14a are demonstrated. The controller 19 outputs an image to the display unit 12 while operating the game, and outputs sound effects through the first and second speakers 14a and 14b. The second telecommunication device 20 acts as a kind of space input device in the game, such that the first speaker 14a of the first telecommunication device 10 and the first microphone 25a of the second telecommunication device 20 are used. The continuous distance change (movement) between the inputs becomes the input value of the game software, and the state of the software, such as winning or losing the game, is changed, and the result is output back to the display unit 12 and the first and second speakers 14a and 14b. do.

For the location tracking process for the distance measurement, it is necessary to continuously and accurately measure the distance S1 between the first speaker 14a and the first microphone 25a continuously. According to the conventional method, even when sounding a particular waveform that is annoying to a person's ear for distance measurement or using ultrasonic waves, there is a problem that clicking noise is heard in the first speaker 14a, so that the effect sound of the game cannot be output properly.

The controller 19 performs a continuous distance S1 between the first speaker 14a and the first microphone 25a by using sound source data including sound effects of the game in performing the position tracking process. The controller 19 reproduces the original data (reference data) O1 of FIG. 3A through the first speaker 14a and emits the sound. In FIG. 3A, the horizontal axis is a waveform in which an amplitude, which is a vertical axis, changes with time t. In addition, the control unit 19 transmits the original data O1 to the second telecommunication device 20 through the communication unit 16 at the same time as the reproduction of the original data O1 or before or after the reproduction.

The sound by the original data is advanced by the distance S1, vibrates the first microphone 25a, generates the first input waveform data as shown in FIG. 3B, and applies it to the controller 30.

The controller 30 stores the first input waveform data in the storage unit 23, receives original data O1 from the first telecommunication device 10, and stores the original data O1 in the storage unit 23. Alternatively, the storage unit 30 may already store the original data O1 through another telecommunication device.

The controller 30 compares the waveform of the first input waveform data with the waveform of the received or prestored original data O1 to determine the delay time d. For example, the controller 30 compares the first input waveform data with the original data O1 by using a generalized cross-correlation (GCC) or the like, and thus delays a certain amount of time with respect to the waveform of the original data O1. After (d), it is determined whether the same waveform appears in the first input waveform data. The controller 30 measures similarity in which the waveform of the original data O1 and the waveform of the first input waveform data overlap with each other over time, and determine that the similarity is the delay time d when the peak reaches a peak. can do. The time (0) point in FIGS. 3A and 3B is a point in time when the control unit 30 receives the original data O1 or a point in time when the control unit 19 outputs a sound or the control unit 19 transmits the original data O1. One point in time, the same applies to other waveform graphs. The controller 19 may also transmit time information for checking the time point O to the second telecommunication device 20. In addition, each of the first telecommunication device 10 and the second telecommunication device 20 includes an apparatus for receiving and transmitting a medium (such as infrared rays) without a time delay, and the first telecommunication device 10 includes a first telecommunication device 10. 2 When the infrared ray is received from the telecommunication device 20, the original data O1 may be reproduced.

Of course, the first input waveform data includes noise, echo, etc. in addition to the waveform of the original data O1. However, when a TDE (or SSL) algorithm such as GCC is applied, the first input waveform data has a delay time with only one first microphone 25a. Can be detected accurately.

Next, the distance S1 is calculated from the delay time d. That is, since the delay time d is determined as the distance S1 / (velocity of the sound), the control unit 30 controls the distance S1 by using the known velocity value of the sound through the measurement of the delay time d. You can find out exactly.

In addition, in order to more accurately measure the delay of the sound, the first telecommunication device 10 mixes a continuous ultrasonic wave that does not cause clicking noise to the sound to be heard by the user, thereby outputting it in an unobtrusive range. Intentional processing can be done for sound.

The control unit 30 transmits the location information including the distance S1 to the first telecommunication device 10 through the communication unit 28. The control unit 19 receives the positional information through the communication unit 16 and controls the software using the distance S1 included in the positional information as an input value of the operating software.

The game may be performed by using a change in which the distance S1 becomes larger or smaller, or the absolute distance value may need to be reflected in the game. In this case, in a state where the first speaker 14a and the first microphone 25a are in contact or as close as possible, the controller 19 reproduces the original data O1 through the first speaker 14a, and controls the controller. 30 receives the sound through the first microphone 25a at about the same time. Through this operation, the clock (time) of the first telecommunication device 10 and the clock (time) of the second telecommunication device 20 are synchronized. When such synchronization is made, the control unit 19 causes the reproduction of the original data O1 to be performed through the first speaker 14a at a predetermined time, and the control unit 30 sets the determined time to the origin O of time. In consideration of this, the delay time d may be calculated.

In addition, when the control unit 19 or the control unit 30 performs communication and data processing, transmission or reception of the original data O1 is delayed, so that the sound is detected through the first microphone 25a. It may be prepared later. However, the controller 30 stores the first input waveform data detected by the first microphone 25a in the storage unit 23 together with time information (receipt time of the first input waveform data) and stores the original data O1. After the reception (including the time information transmitted by the control unit 19), it is obvious that the TDE can be performed, and there is no problem in game performance unless the user is excessively late enough to feel a processing delay.

4 is a second embodiment of the position tracking system of FIG. 1, and FIGS. 5A-5C are signal graphs processed in the second embodiment of FIG. 4.

The second embodiment is an embodiment in which a flat type mouse capable of providing an input value of software of the first telecommunication device 10, which is a notebook PC, to the second telecommunication device 20, which is a smartphone, without any additional hardware. The second telecommunication device 20 has a fixed plane coordinate system O of the first telecommunication device 10 having a moving point M, which is a reference point of the position of the second telecommunication device 20 attracted to the plane by the user's hand. ) Is measured based on which coordinate (x, y) (location information) is measured, and transmitted to the first telecommunication device 10, the first telecommunication device 10 is measured position in the operating software Use information (x, y) as input.

In this case, it is assumed that the second telecommunication device 20 to be measured does not float arbitrarily and moves freely in space, but moves two-dimensionally on a known curved plane or geometrical configuration. With more restrictions, most users do not feel uncomfortable even if they assume that the first and second telecommunications devices 10 and 20 lie on the same plane and allow the user to actually use both computers on the same plane. Under this assumption, the control unit 19 stores the above-mentioned information (preliminary information) about the assumption of the position and movement in the storage unit 13, and the original data O2 through the first and second speakers 14a and 14b. , O3) as a sound, and the controller 30 acquires the second and third input waveform data through the first microphone 25a, and as shown in FIG. 2, the position tracking process (for example, TDE). Calculate the location information (x, y) by estimating the distance.

First, as described above, the first microphone 25a (moving point M) of the second telecommunication device 20 and the first speaker 14a (fixed origin O of the first telecommunication device 10). )) To synchronize the clocks of the first and second telecommunication devices 10, 20.

Next, the controller 19 emits original data O2 through the first speaker 14a, and emits original data O3 through the second speaker 14b. Here, the original data O2 and O3 have waveforms that can be distinguished in phase or magnitude (amplitude) with time, as shown in FIG. 5A. In addition, the control unit 19 transmits the original data O2 and O3 to the second telecommunication device 20 through the communication unit 16.

In FIG. 4, the moving point M is spaced apart from the first speaker 14a by a distance S2 and spaced apart from the second speaker 14b by a distance S3. By this separation, it is after the delay time d2 that the sound of the first speaker 14a reaches the first microphone 25a, as in FIG. 5B, and the sound of the second speaker 14b is the first microphone. It is after the delay time d3 that reaches (25a). However, the waveform graph actually obtained by the first microphone 25a corresponds to a waveform in which the waveform graph of FIG. 5B is superimposed, as shown in FIG. 5C, and includes other attenuation, noise, and echo. The control unit 30 obtains the superimposed waveform data in which the second and third input waveform data are superimposed from the first microphone 25a, and compares the superimposed waveform data with the original data O2 and O3 representing the waveform graph of FIG. 5A. The delay times d2 and d3 are calculated. For example, the controller 30 accurately calculates both the delay time d2 and d3 of the original data O2 and O3 and the superimposed waveform data by using a multi-source TDE technique such as a cross correlation algorithm. .

Here, the delay times d2 and d3 and the coordinates x and y of the moving point M are defined as follows.

Equation 1

In this case, the functions f1 and f2 are functions for calculating the delay time of the sound that passes according to the position (x, y) of the first microphone 25a. When the delay times d2 and d3 are known, the coordinates x and y of the moving point M are calculated through the quadratic simultaneous equations. The control unit 30 transmits the position information including the coordinates (x, y) of the moving point M to the first telecommunication device 10 through the communication unit 28 using Equation 1. The control unit 19 uses the coordinates (x, y) of the received position information to use the input value of the software in operation.

If the dragging mouse is dragged more than a plane, the mouse is moved to a position that is convenient for the user to continue to operate, and then dragging again. In order to implement a planar mouse, the second telecommunication device 20 distinguishes whether the user is in contact with the floor (plane) to be dragged (input state) or lifted from the floor to a convenient position in the air (movement state). Should be.

In order to distinguish the states, the controller 30 uses the detected values from at least one sensor among the magnetic field sensor 26a, the gyroscope 26b, and the accelerometer 26c provided in the second telecommunication device 20. In other words, if the second telecommunication device 20 is being dragged on a plane near the horizontal, the acceleration determined from the sensed value from the accelerometer 26c will be constant in the direction of gravity. Therefore, the controller 30 is placed on a general floor (plane) if the change in the detected acceleration is sustained only in a direction belonging to a certain plane rather than any direction in the three-dimensional space, otherwise it is heard if the acceleration occurs in an arbitrary direction. It can be judged as being moved in the air.

In addition, the controller 30 may also refer to an abrupt change in acceleration when the second telecommunication device 20 reaches the floor, or may detect vibrations or sounds generated by being placed on a plane or attracted to the accelerometer 26c and the first and second microphones ( 25a, 25b), it can be judged that it is attracted. In other words, after reaching the floor with vibration and having a relatively large angular acceleration or linear acceleration, if the acceleration remains constant in the direction of gravity, it is judged that the ground has started to attract. In addition, when the second telecommunication device 20 makes a specific sound from the speaker 24 facing the floor and touches the floor, the speaker 24 is blocked on the floor, and thus the characteristics of the sound are changed to determine whether the speaker 24 is attracted to the floor. It may be. These different conditions can be used together.

In addition, by using the touch screen included in the input unit 21 of the second telecommunication device 20 as a button function of a mouse, a user clicks the touch screen with a finger or presses the touch screen to hold the second telecommunication device 20. It is possible to implement the existing mouse button click and drag & drop functions by receiving input such as releasing the touch screen and releasing the touch screen. The control unit 30 also transmits an input value from the input unit 21, which is a touch screen, to the first telecommunication device 10 through the communication unit 28, and the control unit 19 transmits the received input value and position information to the software. Use as input value of. In addition, various user interfaces may be provided by referring to an input such as swiping a touch screen with a finger or using a multi-touch function to click or scrape with multiple fingers.

FIG. 6 is a third embodiment of the position tracking system of FIG. 1, and FIG. 7 is signal graphs processed in the third embodiment of FIG.

In the third embodiment, the smart TV corresponding to the first telecommunication device 10 is located in the spaces C1 (0,0,0) of the smart phone, which is the second telecommunication device 20 adjacent thereto, and (C2 (x). , y, z)) is used as an input of software operating in the first telecommunication device 10. This makes it possible to intuitively change the position and orientation of the three-dimensional mouse cursor 18a of the software output to the display unit 12 in the form of an image, and operate the various software objects 18b. ) Is determined according to the input waveform data obtained through the first and second speakers 14a and 14b. As the first and second telecommunication devices 10 and 20 transmit and receive sound to track the position of the second telecommunication device 20 with respect to the first telecommunication device 10, a delay time measurement in which sound is detected ( TDE) to measure the distance (D1-D4). That is, two first and second microphones of the adjacent second telecommunication device 20 are generated by generating sound through the two first and second speakers 14a and 14b included in the first telecommunication device 10. The control unit 30 calculates when the sounds 25a and 25b detect this sound. The delay time for the sound generated by the first speaker 14a to reach the first and second microphones 25a and 25b is determined by the distance (the distance between the first speaker 14a and the first and second microphone pairs 25a and 25b). D1, D3). The second telecommunication device 20 applies the conventional TDE algorithm used for the microphone array to the time series data of the sound detected by the two first and second microphones 25a and 25b as it is, the first speaker as one sound signal source. It is possible to calculate that the same sound from the furnace 14a is obtained with some time difference in each of the first and second microphones 25a and 25b. Similarly, the delay time for the sound generated by the second speaker 14b to reach the first and second microphones 25a and 25b is also the second speaker 14b and the first and second microphones 25a and 25b. Determined by the distance (D2, D4) of the pair. The second telecommunication device 20 applies the conventional TDE algorithm used for the microphone arrangement to the time series data of the sound detected by the two first and second microphones 25a and 25b, as a second speaker as one sound signal source. It is possible to calculate what time difference the same sound from 14b is caught in each of the first and second microphones 25a and 25b. In particular, since the signal sources are two first and second speakers 14a and 14b, it is preferable to use a TDE for a conventional multi-source signal source.

Sounds generated by the first and second speakers 14a and 14b may be artificially processed such that they are generated while mixing with each other or intersecting with each other so that they can be distinguished from each other even if they overlap. As described above, the second telecommunication device 20 compares only the data of the sound coming into the two first and second microphones 25a and 25b with each other to compare the two multi-sound signals of the first and second speakers 14a and 14b. You can perform a TDE on a circle. At this time, the first telecommunication device 10 transmits the original data O2 and O3 of the sound generated from its first and second speakers 14a and 14b through the communication unit 16 to the second telecommunication device ( 20), the second telecommunication device 20 is detected by overlapping two sounds from each of the first and second microphones (25a, 25b) based on the correct original data (O2, O3) for the generated sound. Performing a TDE on a waveform can estimate delay times and distances D1 through D4 much more accurately than using only the waveform detected by a conventional microphone.

Fixed to the first telecommunication device 10 and fixed to the C1 coordinate system of the X, Y, Z axes and the second telecommunication device 20 defined in space by the position and orientation angle of the first telecommunication device 10. Considering the moving C2 coordinate system (X ', Y', Z 'axes), the origin of the C2 coordinate system is located at a certain coordinate (x, y, z) of the C1 coordinate system, roll, yaw, and pitch), the software state of the first telecommunication device 10 can be changed in accordance with the position and orientation angle of the C2 coordinate system. In particular, the three-dimensional cursor 18a is driven to coincide with the position and orientation of the second telecommunication device 20 to be used for applications such as allowing the user to manipulate the software object 18a displayed on the display unit 12. As an example of an implementation for this purpose, the position of the first speaker 14a of the first telecommunication device 10 is the origin of the C1 coordinate system, and the x-axis is a direction from the first speaker 14a to the second speaker 14b. If the y-axis defines the C1 coordinate system to be on the plane of the first telecommunication device 10, the positions of the speakers 14a and 14b may be defined as in Equation 2.

Equation 2

As described above, the second speaker 14b is spaced apart from the first speaker 14a by y_14b along the y axis. Position coordinates of the two first and second microphones 25a and 25b of the second telecommunication device 20 on the C1 coordinate system position_25a = (x_ (25a), y_ (25a), z_ (25a)) And position_ (25b) = (x_ (25b), y_ (25b), z_ (25b)) is the second telecommunication device 20 or equivalently any three-dimensional position (x, y, z in which the C2 coordinate system is on the C1 coordinate system. ) And the three-dimensional direction angle (roll, yaw, pitch) can be described by the functions F_ (25a) and F_ (25b). At this time, if the Y 'axis of the C2 coordinate system is set to pass through (25a) and (25b), the yaw component in which the second telecommunication device 20 rotates Y' along the axis is 3 of (25a) and (25b). The yaw component is ignored because it does not change the coordinates in the dimension. The result is calculated by the following equation (3).

Equation 3

It is an equation of 5 degrees of freedom (= 5 variables: x, y, z, roll, pitch), in which at least five independent geometric quantities with respect to the position of the first and second microphones 25a, 25b must be measured. Five variables can be obtained: x, y, z, roll, and pitch. In the configuration of FIG. 6, sounds generated by the two first and second speakers 14a and 14b of the first telecommunication device 10 may be divided into two first and second microphones of the second telecommunication device 20. 2x2 = 4 time points t0, t1, t2, t3 detected by 25a, 25b) can be calculated | required from the clock of the 2nd telecommunication device 20. FIG. The time points t0, t1, t2, t3 are the distances D1, D2, D3, D4 determined by the two first and second speakers 14a, 14b and the two first and second microphones 25a, 25b pairs. Determined by If the clocks of the first and second telecommunication devices 10 and 20 are synchronized with each other, the time when the sound is generated from the clock of the first telecommunication device 10 and the sound from the clock of the second telecommunication device 20. Four distances, such as D1, D2, D3, and D4, can be obtained by subtracting from each other. However, in the general case where the clocks of the first and second telecommunication devices 10 and 20 are not synchronized, only three measurement values such as t1-t0, t2-t0, and t3-t0 can be measured since only the difference between four viewpoints can be measured. The difference between them is geometrically significant, and these are equations added and subtracted from the combination of Position_ (25a), Position_ (25b), Position_ (14a) and Position_ (14b) of Equations 2 and 3 above. That is, equation (4) is determined from equations (1) and (2).

Equation 4

If the second telecommunication device 20 is a device having a magnetic field sensor, an accelerometer, a gyroscope and other sensors capable of measuring the absolute value of the direction angle, the values of the roll and pitch, which are variables related to the angle, are obtained from these sensors. Whereby Equation 5 is as follows:

Equation 5

The three constraints related to the three degrees of freedom, x, y, and z, can be used to find x, y, and z through nonlinear optimization or numerical methods of solving equations. When x, y, and z are obtained and roll and pitch are obtained from the sensors 26a to 26c of the second telecommunication device 20, the relative from the first telecommunication device 10 of the second telecommunication device 20 is obtained. Both the position and the angle can be measured, through which the second telecommunication device 20 recognizes the position and the angle of the second telecommunication device 20 having 5 degrees of freedom. It can be used as a pointing device input by passing it to the wifi communication means. Since input related to a rigid body moving freely in a space of five degrees of freedom is possible, various pointing devices such as a mouse, a joystick, a three-dimensional mouse, and a game controller for a sports game can be implemented.

The first and second telecommunication devices 10 and 20 are connected to a network for transmitting and receiving data such as wifi, wifi direct, and Bluetooth, and share information about a viewpoint or direction angle of a sound to obtain the above-described position and angle, and a pointing device. Algorithms for converting to input may be performed, and such algorithms may be distributed and performed in the first and second telecommunication devices 10 and 20 or in third locations.

Most existing PCs have built-in stereo speaker output, while existing smartphones have accelerometers, magnetic field sensors, compasses, gyroscopes, and two or more microphones for noise canceling or call / video recording switching. Since built-in, Figure 2 can be built as a general-purpose computer and smartphone without the addition of hardware. In addition, the second telecommunication device 20 may be several instead of one, and since the sound is not greatly attenuated according to the distance, the second telecommunication device 20 may be measured in a large space so that a plurality of users may have a first electric device having a large screen and a high output speaker. The communication device 10 may be shared in various ways such as playing a game.

In addition, the control unit 30 also transmits the input from the input unit 21 to the first telecommunication device 10, and uses the control unit 19 as the input value of the software. For example, the present invention simultaneously receives a gesture of pressing and dragging a user's finger with various positions of a touch screen (input unit 21) that is already provided in the second telecommunication device 20 held in a user's hand. The installation allows for more convenient and intuitive input. For example, it is also possible to change the shape and type of the operating device such as a button displayed on the display unit 22 of the second telecommunication device 20 in accordance with the state of the application software.

The control unit 19 of the first telecommunication device 10 transmits a vibration command to the second telecommunication device 20 through the communication unit 16 according to the content of the game played by the first telecommunication device 10. The controller 30 may receive the vibration command through the communication unit 28, drive the vibration unit 27 to perform the vibration operation, and give force feedback to the user. In addition to the vibration command, a command such as driving the speaker 24 included in the second telecommunication device 20 or changing the brightness or color of the display unit 22 may be performed. Various feedbacks may be provided to a user according to a state of software performed in the first telecommunication device 10.

The first telecommunication device 10 is installed with software that recognizes the second telecommunication device 20 as a user input device and performs the above-described control process, and the control unit 19 operates the software to operate it. The second telecommunication device 20 is used as a user input device, and software for performing the above-described control process, such as transmitting location information to the first telecommunication device 10, is installed, and the control unit 30 provides It works by running software.

2, 4, and 6 described above, the first telecommunication device 10 is fixed and the second telecommunication device 20 is moved, but the first telecommunication device 10 is moved. It is also possible when the second telecommunication device 20 is in a fixed state. It is also possible that at least two telecommunication devices and telecommunication devices are in a moving state, respectively.

In addition, the control unit 30 of the second telecommunication device 20 calculates the location information and uses the calculated location information as an input value for the state and control of the currently operating software. You can also use the same as). In particular, in this case, the first telecommunication device 10 may be configured by only a device for simply outputting sound according to at least one or more sound source data instead of the complicated configuration shown in FIG. 1. In addition, the control unit 30 transmits the state and control changes of the software according to the location information to the first telecommunication device 10, and the control unit 19 receives the state of the software of the received second telecommunication device 20. And control changes can also be used as inputs to the status and control of the software currently running.

A location tracking process using a sound delay performed by the location tracking system discussed in the present invention, a process of generating location information, or a process of receiving location information and using the received location information as an input value of software, between telecommunication devices. The process of synchronizing a clock may be stored in a storage medium as a software (program) file, which is transferred between telecommunication devices by transmission over a network, installed in various telecommunication devices, and the same operation is performed. . That is, the processes implemented in the location tracking system may be provided as a program stored in a computer readable storage medium.

In addition, the first to third embodiments may be performed in combination with each other.

Although embodiments of the present invention have been described above with reference to the accompanying drawings, those skilled in the art to which the present invention pertains may implement the present invention in other specific forms without changing the technical spirit or essential features thereof. You will understand that. Therefore, it should be understood that the embodiments described above are exemplary in all respects and not restrictive.

Claims (16)

1. A first speaker for emitting or outputting sound;

   A communication unit for communicating with the second telecommunication device;

   A storage unit which stores sound source data of sound and at least one software;

   The first speaker is controlled to output sound source data as sound, and the position information of the second telecommunication device correlated with the delay of the output sound from the second telecommunication device is received through the communication unit, and received as an input value of software. Position tracking system comprising a first telecommunication device configured with a control unit for using the location information.
2. The method of claim 1,

   The control unit of the first telecommunication device transmits the sound source data of the output sound to the second telecommunication device through the communication unit.
3. The method of claim 2,

   The second telecommunication device includes a first microphone for acquiring sound, a communication unit for communicating with the first telecommunication device, receiving sound source data, a storage unit for storing sound source data, and a first microphone obtained from the first microphone. 1 compare the input waveform data with the received or prestored sound source data, detect a delay time of the sound, calculate a distance between the first speaker and the first microphone based on the detected delay time, and include the calculated distance And a control unit for transmitting the position information to the first telecommunication device.
4. The method of claim 3,

   And a control unit of the second telecommunication device calculates a distance based on time information included in the received sound source data.
5. The method of claim 2,

   The first telecommunication device includes a second speaker sufficiently spaced from the first speaker, and outputs first and second sound source data having different waveforms as first and second sounds through the first and second speakers, respectively. Position tracking system, characterized in that.
6. The method of claim 5,

   The second telecommunication device comprises a first microphone for acquiring first and second sounds, a communication unit for communicating with the first telecommunication device, and receiving first and second sound source data, and a first and second sound source. A storage unit for storing data, mixed waveform data of the first and second sounds from the first microphone, and received or prestored first and second sound source data to compare the first and second sounds of the first and second sounds. Detecting a second delay time, calculating a first distance between the first speaker and the first microphone, and a second distance between the first speaker and the first microphone based on the detected first and second delay times; And a control unit configured to transmit location information including a two-dimensional coordinate system from the first or second speaker of the first telecommunication device to the first telecommunication device based on the first and second distances. Location tracking system.
7. The method of claim 6,

   The second telecommunication device includes a magnetic field sensor and at least one sensor of a gyroscope and an accelerometer, and the control unit of the second telecommunication device is based on a detected value from the sensor so that the second telecommunication device is flat or geometric. A position tracking system, characterized in that it is determined whether the configuration is in the input state moving in contact with a known curved surface or moving state floating in the air.
8. The method of claim 5,

   The second telecommunication device acquires first and second sounds, and communicates with the first and second microphones spaced apart by a predetermined distance, the first telecommunication device, and receives the first and second sound source data. A communication unit, a storage unit storing first and second sound source data, first and second mixed waveform data of first and second sounds from the first microphone, and received or prestored first and second sound sources. Comparing the data, respectively, the first and third delay times of the first sound and the second and fourth delay times of the second sound are detected, and the first speaker and the first speaker are based on the detected first to fourth delay times. Calculate a first distance between one microphone, a second distance between the second speaker and the first microphone, a third distance between the first speaker and the second microphone, and a fourth distance between the second speaker and the second microphone, A first or second speech of the first telecommunication device based on the first to fourth distances That the three-dimensional position information including coordinate system from a control unit to be sent to the first telecommunication device location system according to claim.
9. The method of claim 8,

   The second telecommunication device includes a magnetic field sensor and at least one sensor of a gyroscope and an accelerometer, and the control unit of the second telecommunication device uses a sensed value from the sensor to determine an angle (roll, a position tracking system, comprising calculating the pitch) and including the position in the position information.
10. The method according to any one of claims 1 to 9,

    A location tracking system, characterized in that the clock of the first telecommunication device and the clock of the second telecommunication device are synchronized.
11. The method according to any one of claims 1 to 9,

    The control unit of the first telecommunication device transmits a control command corresponding to the state and operation of the software to the second telecommunication device.
12. The method according to any one of claims 3, 6 and 8,

    And a control unit of the second telecommunication device uses the calculated position information as an input value of the currently operating software.
13. The method of claim 12,

    The control unit of the second telecommunication device transmits information related to the state and operation of the software to the first telecommunication device.
14. The method of claim 13,

    And a control unit of the first telecommunication device uses information related to the state and operation of the received software as an input value of the currently operating software.
15. A first microphone for acquiring sound;

    A storage unit which stores sound source data of sound and at least one software;

    And comparing the input waveform data of the sound acquired from the first microphone with the previously stored sound source data to calculate the position information of the external device generating the sound, and using the position information calculated as the input value of the software. A location tracking system comprising an electrical appliance.
16. The method of claim 15,

    The electrical device includes a communication unit for communicating with an external device, and the control unit transmits the position information or information related to the state and operation of the software to the external device through the communication unit.

Images