WO2014171568A2

WO2014171568A2 - Method and apparatus of detecting touch using sound, and device using same

Info

Publication number: WO2014171568A2
Application number: PCT/KR2013/003244
Authority: WO
Inventors: 황성재
Original assignee: 한국과학기술원
Priority date: 2013-04-17
Filing date: 2013-04-17
Publication date: 2014-10-23
Also published as: WO2014171568A3

Abstract

Provided are a method and an apparatus of detecting a touch of a user terminal, and a user terminal using the same. The method of detecting a touch of a user terminal according to the present invention comprises the steps of: generating a sonic signal from the user terminal; inputting the generated sonic signal to the user terminal; simultaneously detecting whether an intensity of the input sonic signal is changed, and a degree of the change in the intensity; and determining whether a user touches the user terminal depending on whether the detected the intensity is changed. According to the present invention, a touch can be efficiently detected by only speaker/microphone and software provided in an existing equipment, without having hardware such as a pressure sensor. Furthermore, touch intensity is measured through a change rate in intensity of a sonic signal, and thus reliable touch intensity can be measured irrespective of outside noise.

Description

Touch detection method, apparatus and device using same

The present invention relates to a touch detection method using a sound, an apparatus and a device using the same. More specifically, it is possible to detect whether a touch is performed by a speaker or a microphone provided in an existing device, without having hardware such as a button or a pressure sensor. The present invention relates to a touch detection method using a sound, an apparatus, and a device using the same.

Touch screen means that if a person's hand or an object touches a character or a specific location on the screen (screen) without using a keyboard, the input screen can be input directly on the screen so that the location can be detected and processed by the stored software. It refers to a screen that can be received, and is widely used as an input screen of a user terminal such as a mobile phone or a computer. The touch screen is generally divided into a pressure sensitive type and a capacitive type (capacitive type), and the pressure sensitive type has an advantage that there is no limitation on the input device by detecting a pressure change on the touch screen. The capacitive type senses a touch by using a minute current flowing through the body, and has the advantage of multi-touch and quick response. However, since the capacitive touch screens commercialized so far cannot distinguish the pressure (intensity) to touch, only the on-off type touch control is possible. Furthermore, in the conventional touch input strength measurement, the touch intensity is measured according to the contact between the touch pad and the electrode, but in this case, a problem of deterioration of image quality due to the spaced space between the touch pad and the electrode layer occurs. On the other hand, Korean Patent Laid-Open Publication No. 10-2009-0127544 measures a contact time using an acceleration sensor, obtains a signal for each time based on signal information of the acceleration sensor during the measured time, and then obtains a dispersion obtained during the on time. All values are added together to provide a technique for measuring contact strength. However, since the above technique is a method of summing signal information of all measured acceleration sensors, when the actual vibration is caused by an external vibration (for example, a bus) rather than a human touch, the vibration is generated according to the external vibration. There is a problem in that all sensor signal information is added to the intensity of the touch input.

Furthermore, due to the nature of the mobile phone, which is gradually miniaturized and needs to perform various functions at the same time, it is required to use various components such as a microphone that is already installed. It becomes difficult to equip smart devices such as phones, glasses and watches. Therefore, it is very urgent to recycle existing components and repurpose them as touch sensing devices.

Accordingly, the problem to be solved by the present invention is to provide a touch detection device and method that can measure whether the touch and the intensity in the user terminal provided with a touch screen without providing a separate detection device.

In order to solve the above problems, the present invention is a touch detection method of a user terminal, the method comprising the steps of generating a sound wave signal from the user terminal; Inputting the generated sound wave signal to the user terminal; Detecting whether the intensity change of the input sound wave signal and the degree of intensity change are simultaneously detected; And determining whether the user touches the user terminal based on the detected change in intensity.

The method according to an embodiment of the present invention further includes determining a user touch intensity with respect to the user terminal according to the degree of change in intensity.

In an embodiment of the present invention, the sound wave signal generation is generated through a speaker provided in the user terminal, and the sound wave signal input is input through a microphone provided in the user terminal. In one embodiment of the present invention, the user touch is a touch on a touch screen provided in the user terminal. In an embodiment of the present invention, the intensity change of the input sound wave signal is proportional to the touch intensity. In one embodiment of the present invention, the sound wave signal is a sound wave signal having a frequency of 2000 Hz or more. The present invention also provides a touch detection device of a user terminal, the device comprising: an output unit 110 for generating a sound wave signal; An input unit 120 for receiving a sound wave signal generated from the output unit 110; It provides a touch detection device of the user terminal comprising a control unit 130 for determining the presence or absence of the touch and the touch intensity in accordance with the change in the sound wave signal strength received from the input unit. In one embodiment of the present invention, the output unit 110 is a speaker, and the input unit 120 is a microphone. In one embodiment of the present invention, the output unit 110 generates a sound wave signal having a frequency outside the audible region. In one embodiment of the present invention, the control unit 130 determines the stronger touch intensity as the change in the sound wave signal intensity increases. The present invention provides a user terminal equipped with the above-described touch detection apparatus of the user terminal.

In one embodiment of the present invention, the user terminal includes a command unit for generating a command signal for operating the user terminal according to whether or not the touch determined by the controller 130 and the touch intensity.

According to the present invention, the presence or absence and the touch intensity of the terminal is measured by changing the intensity of the sound wave signal. Therefore, the touch can be effectively detected only by the speaker / microphone and the software provided in the existing device without having hardware such as a pressure sensor. Furthermore, since the touch intensity is measured through the change amount of the sound wave signal strength, reliable touch intensity measurement is possible regardless of external noise.

1 is a diagram showing an audio range and a specific signal range according to an embodiment of the present invention.

2A and 2B illustrate a touch detection method according to the present invention.

3 is a step diagram of a touch detection method according to an embodiment of the present invention.

Figure 4 is a photograph of an example of measuring the touch and intensity in the user terminal according to the method according to the present invention.

5 and 6 are views illustrating various application methods according to the touch strength measuring method according to an embodiment of the present invention.

7 is a photograph showing an amplitude according to an input frequency band when there is no noise outside, and FIG. 8 is a photograph showing an amplitude according to an input frequency band when there is noise outside.

9 is a block diagram of a touch detection apparatus according to an embodiment of the present invention.

In order to fully understand the present invention, the operational advantages of the present invention, and the objects achieved by the practice of the present invention, reference should be made to the accompanying drawings which illustrate preferred embodiments of the present invention and the contents described in the accompanying drawings.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. As those skilled in the art would realize, the described embodiments may be modified in various different ways, all without departing from the spirit or scope of the present invention. In addition, in order to clearly describe the present invention, parts irrelevant to the description are omitted, and the same reference numerals in the drawings indicate the same members.

Throughout the specification, when a part includes a certain component, this means that it may further include other components, without excluding other components unless otherwise stated. In addition, the terms ... unit, ... unit, module, block, etc. described in the specification means a unit for processing at least one function or operation, which may be implemented in hardware or software or a combination of hardware and software. have.

In order to solve the above-mentioned problems of the prior art, that is, the difficulty of detecting the user's touch and intensity (hereinafter, touch and touch strength) on the touch screen, the present invention receives a microphone, that is, a sound wave or an ultrasonic wave, Microphones, which are electronic components (components) that generate electrical signals, are used.

In general, the microphone is already provided in a user terminal such as a mobile phone is already used for the purpose of voice recording, breathing sensor, etc. The present invention, in particular, by using a sound wave input from the microphone, the user touch of the user terminal, such as a mobile phone Measure the strength. That is, from the characteristic that the sound wave intensity of the predetermined area rapidly input to the microphone according to the touch of the user terminal is measured, the present invention measures whether or not the touch is made by using the change of the sound wave intensity generated according to the touch. In the present specification, whether or not to touch and strength measurement is referred to as 'touch detection'.

Referring to FIG. 1, the frequency of sound typically heard by people is approximately 16-20000 Hz, which is referred to as the audible sound range. However, in the audible range, the practical audible range that we can easily hear is only 250-2000 Hz. Therefore, according to an embodiment of the present invention, after generating a sound wave having a frequency above the conversational range and receiving the same, the touch wave is measured by comparing the input value and the output value of the generated sound wave. In one embodiment of the present invention, through the built-in speaker provided in the user terminal continuously outputs a sound wave signal of a predetermined range (about 2000 Hz), and at the same time continuously measures the strength of the sound wave signal input through the built-in microphone . At this time, if the microphone is covered by the hand, a difference occurs in the output volume value and the input volume value of the sound wave, and the degree of the difference depends on the pressure intensity of the hand. That is, one embodiment of the present invention measures the touch intensity of the user terminal in a manner of emulating (emulation) the input volume value that varies depending on the pressure intensity of the hand.

The touch detection method and its principles used in the present invention will be described in more detail below.

The inventors noted that sound sound waves are transmitted through the surface of the device between the sound output unit (speaker) and the sound input unit (microphone) of the user terminal. That is, the sound waves output through the speaker of the same device is moved along the surface of the device in the form of vibration and has a sound wave flow input through the microphone again. If any type of block occurs in the sound wave transmission process, the vibration value input into the microphone is reduced, and as a result, the amplitude of the input value is reduced. Therefore, when the user's finger or the like touches the terminal, the touched finger performs a sound absorption function, and the touched finger acts as a resistance to sound wave transmission. As a result, the intensity of the input sound wave signal is reduced according to the user's touch, and the presence or absence of the touch can be measured according to whether the decrease in the touch intensity occurs. Further, the touch strength may vary depending on the area where a finger or the like contacts the terminal, the degree of deformation of the surface of the terminal, and the inherent frequency change of the terminal surface. The touch intensity may be detected in proportion to the degree of decrease in the intensity of the sound wave signal. In other words, as the intensity of touching the device increases, the vibration transmitted along the surface of the device is further disturbed, and as a result, the magnitude of the input value is greatly reduced. Through this principle, the presence or absence of touch and its strength can be effectively detected.

Referring to FIG. 2A, it can be seen that a sound wave signal is generated in a sound range indicated by a red circle.

Referring to FIG. 2B, it can be seen that the sound wave signal intensity of the sound band indicated by the red circle is significantly reduced according to the touch of the user terminal. In particular, the present invention has found that such a decrease in the strength of the sound wave signal is proportional to the touch intensity, and this characteristic is directly used for detecting the touch intensity of the user terminal. In particular, the portions indicated by the red circles in FIGS. 2A and 2B are ultrasonic bands having a frequency band of 2000 Hz or higher, and an input volume value (sound intensity) of the ultrasonic bands is measured without and without touch. It can be seen that the value changes abruptly, and the magnitude of the changed volume value is close to the touch intensity. Therefore, the method and apparatus for detecting a touch of a user terminal according to the present invention simultaneously use the output unit and the input unit in a terminal having an output unit (for example, a speaker) and an input unit (for example, a microphone). Used as The position configuration of the output unit and the input unit may vary depending on the terminal, but as long as it is provided in at least one exterior case, the configuration of the present invention is not limited to the position and type of the component.

Referring to FIG. 3, first, a sound wave signal of a specific frequency range is generated (S110). In one embodiment of the present invention, the specific frequency region corresponds to an ultrasonic region above the audible region, and the sound wave signal is generated by a speaker provided in the user terminal. Thereafter, the sound wave signal generated by the speaker is received through the microphone provided in the user terminal (S120). Thereafter, after the intensity (volume value) of the received sound wave signal is measured (S130), it is determined whether the touch is detected according to whether the intensity is changed (S140). Furthermore, in another embodiment of the present invention, the touch intensity is calculated according to the intensity variation of the input sound wave signal (S150). The above-described touch presence and intensity measuring principles are described in more detail below.

According to an embodiment of the present invention, whether or not the touch and the touch intensity are changed according to the amount of change of the intensity (first intensity) of the sound wave signal received when the touch does not proceed and the intensity (second intensity) of the sound wave signal received after the touch is generated. Detect at the same time. Therefore, despite the ambient noise, the present invention measures the touch intensity only by the difference between the first intensity before touch and the second intensity after touch, so that the touch detection method and apparatus according to the present invention can Reliable and reliable touch and strength measurement are possible.

Figure 4 is a photograph of an example of measuring the touch and intensity of the user terminal in accordance with the method according to the present invention.

Referring to FIG. 4, it can be seen that the touch or the touch of the touch screen can be effectively detected according to the change in the intensity of the sound wave signal received through the microphone. In particular, in one embodiment of the present invention, the energy of the sound wave signal output through the rear surface is physically reduced according to the touch intensity, thereby detecting the touch intensity. Furthermore, the inventors noticed that sound (sound waves) are transmitted through the surface of the device as a speaker as an output unit and a microphone as an input unit, and the vibration is reduced when the block is blocked through the human skin. We found that the intensity decreases, and the touch intensity is measured using the decrease of the sound wave signal strength. That is, according to the present invention, the actual sound wave signal is reduced according to the touch on the surface of the user terminal, and the intensity of the reduced sound wave signal is measured through a conventional microphone, and the degree of reduction of the measured sound wave signal intensity is reduced. The touch strength is calculated and detected accordingly. Accordingly, the method and apparatus for measuring touch strength according to the present invention can have the same effect only with the speaker / microphone and software provided in the existing apparatus without having hardware such as a pressure sensor, thereby reducing the cost and correlating to external noise. Can work without

Referring to FIG. 5, the touch intensity of the user terminal may be measured as an intensity change amount of the received sound wave signal, and thus the degree of enlargement and reduction of the screen may be determined.

In addition, referring to Figure 6, it is possible to change the thickness of the input line on the touch screen according to the degree of pressure measured in accordance with the present invention.

As described above, the present invention can operate various application commands for the user terminal by using the touch intensity detected according to the sound wave signal without a separate measuring device.

7 is a photograph showing an amplitude according to an input frequency band when there is no noise, and FIG. 8 is a photograph showing an amplitude according to an input frequency band when there is noise.

Referring to FIGS. 7 and 8, the output intensity of ultrasonic waves, which are not generally generated, does not change even when external noise occurs. Even if the ultrasonic wave is generated from the outside, the present invention is recognized as a touch only when a change in the intensity of the input ultrasonic wave occurs, and thus is not affected by the external environment. That is, the generated ultrasonic signal changes only when it is covered by the hand (that is, when the user terminal is touched), and the degree of difference varies according to the intensity.

The present invention relates to a user interface for performing object conversion in an electronic device, especially a user terminal, including a touch screen or an external case. Here, the touch screen is a form of a screen capable of detecting both a direct or proximity touch of a user, and includes both a capacitive and a resistive film. The user terminal also includes any electronic device with or without a touch screen, such as a cell phone, laptop, e-book, etc., but is preferably a mobile device, glasses, watch.

9 is a block diagram of a device touch detection apparatus according to an embodiment of the present invention.

Referring to FIG. 9, the device touch detection apparatus according to the present invention includes an output unit 110 for generating a sound wave signal, an input unit 120 for receiving a sound wave signal generated from the output unit 110, and a sound wave received from the input unit. And a controller 130 to determine presence or absence of a touch and touch intensity according to a change in signal strength. Since the touch presence and touch intensity determination process by the controller 130 is the same as described above, a detailed description thereof will be omitted.

The present invention further proposes a small sized small device, such as Google Glass or iWatch, by way of example using the above-described scheme.

10 is a step diagram of a touch detection method of a device according to an embodiment of the present invention.

Referring to FIG. 10, first, a sound generated from a device is first detected by the device (S120). The sound may be an inaudible region frequency sound generated through the small speaker provided in the device, and the first detection may be performed by a small microphone such as a pin microphone provided in the device.

Thereafter, the change of the sound generated from the device such as the glasses is detected in the device (S220). For example, when a user touches an area where sound is generated in the device, a change in intensity of the sound occurs, and when the change in the second detected sound exceeds a predetermined value, the touch on the device is It is determined that (S230).

11 to 13 are diagrams showing examples of use of the present invention in smart glasses capable of imaging and the like.

Referring to FIG. 11, smart glasses 310 are disclosed, and the smart glasses 310 are provided with a speaker 320 and a microphone 330. In FIG. 11, sound of a certain energy is generated through the speaker 320, and the generated sound is first detected by the microphone 330.

Referring to FIG. 12, a user touches the speaker 320 with his or her finger, and thus the first detected sound is blocked by the finger, thereby reducing its volume. As a result, the microphone 330 detects the sound change for the second time, thereby determining that there is a touch on the smart glasses. In this case, even if the user touches the microphone 330, the same effect may be obtained.

Referring to FIG. 13, various functions of the smart glasses are performed according to the determination that the touch exists. For example, when it is determined that the change in sound intensity or energy is equal to or greater than a predetermined value, photographing of the camera may be performed through the smart glasses.

Similar to the above-described method, the touch detection method according to the present invention can be used not only for smart glasses but also for smart watches, mobile phones and the like. The method according to the present invention is particularly useful for driving a device in a device which is difficult to install new sensors or the like due to miniaturization.

Another embodiment of the present invention relates to a method of controlling the focus of a camera provided in a device according to the touch intensity of the device.

14 and 15 illustrate a method of controlling a camera focus or shutter opening degree according to an embodiment of the present invention.

Referring to FIG. 14, first, an object of interest is captured by using a camera provided in a mobile phone.

Referring to FIG. 15, imaging of an object of interest is completed by touching the microphone 420 provided in the mobile phone 410. However, depending on whether the microphone is hardly touched or weakly touched, the intensity of sound input from the speaker 430 and input to the microphone 420 provided in the mobile phone 410 is different. Therefore, when the microphone 420 is touched with an intermediate force as shown in FIG. 15, the aperture opening degree or the depth of focus is also controlled to the middle.

Referring to FIG. 16, when a microphone is touched for a predetermined time, an object is photographed at the controlled depth of focus.

The present invention also provides a device having the above-described touch detection apparatus in the scope of the present invention. In this case, a command unit (not shown) for manipulating the user terminal according to the detected presence or intensity of the user may be provided in the user terminal controller to perform various operation commands according to the touch intensity.

As described above, although the present invention has been described by way of limited embodiments and drawings, the present invention is not limited to the above-described embodiments, which can be variously modified and modified by those skilled in the art to which the present invention pertains. Modifications are possible. Accordingly, the spirit of the invention should be understood only by the claims set forth below, and all equivalent or equivalent modifications will fall within the scope of the invention.

In addition, the method and apparatus for measuring touch strength according to the present invention may be embodied as computer readable codes on a computer readable recording medium. The computer-readable recording medium includes all kinds of recording devices in which data that can be read by a computer system is stored. Examples of recording media include ROMs, RAMs, CD ROMs, magnetic tapes, floppy disks, optical data storage devices, hard disks, flash drives, and the like, and may also be implemented in the form of carrier waves (for example, transmission over the Internet). Include. The computer readable recording medium can also be distributed over network coupled computer systems so that the computer readable code is stored and executed in a distributed fashion.

The present invention is applicable to an electronic device provided with a microphone or the like.

Claims

Touch detection method of the user terminal, the method

Generating a sound wave signal from the user terminal;

Inputting the generated sound wave signal to the user terminal;

Detecting whether the intensity change of the input sound wave signal and the degree of intensity change are simultaneously detected; And

And determining whether the user touches the user terminal according to the detected change in intensity.
The method of claim 1 wherein the method

The touch detection method of the user terminal, characterized in that further comprising the step of determining the user touch intensity for the user terminal in accordance with the degree of change in intensity.
The method of claim 1,

The sound wave signal generation is generated through a speaker provided in the user terminal, the sound wave signal input is input via the microphone provided in the user terminal.
The method of claim 3, wherein

The user touch is a touch detection method of the user terminal, characterized in that the touch on the touch screen provided in the user terminal.
The method of claim 2,

And a degree of change in intensity of the input sound wave signal and the touch intensity are proportional to each other.
The method of claim 1,

And the sound wave signal is a sound wave signal having a frequency of 2000 Hz or more.
A touch detection device of a device, the device comprising

An output unit 110 for generating a sound wave signal;

An input unit 120 for receiving a sound wave signal generated from the output unit 110;

And a controller (130) for determining the presence or absence of a touch according to the change of the sound wave signal strength received from the input unit.
The method of claim 7, wherein

The output unit (110) is a speaker, the input unit 120 is a touch detection device of the device, characterized in that the microphone.
The method of claim 8,

The output unit 110 is a touch detection device of the device, characterized in that for generating a sound wave signal having a frequency outside the audible region.
The method of claim 9,

The control unit (130) is a touch detection device of the device, characterized in that the greater the change in the strength of the sound wave signal strength is stronger.
The method of claim 10,

The device of claim 1, wherein the device is glasses or a watch.
The method according to any one of claims 7 to 10,

The device is a user terminal equipped with a camera, the touch detection apparatus of the device, characterized in that the camera focal length of the user terminal is adjusted according to the touch intensity determined according to the change in the sound wave signal strength.
As a device touch detection method,

A sound detected from the device is first detected at the device;

Detecting a change in the sound generated from the device at the device for a second;

And determining a touch on the device when the change of the second detected sound exceeds a predetermined value.
The method of claim 13,

The device is a glasses or a watch, the device touch detection method, characterized in that the sound is generated through a speaker provided in the glasses or watch.
The method of claim 14,

The change of the sound generated from the device is a device touch detection method, characterized in that generated by the user touches the speaker provided in the glasses or watch.
The method of claim 15,

The change in the sound generated from the device is a device touch detection method, characterized in that generated by the user touches the microphone provided in the glasses or watch.