KR20150117013A - Input method and device using ultrasound - Google Patents

Abstract

An input device for a user portable terminal according to an exemplary embodiment of the present invention includes an ultrasonic generator attached to a skin surface of a user and applying ultrasonic waves to a human body; An ultrasonic receiver attached to the surface of the skin at a site other than the site to which the ultrasonic generator is attached and receiving a reflected wave generated while distorting the sound field of the skin formed by the applied ultrasonic wave; And a frequency detector for detecting a frequency of a signal received from the ultrasonic receiver.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention [0001] The present invention relates to an input device and method using an ultrasonic wave,

The following description relates to an apparatus and a method for using a user's skin surface in inputting information to a terminal device carried by the user and executing functions of the corresponding terminal device.

To enhance the convenience of daily life, users are wearing or carrying various electronic devices. For example, many people are carrying smart devices for voice or data communication, and wear HMD (Head Mount Device) such as smart goggles to use for voice communication as well as for photography and video transmission. As the wearable computer technology develops, the number and types of devices mounted on the body are increasingly increasing. In such an apparatus, an input apparatus for interfacing with a user is essential, and by using such an input apparatus, a user can use various functions of the apparatuses.

Keyboard, which is a representative input device currently used, is capable of inputting a variety of information, but it is very bulky and difficult to carry. In order to overcome this problem, a keyboard is displayed on a screen of a device and information is inputted through the keyboard. However, it is difficult to apply to a device having a small screen such as a wrist smartphone. Instead of the screen of the device, a projection keyboard is used to project a keyboard on a wall or desk using a laser, and to detect the position of the user's finger and input keyboard information at the corresponding position. However, In particular, in order to detect the position of the finger in the wrist-type smartphone, the sensor needs to be configured to protrude from the flat-type smartphone. A muscle nerve sensory information input device capable of inputting specific information by sensing the movement of the muscles and the movement of the arms by sensing a change in the potential of the muscles as the user moves has been developed. However, But it has a disadvantage that a separate muscle neuron sensing device should be attached to the forearm region where many muscles are present.

In addition, a minute sound signal is generated in the body when the user touches the user's skin with the finger, and the sound signal is different according to the bone density, the amount of muscle and the joint of the contact part. An input device has been developed. By assigning specific information to each of the various contact areas, the contact area is detected and then the corresponding information is input to the device. Therefore, it is not necessary to provide the sensor in a protruding form when applied to a wrist-type smartphone However, there is a disadvantage in that the movement of the contact portion can not be detected when the finger is moved while being in contact with the finger.

The input device for a user portable terminal according to an embodiment of the present invention can detect information inputted by the user's finger by mounting the sensor for sensing the user's input, The present invention provides an apparatus and method that can detect when a user's finger moves in contact with the finger.

An input device for a user portable terminal according to an embodiment includes an ultrasonic generator attached to a skin surface of a user and applying ultrasonic waves to a human body; An ultrasonic receiver attached to the surface of the skin at a site other than the site to which the ultrasonic generator is attached and receiving a reflected wave generated while distorting the sound field of the skin formed by the applied ultrasonic wave; And a frequency detector for detecting a frequency of a signal received from the ultrasonic receiver.

According to one aspect of the present invention, the input device for a user portable terminal compares a frequency of a signal received through a frequency detector with a frequency of an ultrasonic signal applied to the human body, thereby detecting an ultrasonic wave input from the ultrasonic wave generator, It is possible to detect a change in the position of the distortion point of the sound field on the basis of the plane including the traveling direction of the sound field.

According to another aspect of the present invention, the input device for a user portable terminal includes at least two ultrasonic generators for adjusting a range of a sound field area formed on the surface of the skin, and controls the phase of signals input to the respective ultrasonic generators A phase adjuster may be additionally provided.

According to another aspect of the present invention, the input device for user portable terminal may include an amplifier for amplifying a signal input to the ultrasonic generator.

According to another aspect of the present invention, the input device for a user portable terminal may further include a comparator that compares two or more of the ultrasonic receivers and compares magnitudes of signals received from the respective ultrasonic receivers.

According to another aspect of the present invention, the input device for a user portable terminal detects the position of an ultrasonic receiver that receives a signal having a maximum size from the comparator, thereby detecting the position of the ultrasonic wave input from the ultrasonic generator, the ultrasonic receiver, The positional change of the distortion point of the sound field can be detected based on the vertical plane of the plane including the traveling direction.

According to another aspect of the present invention, the input device for user portable terminal may include an amplifier for amplifying a signal received from the ultrasonic receiver.

According to another aspect of the present invention, the input device for a user portable terminal may input information of a distortion point of the sound field to a terminal device.

According to another aspect of the present invention, the input device for a user portable terminal may further include an impedance matching layer on the surfaces of the ultrasonic generator and the ultrasonic receiver to reduce ultrasonic reflection on the surface of the user's skin.

According to an embodiment of the present invention, there is provided an input method for a user portable terminal, including the steps of: applying ultrasonic waves to a human body of a user to form a sound field on the skin surface; Receiving a reflected wave generated by distortion of the sound field; Positioning the cursor in the terminal center in the center of the screen when the reflected wave is received; Detecting position information of a distortion point of the sound field by comparing magnitudes of signals received from a plurality of ultrasonic receivers; Detecting position information of a distortion point of the sound field by comparing the largest signal among the signals received from the plurality of ultrasonic receivers with a frequency of an ultrasonic wave applied to the human body; Inputting the detected position information to the mobile terminal and moving a cursor in the terminal apparatus by a position change of the distortion point; And performing a portable terminal function corresponding to the cursor when the reflected wave is not received.

According to one aspect of the present invention, the input method for a user portable terminal may further include a step of waiting for reception of a reflected wave when a reflected wave is not received after a sound field is formed on the surface of the skin.

According to another aspect of the present invention, the input method for a user portable terminal may further include a step of starting from the step of comparing the intensity of the reflected wave signal when the reflected wave is continuously received while detecting the change of the position of the distortion point .

According to an embodiment of the present invention, there is provided an input device using ultrasonic waves, comprising at least one ultrasonic generator attached to a skin surface of a user and applying ultrasonic waves to a human body; At least one ultrasonic receiver for receiving reflected waves generated by the applied ultrasonic waves; And a frequency detector for detecting a frequency of a signal received from the ultrasonic receiver.

According to one aspect of the present invention, the ultrasonic generator can adjust a region where a sound field of a standing wave is formed by adjusting the beam width and direction of the ultrasonic wave.

According to another aspect of the present invention, the ultrasonic receiver may be attached to a surface of the skin at a site other than the site to which the ultrasonic generator is attached.

According to another aspect of the present invention, the ultrasonic receiver can determine the reflected wave having the largest magnitude of the received reflected wave as a vertical position of the contact point.

According to another aspect of the present invention, the ultrasonic receiver can measure a direction and a velocity of a user's input by measuring a frequency variation of the reflected wave.

According to an exemplary embodiment of the present invention, there is provided a method of input using an ultrasonic wave, comprising: sensing input from a user; Inputting an ultrasonic signal to a user's body by the sensed input; Reaching the ultrasonic signal input into the inside of the human body to the skin surface and reflecting the ultrasonic signal reaching the skin surface; And forming a standing wave type sound field on the surface of the skin, the sound field being generated by the sum of the input ultrasonic waves and the reflected ultrasonic waves.

According to one aspect of the present invention, there is provided a method of inputting using an ultrasonic wave, the method comprising: generating an input area through which a user can input; Causing a skin surface of the generated input area to contact the user and causing a variation in the acoustic impedance of the skin surface shifted by the contact; Forming a reflected wave on the skin surface by the generated acoustic impedance; And receiving the formed reflected wave and detecting a frequency from the received reflected wave.

According to another aspect of the present invention, the step of forming a standing wave type sound field generated by the sum of the input ultrasonic waves and the reflected ultrasonic waves on the skin surface may include adjusting the beam width and direction of the ultrasonic signal input to the body, Adjusting a region where the sound field of the standing wave is formed by the adjusted beam width and direction; And generating an input region capable of sensing the input of the user by the region where the sound field of the standing wave is formed.

According to another aspect of the present invention, the step of receiving the generated reflected wave and detecting the frequency from the received reflected wave includes the steps of: changing the frequency of the received reflected wave according to the direction of the input by the Doppler effect; And comparing the frequency of the sound wave of the standing wave with the frequency of the received reflected wave.

The input device for a user portable terminal according to the embodiment can synchronize the electronic devices close to the user in a very simple manner by using the electrocardiogram of the user, thereby improving the functions of the electronic devices.

1 is a block diagram illustrating a terminal device including an input device for a user portable terminal according to an exemplary embodiment of the present invention.
FIG. 2 is a diagram illustrating a position of a finger in a vertical direction through a terminal device including an input device for a user portable terminal according to an embodiment.
3 is a diagram illustrating an input method for a user portable terminal of an input device for user portable terminal according to an exemplary embodiment.
4 is a block diagram illustrating a configuration of an input device for a user portable terminal according to an exemplary embodiment.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings.

1 is a block diagram illustrating a terminal device including an input device for a user portable terminal according to an exemplary embodiment of the present invention.

1 shows a terminal device 100 having an input device using a sound field formed on a skin surface of a human body. The terminal device 100 may include an input device for a user portable terminal.

An ultrasonic wave generating device 110 attached to the human body 101 for inputting the ultrasonic wave signal 107 to the human body 101 and forming a sound wave field 102 of a standing wave on the surface of the skin, And an ultrasonic receiving apparatus 120 which is attached to the human body 101 and receives the reflected wave 104 generated by the contact of the finger 103 and detects a change in the frequency of the reflected wave 104. In this case, the word corresponding to the normal blue traveling wave may mean a wave that vibrates in place without advancing in any direction when two waves having the same amplitude and frequency proceed in opposite directions and overlap.

When the terminal device 100 receives information from a user, the terminal device 100 may drive the ultrasonic wave generating device 110 to input the ultrasonic signal 107 to the user's body 101. [ The ultrasonic signal 107 inputted into the human body 101 can be transmitted through the human body and reach the skin surface. Since the acoustic impedance of the skin and air has a great difference, most of the ultrasonic signals 107 arriving at the surface of the skin are reflected. As a result, the surface of the skin has a standing wave type sound field 102 generated by the sum of the input ultrasonic waves and the reflected ultrasonic waves. Can be formed.

The area where the sound field 102 of the standing wave is formed can be adjusted by adjusting the beam width and the direction of the ultrasonic signal 107 inputted to the human body from the ultrasonic wave generating device 110, An input area 105 for inputting information using the finger 103 of the user. The user touches the surface of the skin of the input area 105 using the finger 103. Since the shape of the surface of the skin is changed by this contact, a variation occurs in the acoustic impedance of the surface of the skin, May be formed.

Thereafter, the formed reflected wave 104 can be received by using the ultrasonic receiving apparatus 120. [ The frequency of the received reflected wave 104 varies depending on the direction of movement of the finger 103 in comparison with the terminal device 100 due to the Doppler effect. For example, when the skin contact point 106 of the finger 103 moves in the direction of the terminal device 100, the frequency of the reflected wave 104 formed by the finger 103 is equal to the frequency of the sound field 102 of the standing wave And in the opposite case, the corresponding frequency can be reduced. For example, the same principle as the speed gun used to measure the speed of a moving object is that the baseball player throws an ultrasonic wave or a radio wave into a thrown ball or a car that is running, measures the ultrasonic waves or radio waves that are reflected, Since the frequency of ultrasonic waves and waves that are reflected and reflected by the object changes with the direction and velocity of the object, the direction and speed of the object can be measured by measuring the frequency variation.

The input device for a user portable terminal according to an embodiment can detect whether the finger approaches or disappears to the terminal device by detecting the frequency of the reflected wave and comparing the frequency with the frequency of the sound wave of the standing wave, Can be detected.

FIG. 2 is a diagram illustrating a position of a finger in a vertical direction through a terminal device including an input device for a user portable terminal according to an embodiment.

FIG. 2 is a top view of a terminal device 200 having an input device for a user portable terminal using a sound field formed on the skin surface of FIG. 1. As shown in FIG. 2, in order to detect the position of the finger 203 in the vertical direction, . ≪ / RTI > At this time, the ultrasound receiving apparatus may include a plurality of ultrasound receivers 220. The intensity of the reflected wave 204 received by the ultrasonic receiver 220 of the ultrasonic receiver is inversely proportional to the distance between the contact point 206 and the ultrasonic receiver. For example, since the reflected wave 204 generated at the contact point 106 is transmitted along the surface of the skin, if the distance between the contact point 206 and the ultrasonic receiving apparatus is long, the signal loss of the reflected wave 204 due to transmission increases, The receiving apparatus receives the reflected wave 204 of small size.

A plurality of ultrasonic receivers of the ultrasonic receiving apparatus according to an embodiment are arranged and then the magnitudes of the reflected waves received through the respective ultrasonic receiving apparatuses are compared and the vertical position of the ultrasonic receiver that receives the largest reflected wave is referred to as a vertical It can be judged as a position.

3 is a diagram illustrating an input method for a user portable terminal of an input device for user portable terminal according to an exemplary embodiment.

FIG. 3 shows a procedure for detecting a horizontal movement direction and a vertical position of a touch point by using an input method of a user portable terminal input device and performing a specific operation in the terminal device through the detection. At this time, the terminal device is an electronic device capable of communicating with, for example, a smart phone, a wearable smart device, and the like.

The terminal device may periodically detect the reception state of the reflected wave through a plurality of ultrasonic receivers of the ultrasonic receiver to determine whether the reflected wave is received (S300). At this time, if the reflected wave is not received, the terminal waits for reception of the reflected wave (S301). If the reflected wave is received, the terminal device can place the mouse cursor at the center of the screen (S302).

The terminal device compares the intensity of the reflected wave received from the plurality of ultrasonic receivers (S303) and determines the position of the ultrasonic receiver that receives the reflected wave having the maximum size as the vertical position of the contact point. Thereafter, in the ultrasonic receiver having the maximum reflected wave signal amplitude, the frequency shift of the reflected wave signal is detected (S304), and the moving direction and the moving velocity in the horizontal direction with respect to the contact point can be detected.

The terminal device can move the displayed mouse cursor at the center in accordance with the vertical direction and the moving direction and the moving speed in the horizontal direction (S305). The terminal device can detect whether or not the reflected wave is continuously received and determine whether the reflected wave is continuously received (S306). In this case, if the reflected waves are continuously received, the process returns to the step of comparing the intensity of the reflected wave signal of S303, and the process of S303 to S307 may be performed. If reflected waves are not continuously received, The operation can be performed by executing the corresponding menu (S307). Thereafter, in order to perform step S300 again, it is possible to wait for execution of the new operation by waiting for reception of the reflected wave (S301).

Synchronization can be achieved in a very simple manner by using the user's electrocardiogram in synchronizing the electronic devices close to the user according to the embodiment, thereby improving the functions of the electronic devices.

4 is a block diagram illustrating a configuration of an input device for a user portable terminal according to an exemplary embodiment.

FIG. 4 is a block diagram showing the configuration of the ultrasonic wave generating apparatus 110 and the ultrasonic receiving apparatus 120 for performing the operation of the flowchart. The input apparatus 400 for user portable terminal uses a sound field formed on the skin surface An ultrasonic wave generating device 410, and an ultrasonic wave receiving device 420.

The ultrasound generator 410 may include a plurality of ultrasound generators 411, an amplifier 412, and a phase adjuster 413. The ultrasonic generator 411 may be attached to the skin surface of the user and apply ultrasonic waves to the human body 401. The ultrasonic generator 410 may include at least one ultrasonic generator 411 for adjusting the range of a sound field area formed on the surface of the skin and may adjust the phase of a signal input to the respective ultrasonic generator 411 A phase adjuster 413 may be additionally provided. The ultrasonic wave generator 410 may include an amplifier 412 for amplifying a signal input to the ultrasonic generator 411.

An ultrasonic generator 411 for applying an ultrasonic signal to a human body to form a sound wave of a standing wave on the human body (e.g., skin surface) may be attached to the skin surface. The plurality of ultrasound generators 411 may convert an electrical input signal into an ultrasound wave and input the converted ultrasound wave into the human body. At this time, the amplifier 412 may be further used to amplify an electrical input signal to increase the size of the input ultrasonic waves.

The ultrasonic wave generator 410 may adjust the beam width and direction of the ultrasonic wave input to the human body when it is necessary to adjust the area of the input area. For this purpose, the ultrasonic wave generator 410 includes a plurality of ultrasonic generators 411 and an amplifier 412 The beam width and the direction of the ultrasonic waves can be adjusted by adjusting the phase of the electric signal inputted to each of the amplifiers 412 by the arrangement effect. A phase adjuster 413 may additionally be used to adjust the phase of the electrical signal input to the amplifier 412.

The ultrasound receiving apparatus 420 may include a plurality of ultrasound receivers 421, an amplifier 420, a comparator 423, and a frequency detector 424. The ultrasonic receiver 421 can receive reflected waves generated by distorting the sound field of the skin surface formed by the applied ultrasonic waves attached to the skin surface of the other area than the site where the ultrasonic generator 411 is attached.

The frequency detector 424 can detect the frequency of the signal received from the ultrasound receiver. The frequency of the signal received through the frequency detector 424 is compared with the frequency of the ultrasonic signal applied to the human body so that the ultrasonic wave from the ultrasonic wave receiver and the ultrasonic wave generator A position change can be detected.

The comparator 423 includes at least one ultrasound receiver 421 in the ultrasound receiver 420 and can compare the magnitudes of the signals received from the respective ultrasound receivers 421. [ The position of the ultrasonic receiver that receives the signal having the maximum size can be grasped.

The amplifier 422 can amplify the signal received from the ultrasonic receiver 421. [

In order to receive the reflected wave, the ultrasonic receiver 421 may be attached to the surface of the skin, thereby converting the reflected wave of the ultrasonic wave form into an electrical signal. At this time, a plurality of ultrasonic receivers 421 may be used to detect the vertical position of the contact point.

The ultrasonic receiving apparatus 420 may further include an amplifier 422 that increases the magnitude of a signal converted into an electrical signal in order to increase reception sensitivity of ultrasonic reception. The comparator 423 compares the output signals of the respective ultrasonic receivers 121 and compares the magnitudes of the reflected waves received through the respective ultrasonic receivers 421, Can be determined.

In addition, the frequency detector 424 can detect the frequency of the output signal of the ultrasonic receiver 421, and the terminal device 400 can determine the moving direction and the moving speed of the contact point in the horizontal direction.

Each of the ultrasonic wave generator 411 and the ultrasonic wave receiver 421 has a difference in acoustic impedance between the skin of the human body 401 and the ultrasonic wave reflected from the skin. In order to minimize the signal loss due to the reflection of the ultrasonic waves, a separate acoustic impedance matching layer may be used between the ultrasonic generator 411 and the ultrasonic receiver 421 and the skin.

The apparatus described above may be implemented as a hardware component, a software component, and / or a combination of hardware components and software components. For example, the apparatus and components described in the embodiments may be implemented within a computer system, such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable array (FPA) A programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to execution of the software. For ease of understanding, the processing apparatus may be described as being used singly, but those skilled in the art will recognize that the processing apparatus may have a plurality of processing elements and / As shown in FIG. For example, the processing unit may comprise a plurality of processors or one processor and one controller. Other processing configurations are also possible, such as a parallel processor.

The software may include a computer program, code, instructions, or a combination of one or more of the foregoing, and may be configured to configure the processing device to operate as desired or to process it collectively or collectively Device can be commanded. The software and / or data may be in the form of any type of machine, component, physical device, virtual equipment, computer storage media, or device , Or may be permanently or temporarily embodied in a transmitted signal wave. The software may be distributed over a networked computer system and stored or executed in a distributed manner. The software and data may be stored on one or more computer readable recording media.

The method according to an embodiment may be implemented in the form of a program command that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like, alone or in combination. The program instructions to be recorded on the medium may be those specially designed and configured for the embodiments or may be available to those skilled in the art of computer software. Examples of computer-readable media include magnetic media such as hard disks, floppy disks and magnetic tape; optical media such as CD-ROMs and DVDs; magnetic media such as floppy disks; Magneto-optical media, and hardware devices specifically configured to store and execute program instructions such as ROM, RAM, flash memory, and the like. Examples of program instructions include machine language code such as those produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. For example, it is to be understood that the techniques described may be performed in a different order than the described methods, and / or that components of the described systems, structures, devices, circuits, Lt; / RTI > or equivalents, even if it is replaced or replaced.

Therefore, other implementations, other embodiments, and equivalents to the claims are also within the scope of the following claims.

100: terminal device
101: Human body
102: Sound field
103: Finger
104: reflected wave
105: input area
106: contact point
107: Ultrasonic signal
110: Ultrasonic generating device
120: Ultrasonic receiving device

Claims (1)

An ultrasonic generator attached to a skin surface of a user and applying ultrasonic waves to a human body;
An ultrasonic receiver attached to the surface of the skin at a site other than the site to which the ultrasonic generator is attached and receiving a reflected wave generated while distorting the sound field of the skin formed by the applied ultrasonic wave; And
A frequency detector for detecting a frequency of a signal received from the ultrasonic receiver;
And an input unit for inputting data to the portable terminal.

Images