JP5794526B2 - Interface system - Google Patents

Description

The present invention relates to an interface technology that is used by being worn on the body, and in particular uses an bone conduction sound transmitted through the bone of an arm.

In recent years, research and development has been conducted on an interface that can be worn on the body using sensing of the body so that device operation, action recording, information provision, etc. can be used immediately when desired.

By using the body as an operation region or an information transmission path, it is possible to sense contact information and use it as an input interface, or sense movement and use it as a controller, for example. These interface conditions include intuitive and stable operability, robustness to environments that can be used comfortably, such as indoors and outdoors, miniaturization and weight reduction that can be worn on the body, and versatility that can be used for various commands and applications. Therefore, there is a need for an interface technology that allows a user to always operate intuitively and stably without worrying about the external environment, and can always be worn without hindering daily life even when worn.

2. Description of the Related Art Conventionally, as an interface using body sensing for always wearing, a small interface using fingers as part of a sensor system has been developed. A ring-type interface “finger button” (Non-Patent Document 1), in which a ring-type acceleration sensor is attached to the base of each finger, and vibration generated by a striking tendon on an arbitrary surface such as a desk or thigh is input. A ring-type interface “finger Nya” (Non-Patent Document 2) that identifies the open / closed state of a “finger ring” formed by bringing a thumb and index finger into contact with each other is known.

In the case of using bone conduction, each bone conduction microphone is placed at the position where the fingertip holding the terminal and the base of the finger contact, and the estimated result of the tapped position when the back of the hand is tapped is used as input. There is a FingerKeypad (patent document 1) (non-patent document 3). These interfaces are small, lightweight, and have a simple structure. However, since a finger is used for input, it is necessary to attach the input interface directly to the finger, which is a heavy burden for daily use from the viewpoint of operability. In addition, there is a problem including a risk of malfunction.

A method has also been proposed in which an arm, not a finger, is used as an operation area, and an estimation of a collision position tapped from the outside is used as an interface. In Non-Patent Document 4, two sets of sensor arrays having five sensor elements are arranged so as to hit the ribs and the ulna, respectively, so that the bone conduction sound transmitted through the bone and the meat transmitted through the meat when the arm is tapped. A method has been proposed in which both conducted sounds are measured and the collision position due to the tap is estimated. In addition, the present inventors have proposed a method of using bone conduction sound transmitted through living bones for contact estimation in Non-Patent Document 5, and bone conduction sound generated when tapping an arm is divided into two bones. A method of detecting by a conduction microphone, estimating a collision position by a tap, and using the same for an interface is disclosed.

However, in these methods, only the collision position by the tap of the arm is an input signal, and it is necessary to increase the input amount by increasing the collision position in order to make an input interface that can perform more complicated operations. Requires more sensors to be worn on the body. For this reason, in terms of reduction in size and weight, the constant wearability is impaired, and the sense of restraint of the user is also increased. In addition, there is a problem that the cost increases.

JP2009 / 0783350

Masaaki Fukumoto, Yoshinobu Tonomura: Finger buttons: Wrist-mounted command input mechanism. IPSJ Journal, vol.40, no.2, pp.389-398, 1999. Saori Noda, Kyo Yamamoto, Mitsuo Okumura, Ichiro Akahori. Finger Nya: A ring that detects the ring of a finger. IPSJ Interaction 2009 Proceedings, pp.135-136, 2009. Masanori Norie, Kazuhito Murata, Mitsuru Sentoda, Hideyuki Mitsuhashi. Fingerkeypad: Mobile device operation by tap input to fingers. IPSJ Interaction 2010 Proceedings, PA15, 2010. Chris Harrison, DesneyTan, and Dan Dan Morris. Skinput: Apppropriating the body as an input surface. Satoshi Ito, Kentaro Takemura, Go Suenaga, Satoshi Takamatsu, Tsukasa Ogasawara. Always-on interface using bone conduction sound with the forearm bone as the transmission path. Proceedings of the 28th Annual Conference of the Robotics Society of Japan.

In the conventional input interface described above, the method of using a finger in the operation area requires a sensor to be directly attached to the finger, which is a burden for daily use, and includes a risk of malfunction. .

In addition, in the method of using the arm for the operation area, only the collision position by the tap is an input signal. To increase the input amount by increasing the tap location, that is, the collision position, an input interface that can perform more complicated operations It is necessary to wear more sensors on the body. For this reason, the mounting ability at the time is reduced in terms of size and weight reduction, and the sense of restraint of the user is also increased. In addition, there is a problem that the cost increases.

In view of such a problem, the present invention enables a user to casually always wear an interface in which the body is a sensing region, and stably inputs various input information intended by the user with a small sensor configuration. It is an object to provide an interface system that can be generated.

In order to achieve the above object, the present invention focuses on the bone conduction sound signal when the bone conduction sound using the body bone as a transmission path changes due to the posture of the body and contact with the body.

Here, the bone conduction sound is a sound generated by bone conduction in which vibrations such as vocal cords are directly transmitted to the auditory nerve through the skull. In the present invention, the propagation of sound through a living bone is regarded as bone conduction, and the entire sound transmitted through the living bone is regarded as a bone conduction sound. The bone conduction sound is resistant to external noise and is relatively easy to sense.

The interface system of the present invention is an interface system used by being worn on the body, and is equipped with at least one vibration input actuator for inputting bone vibrations by inputting at least one vibration to the body, and at least one other. A bone conduction microphone for acquiring the bone conduction sound is attached. An interface system is constructed in which bone conduction sound transmitted through the body skeleton is actively generated by a vibration input actuator, and a signal of the bone conduction sound is acquired by a bone conduction microphone.

Furthermore, the interface system of the present invention is an interface system used by being worn on the body, and includes the components shown in the following (1) to (4).
(1) A vibration input actuator that is attached to the first part of the body and generates bone conduction sound by inputting vibration.
(2) A bone conduction microphone that is attached to a second part of the body and acquires the bone conduction sound.
(3) Signal processing means including a function for estimating the posture of the body based on a bone conduction sound signal acquired by the bone conduction microphone.
(4) Input information generation means for generating predetermined input information from the processing result of the signal processing means.

By adopting such a configuration, a bone conduction sound signal generated by the vibration input actuator is acquired by the bone conduction microphone of the second part, and signal processing is performed based on the signal, whereby the posture of the body And the input information can be generated from the estimation result.

Interface system according to claim 2 of the present invention, a first portion for mounting the vibration actuator and the bone conduction microphone of claim 1, the second site is the same wearer's arm (hereinafter, the user) It is characterized by that.

The arm is a body part suitable for displaying a user's intention by a gesture. For example, input range by gestures such as remote operation of a robot, character operation in a virtual space, ON / OFF of home appliances, volume operation, etc. is wide and promising as a user interface.

Further, the distance between the body surface and the bone is relatively short, and it is easy to generate bone conduction sound by external vibration input. By placing the vibration input actuator and bone conduction microphone on the same arm, the bone type and number of joints that serve as the transmission path are limited, and the distance between the actuator and the microphone is relatively short, and the path is also linear. Therefore, there is an advantage that it is relatively easy to prevent a decrease in bone conduction sound due to attenuation of the bone conduction sound while passing through the bone, and mixing of noise, and it is easier to acquire the bone conduction sound.

In addition, the arm is easy to be worn without a sense of incongruity for the user as compared with other parts of the body, and the restraint feeling is easy to relax. It is a body part that is easy to wear at all times from the viewpoint of daily life by making it smaller and lighter. If the vibration output portion of the vibration input actuator is on the body surface side, the mounting method is not particularly limited. Also, in the bone conduction microphone, if the bone conduction sound collection part is on the body surface side, there is no particular limitation on the wearing method.

Moreover, by mounting on the same arm, the system can be easily made compact.

Also, by adding a harmonious shape and fashionability between the actuator and the microphone, it is easy to remove the appearance and discomfort. For example, an interface system that enhances the user's willingness to wear can be obtained by using a design such as a decorative ring or a wristwatch.

An interface system according to claim 3 of the present invention is characterized in that it includes a function for estimating an elbow flexion angle by bending / extending / flexing of the elbow as a body posture as signal processing means. Elbow flexion / extension is large as arm movement, and it is easy to acquire bone conduction sound signals. It is easy to understand both in appearance and as a gesture. By combining the bend angle estimation result with predetermined external input information, for example, the operation information of the external device can be output according to the flexion / extension / flexion of the elbow, allowing intuitive operation of the device by gesture input Become.

An interface system according to a fourth aspect of the present invention is an interface system used by being worn on the body, and includes the following components (1) to (5).
(1) A vibration input actuator that is attached to the first part of the body and generates bone conduction sound by inputting vibration.
(2) A bone conduction microphone that is attached to a second part of the body and acquires the bone conduction sound.
(3) A bone conduction microphone that is attached to a third part of the body and acquires the bone conduction sound.
(4) a function of estimating the posture of the body based on a bone conduction microphone attached to the second part and a bone conduction sound signal acquired by the bone conduction microphone attached to the third part; Signal processing means including a function of estimating a contact position due to contact with the body.
(5) Input information generation means for generating predetermined input information from the processing result of the signal processing means.

With this configuration, the bone conduction sound signal generated by the vibration input actuator is acquired by the bone conduction microphones of the second part and the third part, and signal processing is performed based on these signals. By doing so, it is possible to estimate the posture and contact position of the body and generate input information from the estimation results. By increasing the wearing position of the bone conduction microphone, it is possible to estimate both the posture of the body and the contact position, thereby increasing the input information.

In the interface system according to claim 5 of the present invention, the first part, the second part, and the third part to which the vibration actuator and the bone conduction microphone of claim 4 are attached are the same arm of the user. It is characterized by. By mounting a plurality of vibration actuators and bone conduction microphones at three locations on the same arm, it is possible to estimate both the posture of the body and the contact position on the same arm.

The interface system according to claim 6 of the present invention is characterized in that the function of estimating the elbow flexion angle due to bending / extending / flexing of the elbow as a body posture is included as signal processing means. Elbow flexion / extension is large as arm movement, and it is easy to acquire bone conduction sound signals. It is easy to understand both in appearance and as a gesture. By combining specific bend angles and bending angle changes with predetermined external input information, for example, operation information of external devices can be output according to the bending and stretching of the elbow, and the device can be viewed intuitively by gesture input. Can be operated manually.

The interface system according to claim 7 of the present invention is characterized in that the contact with the body is a collision caused by a tap on the arm, and the signal processing means includes a function of estimating the collision position as the contact position. A collision with a tap on an arm is relatively easy to acquire a bone conduction sound signal. In addition, it is easy to understand both as an appearance and as a gesture. By combining the collision position estimation result with predetermined input information, for example, operation information of an external device can be output in accordance with the collision position by a tap, and intuitive operation of the device by gesture input becomes possible. .

In the interface system according to claim 8 of the present invention, the signal processing means includes both a bending angle estimating means for estimating the bending angle of the elbow and a collision position estimating means for estimating the collision position by the tap on the arm. Both corners and collision positions can be estimated. It can recognize both elbow flexion / extension and tap gestures, and can increase the amount of information generated as an external input signal. By mounting two bone conduction microphones on the same arm and acquiring signals of the same bone conduction sound at different locations, it is possible to estimate both the body posture and the contact position on the same arm.

The interface system according to claim 9 of the present invention is characterized in that the flexion angle estimation means estimates the flexion angle of the elbow based on a change in the amplitude of the bone conduction sound accompanying flexion / extension / flexion of the elbow. The amplitude of the bone conduction sound signal is a feature quantity that can be easily acquired, and based on the change in the amplitude, a highly reliable estimation is possible.

The interface system according to claim 10 of the present invention is characterized in that the collision position estimation means estimates a collision position by a tap on an arm based on a change in amplitude of the bone conduction sound accompanying the tap. To do. The amplitude of the bone conduction sound signal is a feature quantity that can be easily acquired, and based on the change in the amplitude, a highly reliable estimation is possible.

Furthermore, the interface system of the present invention is characterized in that the bone conduction microphone is a microphone formed by integrally combining a sound collector and a small microphone. With such a configuration, sufficient sensitivity can be easily obtained even when the bone conduction sound is small.

In addition, the tap to the arm in the present invention is a contact action to the body that generates a bone conduction sound, including a contact by a body part such as a finger, an arm, a contact tendon, a contact by a device such as a stick, and a tendon. is there.

According to the present invention, the interface system used by being worn on the body senses bone conduction sound using the bone of the user as a transmission path, and can be used as an input interface. In particular, by mounting a vibration input actuator and bone conduction microphone on the arm, estimating the elbow flexion angle, and estimating the collision position when the arm is tapped, it is possible to use the flexion angle estimation and the collision position estimation simultaneously. It becomes. This makes it possible to configure an interface system that can generate a large amount of input information by using two gestures that are simple and easy for the user to understand (flexion / extension / flexion of the elbow and tap on the arm).

In addition, a mounting system can be realized with a simple configuration using only a vibration input actuator and a bone conduction microphone. Further, there is no need for means for measuring body movement on the external environment side. Therefore, it is possible to configure an interface system that is small and light, is strong against the external environment, and can be used in daily life even when worn by the user at all times.

In an environment such as a wearable computer or ubiquitous computing, a user interface that can be easily operated by the body can be provided.

Schematic diagram showing an example of an interface system installation example Schematic diagram showing an example of an interface system installation example Functional block diagram of the interface system Functional block diagram of the interface system Schematic diagram of mounting the actuator and microphone on the forearm Schematic of mounting two microphones on the forearm Schematic diagram of mounting the actuator and two microphones on the forearm Maximum amplitude of signal acquired by each bone conduction microphone Schematic diagram of the mounting state of the interface system of the first embodiment Stethoscope-type bone conduction microphone Bone conduction amplitude at each elbow flexion angle when bone conduction is input from the humeral head. Schematic diagram of the mounting state of the interface system of Examples 2 and 3 Actual mounting state of the interface system of Examples 2 and 3 Bone conduction amplitude at each elbow flexion angle when bone conduction sound is input from the elbow head. Result of function approximation by calibration of subject A Result of function approximation by calibration of subject B Results of an evaluation experiment for estimating the elbow flexion angle of subject A Results of an evaluation experiment for estimating the elbow flexion angle of subject B Actual mounting state of the interface system of Examples 4 and 5 Bone conduction sound caused by collision (ulnar styloid process side) Bone conduction sound caused by a collision (the elbow head side) Frequency spectrum of bone conduction sound generated by collision (ulnar styloid process side) Frequency spectrum of bone conduction sound generated by a collision (on the elbow) Example of bone conduction sound input from a speaker Frequency spectrum of bone conduction sound input from a speaker The figure which shows the state which divided the forearm into 3 sections Result of subject A's collision position estimation Result of subject B's collision position estimation Result of subject A's elbow flexion angle estimation Result of subject B's elbow flexion angle estimation Processing flow to estimate both the bending angle and the collision position Robot operation commands for estimated collision positions Robot operation commands for combinations of estimated collision positions Using collision position estimation for robot operation Using elbow flexion angle estimation to control the robot Overview of interface system using bone conduction sound

Hereinafter, embodiments of the present invention will be described with reference to the drawings. The following embodiments are examples embodying the present invention, and the technical scope of the present invention is not limited to the examples and the illustrated examples.

FIG. 1a is a schematic diagram showing one embodiment when the interface system of the present invention is worn on the body. One vibration input actuator 10 is attached to the upper arm portion of the user 1000, and one bone conduction microphone 20 is attached to the forearm portion of the same arm via the joint. Vibration is applied from the vibration input actuator 10 toward the body to generate bone conduction sound for the skeleton of the arm. The bone conduction microphone 20 senses bone conduction sound transmitted through the skeleton and joints of the arm. The vibration input from the vibration input actuator may be continuously input to the body or may be input only when necessary.

In such a state, the bending angle of the elbow when the elbow of the arm to be sensed is bent / stretched / bent is estimated by performing signal processing on the bone conduction sound acquired by the bone conduction microphone 20 using a computer or the like. Based on the result of the signal processing, it has a function of generating input information such as a command by a computer or the like for an external device or application software.

The function of performing signal processing by the computer or the like and the function of generating input information may be incorporated in the bone conduction microphone 20 or may be provided outside the body not in direct contact with the body. Further, the interface system of the present invention can be provided with a function capable of transmitting the generated input information to a device to be input regardless of wireless or wired. In addition, it is possible to provide a function for controlling transmission / stop / strength of vibration of the vibration input actuator.

FIG. 1 b is a schematic view showing another embodiment when the interface system of the present invention is worn on the body. One vibration input actuator 30 and one bone conduction microphone 40 are attached to the upper arm portion of the user 2000, and one bone conduction microphone 50 is attached to the forearm portion of the same arm via the joint. A vibration is applied from the vibration input actuator 30 toward the body to generate a bone conduction sound for the skeleton of the arm. The bone conduction microphones 40 and 50 respectively sense bone conduction sound transmitted through the skeleton and joints of the arm. The vibration input from the vibration input actuator may be continuously input to the body or may be input only when necessary.

In this state, the bending angle of the elbow when the elbow of the arm to be sensed is bent / extended / bent is estimated by performing signal processing on the bone conduction sound acquired by the bone conduction microphones 40 and 50 using a computer or the like. Further, the collision position when the sensed arm is tapped is estimated by performing signal processing on the bone conduction sound acquired by the bone conduction microphones 40 and 50 with a computer or the like. Based on the result of the signal processing, a function of generating input information such as a command by a computer or the like is provided for an external device or application software.

The function of performing signal processing by the computer or the like and the function of generating input information may be incorporated in the bone conduction microphones 40 and 50, or may be provided outside the body not in direct contact with the body. Further, the interface system of the present invention can be provided with a function capable of transmitting the generated input information to a device to be input regardless of wireless or wired. In addition, it is possible to provide a function for controlling transmission / stop / strength of vibration of the vibration input actuator.

FIG. 2a shows a functional block diagram of the interface system of the present invention for the embodiment of FIG. 1a. As shown in FIG. 2a, the functional block of the interface system includes a bone conduction sound input means 61 for inputting a vibration by a vibration input actuator and generating a bone conduction sound, and a signal of the bone conduction sound by one bone conduction microphone. Based on the acquired bone conduction sound signal, the signal processing means 63 for performing signal processing for estimating the posture of the body, and the processing result of the signal processing means, the input information is generated. External input information generating means 64.

FIG. 2b shows a functional block diagram of the interface system of the present invention for the embodiment of FIG. 1b. As shown in FIG. 2b, the functional block of the interface system includes a bone conduction sound input means 71 for inputting a vibration by a vibration input actuator and generating a bone conduction sound. Bone conduction sound acquisition means 72 and 73 for individually acquiring signals, signal processing means 74 for estimating the posture and contact position of the body based on the signals of the respective bone conduction sounds acquired individually, and processing of the signal processing means 74 It comprises external input information generating means 75 that generates input information based on the result.

Next, a method for estimating the bending angle when the elbow is bent and stretched and bent will be described. As shown in the schematic diagram of FIG. 3a, the vibration input actuator 80 is attached to the elbow head of the same arm and the bone conduction microphone 90 is attached to the ulnar styloid process one by one, and the elbow of the arm is bent and stretched and bent. At this time, the amplitude of the bone conduction sound generated by the vibration input of the vibration input actuator 80 and transmitted to the bone of the arm changes according to the bending / extension / bending operation. The change in amplitude is acquired as a bone conduction sound signal by the bone conduction microphone 90, and the bending angle of the elbow is estimated from the change in amplitude. Specifically, the user is calibrated in advance, and the relationship between the flexion angle adapted to the individual characteristics of the user (bone thickness / density, muscle thickness, etc.) and the amplitude of the bone conduction sound, Obtained in advance as a function approximation formula. The bending angle can be estimated by using the function approximation formula as the bending angle estimation means. Since the function approximation expression can be expressed by a second-order polynomial or the like, the calculation amount is small, and the calculation can be executed in real time.

Next, a method for estimating the collision position when the arm is tapped from the outside will be described.
The estimation of the collision position is nothing but finding the sound source position of the bone conduction sound generated by the collision. As shown in the schematic diagram of FIG. 3b, the bone conduction microphone 100 is attached to the elbow head of the arm and the bone conduction microphone 110 is attached to the ulnar styloid process one by one, and the forearm in between is generated by tapping with the user's own finger. The bone conduction sound is acquired individually by each bone conduction microphone 100 and 110. The position is estimated based on the level difference between the signals of the bone conduction sounds acquired individually. Here, the level difference is a ratio of the amplitudes of the bone conduction sounds acquired by the two bone conduction microphones 100 and 110.

The strength of the amplitude does not depend on the strength of the collision caused by the tap, and changes depending on the distance from the collision position. That is, a difference in amplitude occurs due to a difference in distance from the bone conduction sound generation position, which is the collision position, to each of the bone conduction microphones 100 and 110, and the collision position is estimated by acquiring the difference as a signal level difference. be able to. In addition, it is desirable that the threshold value of the signal level difference adapted to the user's personal characteristics (arm length, muscle attachment, etc.) is obtained in advance by performing calibration with the user.

Here, when the signal level difference between the bone conduction sounds individually acquired by the bone conduction microphones 100 and 110 is L, L is calculated from the following equation, for example.
F ₁ and F ₂ are the maximum amplitudes of the signals acquired by the bone conduction microphones 100 and 110 as indicated by the symbol F shown in FIG. The point where the amplitude exceeds the threshold value is recorded as the collision start time, and the peak value of the amplitude is obtained from the signals before and after that. Since the signal level difference L is determined by the ratio of amplitudes, it varies depending on the distance from the collision position regardless of the collision strength. That is, since attenuation occurs due to a difference in distance from the bone conduction sound generation position between the bone conduction microphones 100 and 110, the collision position can be estimated from the signal level difference L, and can be used as a collision position estimation means. it can. The estimation based on the amplitude requires a small amount of calculation because it is determined by simple threshold processing, and enables calculation in real time.

Next, a method for individually estimating the bending angle when the elbow is bent and stretched and bent with the same arm and the collision position when the arm is tapped will be described. As shown in the schematic diagram of FIG. 3c, the bone conduction microphone is mounted in a state in which one vibration input actuator 120 and one bone conduction microphone 130 are attached to the elbow head and one bone conduction microphone 140 is attached to the ulnar styloid process. Tap the arm between 130 and 140 with your finger. In order to estimate both the bending angle and the collision position by the tap at this time, the bone conduction sound generated by each movement is individually recognized and extracted, and the bending angle estimation method and the collision position estimation method described above are respectively extracted. Can be applied.

In order to recognize bone conduction sound individually, analyze the frequency characteristics of bone conduction sound caused by flexion / extension / flexion of the elbow and bone conduction sound caused by collision by tap, and determine the frequency domain of each bone conduction sound in advance. By heading and performing frequency filtering using a band pass filter or the like, the bone conduction sound caused by bending / stretching and the bone conduction sound caused by the collision of the tap are separated, and the signal is extracted. Signal processing including bending angle estimation and collision position estimation can be performed on each extracted bone conduction sound signal to estimate the bending angle and collision position.

A first evaluation experiment for elbow flexion angle estimation will be described as Example 1. FIG. 5 is a schematic diagram when the interface system according to the first embodiment is worn on the arm. The vibration input actuator 150 is attached to the humeral head of the arm, and the bone conduction microphones 160 are attached one by one to the ulnar styloid process. The ulnar styloid process and its vicinity are thin and are not easily influenced by noise such as flesh-conducted sound, and are suitable as acquisition positions for bone-conducted sound.

The vibration input actuator 150 applies vibration from the skin surface of the humeral head to generate bone conduction sound in the body. Here, a commercially available vibration motor used for vibration generation of a mobile phone was used as the vibration input actuator 150.

The bone conduction microphone 160 detects only bone vibration. In the first embodiment, the bone conduction microphone 160 is used to acquire the generated bone conduction sound from the skin surface of the ulnar styloid process. In addition, the bone conduction microphone 160 employs a stethoscope for newborns and premature babies (Muranaka Medical Equipment Co., Ltd., MMI-608GR) as a sound collector, and a small microphone (Knowles Electronics, SP0103NC3-3) is integrated. A combined stethoscope-type bone conduction microphone is used.

The appearance and structure of this stethoscope-type bone conduction microphone is shown in FIG. The bone vibration is detected by the diaphragm of the stethoscope, the sound is amplified in the chest piece, and the bone conduction sound is detected by the microphone. Since the stethoscope amplifies the vibration sound of the part directly touched by the diaphragm, it is not easily affected by the air conduction sound, and enables highly sensitive detection of only bone vibration. The new stethoscope for newborns and premature babies uses a diaphragm with a diameter of about 22mm, which is smaller than that of a normal stethoscope, so it reduces detection of meat vibrations around the bone at the same time, and only the vibration of the bones. It is because it is suitable for the detection of.

It should be noted that the structure of such a bone conduction microphone is suitable for acquiring the vibration of the skull while the conventional bone conduction microphone is suitable for acquiring the vibration of the skull. Not suitable. For this reason, sufficient sensitivity cannot be obtained with respect to the bone conduction sound transmitted through the bone of the arm whose vibration is smaller than that of the skull. Therefore, the device has the above-described configuration and is combined with a stethoscope. The sound collector is not limited to the above-mentioned stethoscope for newborns and premature babies, and may be any one that can effectively collect bone conduction sound.

In the interface system configured as described above, when the bone conduction sound is generated from the humeral head using the vibration input actuator 150 (the vibration motor) and the elbow flexion angle is changed every predetermined angle, An experiment was conducted in which the bone conduction sound transmitted through the bone was acquired with the bone conduction microphone 160 (the stethoscope type bone conduction microphone) attached to the ulnar styloid process. A sine wave having a constant amplitude and a frequency of 188 Hz was continuously input from the vibration motor. Here, 188 Hz is a frequency used when the vibration motor is used as a vibration for a mobile phone.

The elbow flexion angle was flexed in increments of 30 degrees from 0 degrees to 120 degrees, with the humerus and ulna being horizontal at 0 degrees and the right angle being 90 degrees. At this time, the bone conduction sound generated by the vibration applied to the arm by the vibration motor was acquired by the stethoscope type bone conduction microphone. The result is shown in FIG. Significant amplitude changes were observed at 0 degrees, 30 degrees, 60 degrees, 90 degrees, and 120 degrees. That is, in the configuration of Example 1, it was shown that the function of the elbow gesture functioned effectively as an input interface by organizing the change in the amplitude of the bone conduction sound caused by bending and stretching of the elbow with a command and a predetermined process. .

Next, a second evaluation experiment for estimating the elbow flexion angle will be described as Example 2. As shown in the schematic diagram when the interface system of FIG. 8a is worn on the arm, the bone conduction microphone 180 is mounted at the same ulnar styloid process as in Example 1, and the vibration input actuator 170 is mounted at the elbow head. did. FIG. 8b shows the actual mounting situation.

A small speaker (SD2412L5-T1 manufactured by TDK) was used as the vibration input actuator 170. The reason why the small speaker is adopted is that it is a separately excited type, so that it can generate vibration according to the signal from the outside, the amplitude can be easily controlled, and the vibration is minute, so that the user can feel uncomfortable vibration. This is because there is an advantage that it does not feel. As the bone conduction microphone 180, the above stethoscope type bone conduction microphone was used.

In the interface system configured as described above, when the bone conduction sound is generated from the elbow head using the vibration input actuator 170 (the small speaker) and the flexion angle of the elbow is changed for each predetermined angle, An experiment was conducted in which the bone conduction sound transmitted through the bone was acquired with the bone conduction microphone 180 (the stethoscope type bone conduction microphone) attached to the ulnar styloid process. A sine wave having a constant amplitude and a frequency of 800 Hz was continuously input from the small speaker. Here, 800 Hz is a resonance frequency of the speaker. The frequency band of vibration input is not particularly limited, but when used simultaneously with the collision position estimation, the frequency band is desirable for easy separation from the bone conduction sound generated by the collision, and 600 Hz or more is particularly preferable.

The elbow flexion angle was flexed in increments of 15 degrees from 0 degrees to 120 degrees, with the state where the humerus and ulna were horizontal being 0 degrees and the right angle being 90 degrees. The bone conduction sound signal was acquired with the bone conduction microphone 180 (the stethoscope type bone conduction microphone). The result is shown in FIG. Significant amplitude changes were observed at 0 °, 15 °, 30 °, 45 °, 60 °, 75 °, 90 °, 105 °, and 120 °. That is, in the configuration of Example 2, it was shown that the function of the elbow gesture functioned effectively as an input interface by organizing the change in the amplitude of the bone conduction sound caused by bending and stretching the elbow with commands and a predetermined process. . For example, it is suitable for a master / slave type or command type interface.

Further, by using the elbow head as the vibration input position, it was possible to capture a smaller change in the bending angle than when using the humeral head of Example 1 as the input position. As a result, it was shown that a small change in bending angle can be recognized and the amount of input signal is increased. Since the vibration input actuator 170 and the bone conduction microphone 180 are close to each other and do not involve any joints, even relatively small vibrations can be used as input vibrations, so that it is possible to provide vibration input means with less burden on the user. In addition, it is easy to configure as a system that prevents distributed arrangement on the body and reduces the burden on the user, and it is easy to configure as a system configuration that improves wearability on the arm.

In addition, even in the vicinity of the elbow head and ulnar styloid process as a wearing position, the muscles are thin and are not easily affected by noise such as flesh-conducted sound, and also from the viewpoint that bone conduction sound is transmitted through one ulna. It also shows that it is suitable as an acquisition position for input and bone conduction sound.

Next, a third evaluation experiment of bending angle estimation for subjects A and B will be described as Example 3. As shown in FIGS. 8a and 8b, subjects A and B wear the vibration input actuator 170 on the elbow head and the bone conduction microphone 180 on the ulnar stylus on the forearm, as in the second embodiment. The vibration input actuator 170 is the small speaker, and the bone conduction microphone 180 is the stethoscope type bone conduction microphone.

First, calibration is performed to approximate the relationship between the elbow flexion angle and the bone conduction sound signal. In the calibration, the state where the humerus and the ulna are horizontal is set to 0 degree, and the bone conduction sound is generated by the elbow-head vibration input actuator 170 (the small speaker) 10 times each in the postures of 0 degree, 45 degrees, and 90 degrees. A change in amplitude was obtained with the bone conduction microphone 180 of the ulnar styloid process (the auscultatory stethoscope type bone conduction microphone). From this change in amplitude, the function approximation of the bending angle and amplitude by a second-order polynomial was performed using the least square method, and a function approximation expression was created. The results are shown in FIGS. 10a and 10b. FIG. 10 a is a function approximation expression for subject A, and FIG.

Here, the reason why the quadratic polynomial is used is that when it is assumed that the change in the bone conduction sound with respect to the flexion angle of the arm is due to the reflection of the bone conduction sound at the elbow joint, the change is regular with a peak around 90 degrees. This is because the change can be approximated by a quadratic function which is a simple convex and concave polynomial. In addition, since three points are sufficient for creating an approximate expression, there is an advantage that calibration is easy.

Next, the elbow flexion angle is estimated using the function approximation obtained by calibration. When the state where the humerus and ulna are horizontal is 0 degrees, and the state where the humerus is at right angles is 90 degrees, when bending from 0 degrees to 90 degrees in 15 degree increments, the vibration input actuator 170 (the small speaker) respectively A vibration was input 10 times to generate a bone conduction sound, and a change in amplitude was obtained with the bone conduction microphone 180 (the auscultatory stethoscope type bone conduction microphone). Accordingly, the standard deviation of the error between the result of the bend angle estimation and the true bend angle was obtained and evaluated. The input signal from the small speaker is a sinusoidal wave having a constant amplitude and a frequency of 800 Hz, and vibrations having a signal length of 0.01 sec are intermittently input at intervals of 1 sec or more. did.

The results are shown in FIGS. 11a and 11b. FIG. 11a shows the estimation result of the subject A, and FIG. The vertical axis represents the estimated bending angle, and the horizontal axis represents the true bending angle. The dotted line is a point where the estimated bending angle and the true bending angle coincide. As a result of the evaluation experiment, it was shown that both the subjects A and B can estimate the elbow flexion angle with an error of about 5 to 7 degrees.

An embodiment in which the elbow flexion angle estimation and the collision position estimation by tapping (striking) a finger on the arm can be performed with the same arm will be described as a fourth embodiment. In order to perform the two estimations, first, it is necessary to separate the bone conduction sound generated by the vibration input actuator from the bone conduction sound generated by the collision by the finger tap. Therefore, the frequency characteristics of each bone conduction sound were analyzed, and a method for separating each bone conduction sound by frequency domain filtering was examined.

In the examination, as shown in the actual mounting state of FIG. 12, one vibration input actuator 190 and one bone conduction microphone 200 are attached to the elbow head, and one bone conduction microphone 210 is attached to the ulnar styloid process. In this state, the forearm between the bone conduction microphones 200 and 210 is tapped with the user's own finger. At this time, vibration is continuously input from the vibration input actuator 190, and bone conduction sound signals are acquired by the bone conduction microphones 200 and 210. Here, the above small speaker is used as the vibration input actuator 190, and the above stethoscope type bone conduction microphone is used as the bone conduction microphones 200 and 210.

FIGS. 13a and 13b are results obtained by acquiring the bone conduction sound amplitudes generated by the collision when the predetermined position of the forearm is tapped with the user's own finger with the bone conduction microphones 200 and 210, respectively (FIG. 13a: ulna) Stemoid, FIG. 13b: elbow head). 14a and 14b show the results of frequency conversion of these (FIG. 14a: ulnar styloid process, FIG. 14b: elbow). FIG. 15a shows the result of obtaining the amplitude of the bone conduction sound input from the elbow head through the small speaker. FIG. 15b shows the result of frequency conversion. The input signal from the small speaker is a sine wave having a constant amplitude and a frequency of 800 Hz, and the length of one input signal is 0.01 sec.

From these results, bone conduction sound generated by a finger tap collision exists in a frequency region of approximately 500 Hz or less, whereas bone conduction sound input from a small speaker exists in the vicinity of 800 Hz which is an input signal. It was shown that. Although the length of one input signal is 0.01 sec, the same result is obtained for a long time signal. That is, the bone conduction sound generated by the tap collision can be recognized by extracting a signal of 600 Hz or less, preferably 50 to 600 Hz, using a bandpass filter. It has been found that the bone conduction sound input by the vibration input actuator can be recognized by extracting a signal of 780 to 820 Hz using a bandpass filter. Note that frequency filtering using FFT as a bandpass filter was performed.

Next, a fourth evaluation experiment in which bending angle estimation and collision position estimation for subjects A and B are performed with the same arm will be described. For subjects A and B, as shown in FIG. 12 described above, the vibration input actuator 190 (the small speaker) and the bone conduction microphone 200 (the stethoscope-type bone conduction microphone) are attached to the elbow head one by one, One bone conduction microphone 210 (the stethoscope-type bone conduction microphone) is attached to the ulnar styloid process. In addition, since vibration of a small speaker can be removed by frequency filtering, the vibration input actuator 190 and the bone conduction microphone 200 on the elbow bone side can be combined into one device.

First, calibration was performed on subjects A and B with respect to bending angle estimation and collision position estimation as initial work. Calibration for collision position estimation is performed in order to set a threshold value for the signal level difference L previously adjusted to the subject because individual differences occur due to the effects of arm length, muscle attachment, and the like. As shown in FIG. 16, the forearm is divided into three sections P1, P2 and P3. Each of these three sections was hit 10 times with the subject's own finger, and the change in the amplitude of the bone conduction sound was acquired with the bone conduction microphones 200 and 210. An SVM (support vector machine) model for the signal level difference L was created from the signal data, and a threshold value was set. During this time, no bone conduction sound is generated by the input from the vibration input actuator 190.

Next, calibration was performed to obtain an approximate function of the bending angle and the amplitude of the bone conduction sound for estimating the bending angle of the elbow. Bone conduction sound is continuously generated by the elbow head vibration input actuator 190 at the elbow flexion angles of 0 degrees, 45 degrees, and 90 degrees, and amplitude is generated by the bone conduction microphone 210 of the ulnar styloid process. Got the change. From this change in amplitude, the function approximation of the bending angle and amplitude by a second-order polynomial was performed using the least square method, and a function approximation expression was created. During this period, no bone conduction sound is generated due to the collision.

After completion of these calibrations, the collision position is estimated in a state where an 800 Hz sine wave is continuously input from the vibration input actuator 190. The forearm P1, P2, and P3 sections were hit 50 times each with the subject's own finger and evaluated with the correct rate of discrimination of the collision position. The bend angle is estimated by bending the elbow in 15 ° increments from 0 ° to 90 °, and at the same time hitting the forearm with a finger and simultaneously sending a 0.1 sec signal from the small speaker, which is the vibration input actuator 190, 10 times each. The evaluation was performed using the actual elbow flexion angle and the standard deviation of the error.

FIG. 17 a shows the estimation result of the collision position of the subject A, and FIG. The horizontal axis and the vertical axis are the maximum amplitudes of the bone conduction sounds obtained by the bone conduction microphones 200 and 210 attached to the ulnar styloid process and the elbow head, respectively. The dotted line and the alternate long and short dash line indicate threshold values for discriminating between P1 and P2 and P2 and P3. The part surrounded by the horizontal axis and the dotted line is P1, the part surrounded by the dotted line and the dashed line is P2, and the vertical axis is one point. A portion surrounded by a chain line is determined as P3. The markers in FIGS. 17a and 17b are assumed to have been correctly estimated when the collision position hit with an actual finger matches the determination result obtained by the threshold processing. The evaluation results showed that the coincidence rate was 97.3% for subject A and 92.0% for subject B, and a recognition rate of 90% or more was obtained.

18a shows the result of subject A and FIG. 18b shows the result of subject B's elbow flexion angle estimation. The horizontal axis and the vertical axis are the bending angles estimated as true bending angles, respectively, and the dotted lines are the points where both coincide. In the evaluation results, the elbow flexion angle could be estimated with an error of 3 to 5 degrees for both subjects A and B. FIG. 19 shows a processing flow of the fourth evaluation experiment.

By performing calibration and frequency filtering as described above, it is possible to estimate the elbow flexion angle with the same arm and the collision position by finger tap on the arm individually. It was shown that the position estimation can be used simultaneously.

Next, an embodiment used as a robot control interface is shown as Example 5. As shown in FIG. 12, the operator attaches the vibration input actuator 190 (the small speaker) and the bone conduction microphone 200 (the stethoscope-type bone conduction microphone) one by one to the elbow head, and One bone conduction microphone 210 (the stethoscope type bone conduction microphone) is attached to the dent process. The operation target is HRP-2m Chromet (manufactured by General Robotics). By tapping (striking) the arm with the user's finger, the collision position is estimated, and the estimation result is used as a command-type interface that is used as a robot operation command.

As shown in FIG. 20, the estimation results of the five collision positions including the forearm P1, P2, and P3 sections and the hand and shoulder are associated as five operation commands. Furthermore, as shown in FIG. 21, the estimation results of the collision position when tapped twice are combined and associated as nine operation commands. Here, in the operation command with two taps, the tap on the shoulder and the hand is used as a reset button when the first input is mistaken. In addition, when hitting a part other than the forearm such as the hand and shoulder with a finger, the level of the signal is lower than when the forearm is tapped compared to the bone conduction sound transmitted through the forearm, and the collision position can be identified relatively easily. it can.

The operation command to the robot operation based on the estimation result of the elbow flexion angle is to continuously estimate the flexion angle and bend and extend the robot arm according to the obtained flexion angle.

In Example 5, in order to remove bone conduction sound other than bone conduction sound inputted from the small speaker and prevent malfunction, a bandpass filter in the range of 780 to 820 Hz is obtained so that only an 800 Hz signal as an input signal can be acquired. To remove low-frequency noise caused by wrist flexion and arm movement and bone conduction sound generated from the hand. A signal of 50 Hz or less was also cut by a high pass filter. Moreover, by converting the frequency of bone conduction sound and adopting only when the spectrum intensity peak near 800 Hz as an input signal, it is possible to reduce misrecognition when wearing in daily life and performing daily work. did.

FIG. 22 shows a state in which the collision position estimation result is used for robot operation. You can see that the robot's arms and legs are moving according to the collision position of the pilot's tap. FIG. 23 shows an actual state in which the robot arm moves according to the estimation of the elbow flexion angle of the operator. It can be seen that the arm of the robot is bent according to the bending of the elbow of the arm.

The configuration of the system used in Examples 1 to 5 is shown in FIG. In each example, an AD / DA board (Interface CSI-360116) is used for driving signal output to the vibration input actuator and signals from the bone conduction microphone, and a notebook PC (DELL PRECISION / M4400) ) Was used to process the signal obtained from the bone conduction microphone and control the vibration input actuator.

INDUSTRIAL APPLICABILITY The present invention is useful for an interface system using bone conduction sound in an interface used by being worn on the body. For example, toys, games, home appliances,
A system useful as an input interface for a robot or the like can be provided.

1000 User 2000 User 10 Vibration input actuator 20 Bone conduction microphone 30 Vibration input actuator 40 Bone conduction microphone 50 Bone conduction microphone 80 Vibration input actuator 90 Bone conduction microphone 100 Bone conduction microphone 110 Bone conduction microphone 120 Vibration input actuator 130 Bone conduction microphone 140 Bone conduction microphone 150 Vibration input actuator 160 Bone conduction microphone 170 Vibration input actuator 180 Bone conduction microphone 190 Vibration input actuator 200 Bone conduction microphone 210 Bone conduction microphone

Claims (10)

An interface system used by being worn on the body,
A vibration input actuator that is mounted on a first part of the body and generates a bone conduction sound by inputting vibration;
A bone conduction microphone mounted on a second part of the body for obtaining the bone conduction sound;
A signal processing means including a function of estimating a posture of the body based on the bone conduction sound signal acquired by the bone conduction microphone;
Input information generating means for generating predetermined input information from a processing result by the signal processing means;
An interface system comprising: The interface system according to claim 1 , wherein the first part is an arm, and the second part is the same arm as the first part. Said signal processing means, interface system according to claim 1 or 2, characterized in that it comprises a bending angle estimating means for estimating the bending angle of the elbow by bending, the bending of the elbow as the posture of the body. An interface system used by being worn on the body,
A vibration input actuator that is mounted on a first part of the body and generates a bone conduction sound by inputting vibration;
A bone conduction microphone mounted on a second part of the body for obtaining the bone conduction sound;
A bone conduction microphone mounted on a third part of the body for obtaining the bone conduction sound;
Based on the bone conduction sound signal acquired by the bone conduction microphone attached to the second part and the bone conduction sound signal obtained by the bone conduction microphone attached to the third part, A signal processing means including a function of estimating a posture and a function of estimating a contact position by contact with the body;
Input information generating means for generating predetermined input information from a processing result by the signal processing means;
An interface system comprising: The interface system according to claim 4 , wherein the first part, the second part, and the third part are the same arm. Said signal processing means, interface system according to claim 4 or 5, characterized in that it comprises a bending angle estimating means for estimating the bending angle of the elbow by bending, the bending of the elbow as the posture of the body. 6. The signal processing means according to claim 4 or 5 , wherein the contact position by contact with the body is a collision position by a tap on an arm, and includes a collision position estimation means for estimating the collision position. Interface system. The signal processing means includes a bending angle estimating means for estimating a bending angle due to bending and stretching of the elbow as a posture of the body, and a collision position in which the estimation of the collision position due to the tap on the arm is estimated as the contact position due to contact with the body. e Bei and estimating means, interface system according to claim 4 or 5, characterized in that each of said collision position and the bending angle can be estimated separately. The bending angle estimation means according to claim 3, 6, 8, characterized in that an estimate of the bending angle based on a change in amplitude of said bone conduction sound accompanying the bending-bending of the elbow Interface system. The interface system according to claim 7 or 8 , wherein the collision position estimation means estimates a collision position by a tap on an arm based on a change in amplitude of the bone conduction sound accompanying the tap.