JP2014071823A - Information input device, specific frequency extraction method, and specific frequency extraction program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To keep the practicability and analyze the frequency information based on vibration generated by a fingertip at a high frequency resolution and high time resolution, in a frequency analysis technology.SOLUTION: When a frequency component extraction (Fourier transformation) is executed, a specific frequency belonging to a low frequency is certainly extracted, and simultaneously fine monitoring is performed. In other words, the time width and transition width of a cut window are set independently. In other words, the cut window is set at 0.1 sec and the transition width is set at 0.01 sec. As a result, while transition (time series variation) is performed every transition width, the frequency component in the time width of the cut window is Fourier-transformed, the frequency component of the specific frequency can be certainly extracted, and fine monitoring of a tap operation is allowed.

Description

  The disclosed technology relates to an information input device, a specific frequency extraction method, and a specific frequency extraction program.

  Various wearable computers (wearable information processing devices) that function as devices for sensing human situations and hands-free interfaces have been proposed due to the miniaturization of various sensors and the development of communication infrastructure. For example, in a wrist-worn wearable computer, a feature of human fingertip motion is detected.

  In wrist-worn wearable computers, the frequency characteristics and time-series information of vibrations that occur when a fingertip contacts an object are used to estimate the fingertip portion where a collision or contact (hereinafter collectively referred to as “contact”) has occurred. Estimate the touched object. However, the prior art does not clearly show a technique relating to extraction of frequency characteristics and time-series information in consideration of characteristics of vibration generated at the time of contact. Note that the expression “fingertip” mainly consists of contact at the tip of the finger, but is not specialized for the tip if the type of finger can be specified.

JP-A-7-121294 Japanese Patent Laid-Open No. 10-198478 JP 2005-525635 Gazette JP 11-338597 A

  It is known that when a fingertip and an object come into contact with a human fingertip operation, the vibration generated by the contact has a characteristic of a pulse waveform whose amplitude changes abruptly in a minute time. It is also known that the frequency that is the main component of this vibration is a low frequency within a so-called audible frequency band of 50 to 300 Hz.

  Therefore, in order to extract the characteristics of the pulse vibration generated by the contact between the fingertip and the object, it is necessary to capture a time series change of minute time for each fine frequency around the main component frequency (for example, 100 Hz). is there.

  As described above, this vibration may be weak. However, if it is a wrist-worn wearable computer equipped with a sensor capable of detecting vibration, air vibration from the fingertip or vibration transmitted through the body is detected at the wrist part. Can be captured.

  However, as a conventional frequency analysis technique, a method of extracting a vibration section found in short-time Fourier transform is mainly used. In the extraction method, the frequency resolution and the time resolution at the time of frequency analysis are in a so-called trade-off relationship. For this reason, it is difficult to simultaneously capture the frequency characteristics of vibrations generated at the fingertip with high frequency resolution and high time resolution.

  In addition, when wearing a wrist-worn wearable computer is assumed to be always worn, the installation of advanced arithmetic processing equipment leads to an increase in the size of the device due to an increase in the number of parts, resulting in a loss of practicality. .

  As one aspect, the disclosed technology reliably extracts a specific frequency generated by contact between a specific part of a human body and an object from frequency information detected by a detection unit attached to the human body while maintaining practicality. Is the purpose.

  The disclosed technology can be attached to a human body and includes a detection unit that detects frequency information transmitted from the human body.

  Moreover, the analysis part which analyzes the frequency distribution of the said frequency information detected by the said detection part is provided. The analysis unit sets a frequency analysis time width necessary for analyzing a frequency distribution including a specific frequency component that is assumed to be a contact between a predetermined specific part of the human body and an object. The analysis unit sets a transition width necessary for maintaining the detection accuracy of the specific frequency when the frequency analysis time width is changed in time series for analysis. The analysis unit sets the frequency analysis time width and the transition width to independent resolutions.

  Furthermore, an extraction unit is provided that extracts the specific frequency based on the frequency distribution analyzed by the analysis unit.

  In addition, the extraction unit includes an output unit that outputs an operation signal indicating that a contact operation has been performed by the contact operation part by extracting the specific frequency.

  As one aspect, the disclosed technology has an effect that frequency information based on vibration generated at a fingertip can be analyzed with high frequency resolution and high time resolution while maintaining practicality.

It is a block diagram which shows the structure of the information input device which concerns on this Embodiment. It is a block diagram which shows the hardware constitutions of the information input device which concerns on this Embodiment. It is a perspective view which shows the state which mounted | wore the wrist with the wearable apparatus as an information input device which concerns on this Embodiment. It is sectional drawing which shows the structure of the apparatus main body of the information input device which concerns on this Embodiment. FIG. 4 is a characteristic diagram of a detection signal detected by a body conduction sound sensor according to the present embodiment. It is a specific figure which shows the result of having analyzed the frequency component by stationary Fourier transformation of the specific frequency. It is a specific figure which shows the result of having provided the silence area before and behind the specific frequency, and having analyzed the frequency component by steady Fourier transformation. FIG. 6 is a frequency component characteristic diagram obtained by analyzing the detection signal of FIG. 5 by an analysis unit according to the present embodiment. FIG. 6 is a characteristic diagram (part 1) illustrating a procedure for determining a peak value in the detection signal of FIG. 5; FIG. 6 is a characteristic diagram (part 2) illustrating a procedure for determining a peak value in the detection signal of FIG. 5; It is a form figure of RNN performed with the specific frequency extraction part which concerns on this Embodiment. It is a characteristic figure of the frequency-feature-value for every object of a different kind. It is a control flowchart which shows the specific frequency extraction process sequence in the information input device which concerns on this Embodiment. It is each a front view of the palm side of the wrist which shows the candidate of the position which attaches the sound collection element of a wearable apparatus, and the back side of a hand. It is a sound pressure characteristic figure of each finger | toe in mounting | wearing point A1, A2, A3, B1, B2, B3 shown in FIG.

  FIG. 1 is a configuration diagram of an information input device 10 according to the present embodiment. The information input device 10 includes a body conduction sound sensor 12. The body sound sensor 12 is an example of a detection unit of the disclosed technology. As shown in FIG. 3, the body conduction sound sensor 12 also serves as a wristband type wearable device that can be worn on the human body (here, the wrist 14).

  The output signal of the body sound sensor 12 is connected to the input I / F 22 of the apparatus main body 20 via the communication cable 18. In the present embodiment, the body-conducting sound sensor 12 and the apparatus main body 20 are connected by the wired communication cable 18, but it is not necessary to be wired, and wireless communication may be applied.

  Further, the apparatus main body 20 is provided with an output I / F 24. An operation information receiving device 26 can be connected to the output I / F 24.

  Various devices including a PC (personal computer), a tablet terminal, a portable terminal, a communication processing terminal, a monitor, a speaker, and a lamp can be connected as the operation information receiving device 26.

  As shown in FIG. 1, the apparatus body 20 of the information input apparatus 10 includes a detection signal capturing unit 28. The detection signal capturing unit 28 is connected to the input I / F 22.

  The device body 20 of the information input device 10 includes a signal cutout unit 30, a set value storage unit 32, and a frequency analysis unit 34. The frequency analysis unit 34 is an example of an analysis unit of the disclosed technology. The set value storage unit 32 stores cutout window information and transition width information for frequency analysis.

  Further, the device main body 20 of the information input device 10 includes a specific frequency extraction unit 36, an operation presence / absence determination unit 38, an object type determination unit 40, an object type / feature amount table storage unit 42, and an operation information output unit 44. The operation information output unit 44 is connected to the output I / F 24. The specific frequency extraction unit 36 is an example of an extraction unit of the disclosed technology.

  As shown in FIG. 2, the apparatus main body 20 of the information input apparatus 10 includes a microcomputer including a CPU 70, a RAM 72, a ROM 74, an I / O 76, and a bus 78 such as a data bus and a control bus for interconnecting them. Yes.

  An input I / F 22 and an output I / F 24 are connected to the I / O 76. The output I / F 24 and the operation information output unit 44 are examples of the output unit of the disclosed technology.

  The specific frequency extraction processing program executed in the apparatus main body 20 of the information input apparatus 10 includes a detection signal capturing process 28P, a signal cutout process 30P, a set value storage process 32P, a frequency analysis process 34P, and a specific frequency extraction process 36P.

  The specific frequency extraction processing program executed by the apparatus main body 20 of the information input apparatus 10 includes an operation presence / absence determination process 38P, an object type determination process 40P, an object type / feature amount table storage process 42P, and an operation information output process 44P. .

  The CPU 70 operates as the detection signal capturing unit 28 illustrated in FIG. 1 by executing the detection signal capturing process 28P.

  The CPU 70 operates as the signal cutout unit 30 illustrated in FIG. 1 by executing the signal cutout process 30P.

  The CPU 70 operates as the signal cutout unit 32 illustrated in FIG. 1 by executing the signal cutout process 32P.

  The CPU 70 operates as the frequency analysis unit 34 illustrated in FIG. 1 by executing the frequency analysis process 34P.

  The CPU 70 operates as the specific frequency extraction unit 36 illustrated in FIG. 1 by executing the specific frequency extraction process 36P.

  The CPU 70 operates as the operation presence / absence determination unit 38 illustrated in FIG. 1 by executing the operation presence / absence determination process 38P.

  The CPU 70 operates as the object type determination unit 40 illustrated in FIG. 1 by executing the object type determination process 40P.

  The CPU 70 operates as the object type-feature amount table storage unit 42 illustrated in FIG. 1 by executing the object type-feature amount table storage process 42P.

  The CPU 70 operates as the operation information output unit 44 illustrated in FIG. 1 by executing the operation information output process 44P.

  (Configuration of body conduction sensor 12 “wearable device”)

  FIG. 3 shows a wearable device that also serves as the body conduction sensor 12 of the information input device 10.

  The body sound sensor 12 includes a wristband unit 46 that is worn on the wrist 14 of the human body. The wristband portion 46 is formed in a ring shape with a flat plate having elastic force, and is held by the wrist 14 with the elastic force. As the wristband portion 46, in addition to a structure that is held by an elastic force, a buckle type that is selectively passed through any one of a plurality of holes formed in advance and is held on the wrist according to the circumference of the wrist. Applicable.

  A sound collection unit 48 is attached to a part of the wristband unit 46.

  As shown in FIG. 4, the housing of the sound collection unit 48 is cylindrical. The upper end surface in FIG. 4 of the housing of the sound collecting unit 48 is closed, and the end of the surface side facing the wrist 14 is enlarged in a tapered shape to form an opening 50. A thin-film sound collecting element 52 is attached to the opening 50 so that when the wristband portion 46 is attached to the wrist 14, it directly contacts the skin of the wrist 14. More specifically, the sound collecting element 52 faces the total finger extensor muscle (hereinafter, referred to as “wrist specific part”) that is transmitted from the ulnar styloid process to the flexure in the wrist 14.

  One end portion of the communication pipe 54 is attached to a part of the peripheral surface of the sound collecting portion 48 and communicates with the inner space of the sound collecting portion 48. A sound collecting microphone 56 (for example, a condenser microphone) is accommodated in the communication pipe 54. The sound collecting microphone 56 has a function of detecting the sound in the inner space of the sound collecting portion 48 generated by the vibration of the sound collecting element 52. In other words, the sound collecting microphone 56 mainly detects body-conducted sound transmitted from the wrist specific part without picking up surrounding sounds (outside sounds).

  The signal detected by the sound collecting microphone 56 is sent to the input I / F 22 (see FIG. 3) of the apparatus main body 20 via the communication cable 18.

  In the present embodiment, as a body-conducted sound, as shown in FIG. 3, vibration when a human fingertip 58 contacts (collises) an object 60 is assumed. That is, by extracting the vibration (frequency) generated when the fingertip 58 and the object 60 come into contact with each other, the contact operation by the fingertip 58 is applied as an information input operation.

  (Configuration of the apparatus body 20)

  As shown in FIG. 1, the input I / F 22 is connected to a detection signal capturing unit 28. In the detection signal capturing unit 28, the body conduction sound sensor 12 captures a detection signal (a raw signal of time-series detection intensity).

  FIG. 5 is an example of a detection signal of the body conduction sensor 12. The detection signal includes various frequencies including a specific frequency when an operation (hereinafter sometimes referred to as a “tap operation”) of touching the object 60 (an example is shown in FIG. 3) with the fingertip 58 is performed. Signals are mixed.

  As shown in FIG. 1, the detection signal capturing unit 28 is connected to the signal clipping unit 30.

  The signal cutout unit 30 takes in cutout window information and transition width information from the setting value storage unit 32 in order to set a time interval (cutout window) for analyzing the frequency distribution and a transition time width of the cutout window.

  In the signal cutout unit 30, the detection signal corresponding to the time width of the cutout window information taken in from the detection signal taken in by the detection signal fetching unit 28 is sequentially generated for each transition time corresponding to the fetched transition width information. Send to analysis unit 34

  The frequency analysis unit 34 analyzes the frequency distribution for each clipping window by Fourier transform.

  (Cutout window information and transition width information)

  Here, the frequency of the signal detected by the body-conducting sound sensor 12 by the tap operation with the fingertip 58, that is, the specific frequency is a low frequency region in the audible frequency band, and assumes 50 Hz to 300 Hz. Yes. The frequency analysis unit 34 extracts frequency components including a specific frequency by Fourier transform.

  FIG. 6 and FIG. 7 show the results of Fourier transform when the specific frequency is a 200 Hz sine wave as an example. In the Fourier transform, when a cutout window (time width) is set, the transition width is set to be the same as or 1/2 of the cutout window with a correlation with the cutout window (hereinafter referred to as “steady Fourier transform”). .

  From the frequency component analysis result of FIG. 6, it can be seen that there is no noise and that a specific frequency of 200 Hz can be extracted clearly. However, in the frequency component analysis of FIG. 6, the detection period is indefinite, and the detection signal in a state where it is not known when the tap operation is performed may not be able to be closely monitored.

  On the other hand, FIG. 7 shows an analysis result when a silent period of 0.05 sec is intentionally set for each of the first half and the second half of one cycle of a sine wave with respect to a specific frequency of 200 Hz (cycle is 0.2 sec). It is. In the frequency component analysis result of FIG. 7, the detection period is indefinite, and it is suitable for finely monitoring the detection signal in a state where it is not known when the tap operation is performed, and a specific frequency can be extracted. However, in the frequency component analysis of FIG. 7, the silent section becomes noise, and a wide frequency band is observed.

  Based on the results of one example of FIGS. 6 and 7, in this embodiment, a specific frequency belonging to a low frequency is surely extracted, and fine monitoring is compatible. That is, unlike the “steady Fourier transform” described above, the time width and transition width of the clipping window are set independently.

  In the present embodiment, the extraction window is set to 0.1 sec in order to reliably extract the frequency component of the specific frequency. On the other hand, the transition width is set to 0.01 sec for fine monitoring of the tap operation.

  As a result, as shown in FIG. 1, the frequency component of the time width (0.1 sec) of the clipping window is Fourier-transformed while transitioning (time-series change) by the transition width (0.01 sec).

  As shown in FIG. 5, the frequency analysis unit 34 is sent to the specific frequency extraction unit 36.

  8 is analyzed by the frequency analysis unit 34 with respect to the time series change of FIG. 5 for a total of 0.2 sec, 0.1 sec before and after the time at which the vibration peaks (see arrow A in FIG. 5). The result (frequency distribution characteristic).

  The horizontal axis in FIG. 8 is frequency (Hz), the vertical axis is the sound pressure (dB) as a feature quantity, and each of the plots marked with “+” indicates the value of the discretized frequency component (ui (t )). The variable i is a label number given to each frequency component.

  The detection of the vibration peak is performed by the frequency analysis unit 34. In one embodiment (peak detection 1), as shown in FIG. 9, the sound pressure (dB) at time t is set to St, and from time t to a The S / N ratio is calculated using Nt as the average value (dB) of the sound pressure of the signal before sampling (see region B in FIG. 9). The time t when the SN ratio exceeds a predetermined threshold is set as the frequency analysis start time (the analysis period is time t ± 0.1 sec). Note that the sampling number a may be fixed or may be varied. Since this peak detection 1 is a relative comparison, the peak can be detected regardless of the sound pressure.

  Detection of the vibration peak is performed by the frequency analysis unit 34, and in another embodiment (peak detection 2), as shown in FIG. 10, the sound pressure (dB) at time t is set to St, and 1 at time t. The S / N ratio is calculated with Nt being the sound pressure (dB) of the previous sampling signal. The time t when the SN ratio exceeds a predetermined threshold is set as the frequency analysis start time (the analysis period is time t ± 0.1 sec). Since the peak detection 2 is a relative comparison, the peak can be detected regardless of the sound pressure.

  The specific frequency extraction unit 36 shown in FIG. 1 extracts a specific frequency based on the characteristics of FIG. 8 acquired from the frequency analysis unit 34. In this embodiment, RNN (Recurrent Neural Network) is applied as a method for extracting a specific frequency.

  The form of RNN is shown in FIG. As shown in FIG. 11, in the learning phase, the network weight is updated so as to reduce the error ei (t), and the input time-series change is learned. In this learning phase, the RNN can learn the time series change of the input derived from the pulse vibration (plot ui (t) in FIG. 8). However, a random time-series change of the input (ui (t)) derived from the noise interval cannot be learned. Using this fact, only the input (ui (t)) derived from the pulse vibration is extracted from the error (ei (t)) in the identification phase.

  FIG. 8 shows the result of analysis by the frequency analysis unit 34 for a total of 0.2 sec of 0.1 sec before and after the time when the vibration peaks, as described above. Such characteristic results are sequentially analyzed in time series with a preset transition width.

  That is, in the section where the pulse vibration due to the tap operation exists in the cut-out width of the Fourier transform (section of the cut-out width including the area having the peak at arrow A in FIG. 5), the data sampled in the next cut-out width section The specific frequency can be extracted by learning to converge the error.

  On the other hand, in the section where there is no pulse signal due to the tap operation (section with a cut-out width not including the area having the peak at arrow A in FIG. 5), the noise is an irregular frequency. Cannot be predicted and cannot be converged.

  Here, in the present embodiment, the range error Ei is set in consideration of the case where the error ei (t) is accidentally reduced with respect to noise.

  The range error Ei can be expressed by the following equation (1).

In equation (1), ta indicates a time a sampling before time t.

  Accordingly, in the RNN of FIG. 11, in the identification phase, the frequency component corresponding to the label number i (see FIG. 8) when the range error Ei is continuously equal to or lower than the threshold value is determined as the fingertip 58 and the object 60. This is extracted as the frequency that is the main component of the pulse vibration generated by contact. In the identification phase, an input ui (τ) in a time range in which the range error Ei is continuously below the threshold is extracted as a time-series change of the extracted frequency component.

  As shown in FIG. 1, the result extracted by the specific frequency extraction unit 36 is sent to the operation presence / absence determination unit 38.

  The operation presence / absence determination unit 38 determines that “operation is present” by inputting the label number i extracted when the range error Ei continuously falls below the threshold value extracted by the specific frequency extraction unit 36, and outputs operation information. To the unit 44. Further, the time corresponding to the label number i is determined as the operation time, and is sent to the operation information output unit 44. The operation presence / absence determination unit 38 determines that there is no operation when the specific frequency extraction unit 36 does not extract the label number i when the range error Ei is continuously below the threshold value.

  Further, the operation presence / absence determination unit 38 is connected to the object type determination unit 40, and the sound pressure of the label number i determined as “operation present” is determined to the object type determination unit 40 together with the determination of “operation present”. Information is sent out. An object type / feature quantity table storage unit 42 is connected to the object type determination unit 40.

  In the object type-feature amount table storage unit 42, as shown in FIG. 12, characteristics indicating differences in feature amounts (sound pressure) when a plurality of objects 60 are tapped 50 times with the fingertip 58. The figure is stored.

  A characteristic diagram indicated by a dotted line a in FIG. 12 is a tap operation characteristic when the object 60 is a desk. A characteristic diagram indicated by a thin solid line b in FIG. 12 is a tap operation characteristic when the object 60 is a keyboard. A characteristic diagram indicated by a thick solid line c in FIG. 12 is a tap operation characteristic when the object 60 is a rubber button. 12 is a tap operation characteristic in the case of an air-shot that does not contact the fingertip 58 object 60. Here, it is assumed that swinging is also included in the tap operation.

  The horizontal axis represents frequency, and based on this characteristic diagram, it can be seen that characteristic attenuation and peaks that can distinguish each object 60 exist in a period of 50 Hz to 100 Hz belonging to a specific frequency. Therefore, the object type discriminating unit 40 discriminates the type of the object based on the sound pressure information of the specific frequency input from the operation presence / absence determining unit 38 and sends it to the operation information output unit 44.

  The operation information output unit 44 outputs output information generated by integrating the operation presence / absence information from the operation presence / absence determination unit 38 and the object determination information from the object type determination unit 40 via the output I / F 24. Send it out. The output information may be variable depending on the type of the operation information receiving terminal 36 connected to the output I / F 24. The output information may be unified regardless of the type of the operation information reception terminal 26 connected to the output I / F 24 and analyzed on the operation information reception terminal side.

  Hereinafter, the operation of the present embodiment will be described.

  FIG. 13 is a control flowchart showing a specific frequency extraction processing procedure in the information input apparatus according to the present embodiment.

  This specific frequency extraction processing control is activated when the information input device 10 is turned on.

  In step 100, initial setting (memory clear or the like) is executed, and then the process proceeds to step 102 where the detection signal from the body conduction sound sensor 12 is captured. In the initial setting, the sensitivity adjustment of the body conduction sound sensor 12 may be automatically or manually executed regularly or irregularly.

  In the next step 104, it is determined whether or not the body conduction sound sensor 12 has detected a predetermined pulse vibration. If a negative determination is made, the process returns to step 102. If an affirmative determination is made in step 104, it is determined that the analysis start time is reached, and the routine proceeds to step 106.

  In step 106, the time width and transition time width of the cutout window at the time of frequency analysis (Fourier transform processing) stored in advance in the set value storage unit 32 are read out, and the process proceeds to step 108 to extract frequency components.

  In the present embodiment, when performing frequency component extraction (Fourier transform), a specific frequency belonging to a low frequency is surely extracted, and fine monitoring is compatible. That is, the time width and transition width of the cutout window are set independently.

  That is, in this embodiment, the cutout window is set to 0.1 sec and the transition width is set to 0.01 sec. As a result, as shown in FIG. 1, the frequency component of the time width (0.1 sec) of the extraction window is Fourier-transformed while transitioning (time-series change) by the transition width (0.01 sec). The frequency component can be reliably extracted and fine monitoring of the tap operation can be performed.

  In the next step 110, the analysis data of the frequency component is sent to the specific frequency extraction unit 36, and then the RNN process is executed in step 112. That is, the analysis data ui (t) and the next time series analysis data ui (t + 1) are sequentially input to the input layer shown in FIG.

  In the next step 114, the range error Ei is calculated based on the error ei (see the above-described equation (1)), and the process proceeds to step 116.

  In step 116, it is determined whether or not the error range Ei is greater than or equal to the threshold value for all label numbers i. If a negative determination is made, a time range that is continuously below the threshold is included. Therefore, it is determined that it is not the end time of the RNN, the process returns to step 112, and the above process is repeated.

  If an affirmative determination is made in step 116, it is determined that the time range continuously below the threshold is not included and the end time of the RNN is reached, and the routine proceeds to step 118.

  In step 118, a time range in which the range error Ei is continuously less than or equal to the threshold value is searched, and then the process proceeds to step 120, where the specific frequency generated due to the tap operation and the frequency characteristics including the time series change are included. To extract.

  In the next step 122, the presence / absence of a tap operation is determined based on the extracted frequency characteristics, and then the process proceeds to step 124, the type of the object 60 on the tapped side is determined, and the process proceeds to step 126. .

  In step 126, operation information including operation presence / absence information and object type information is output to the outside, and the process returns to step 102. This routine ends when the information input device 10 is powered off. Further, the operation information to be output may be only the presence / absence of a tap operation.

  In the present embodiment, the body sound sensor 12 is applied as a detection unit that detects frequency information transmitted from the human body. However, the sensor is limited to the body sound sensor 12 as long as the sensor can sense low-frequency vibrations. Is not to be done.

  For example, as one example, an acceleration sensor can be applied. The acceleration sensor can detect the movement of the fingertip. However, the body sensor sound sensor 12 is advantageous in that the acceleration sensor detects a large movement of the entire arm and the vibration generated at the fingertip 58 may be extinguished by the vibration of the movement of the entire arm.

  Further, as another embodiment, a microphone using air conduction sound (air vibration) can be applied. However, this microphone is a so-called steady-state microphone, and the body-conducting sound sensor 12 has a possibility of picking up not only the contact sound between the fingertip 58 and the object 60 but also the surrounding constant sound. It is advantageous.

  Moreover, in this Embodiment, as a mounting position of the body-conducted sound sensor 12, the thin film-like sound collection element 52 is on the total extensor extensor muscle which transmits a calcaneus from the ulnar pedicle process in the wrist 14 (wrist specific site | part). However, six candidates shown in FIG. 14 are conceivable including the wrist specific position.

  In FIG. 14, as the attachment position of the thin film sound collecting element 52, along the circumferential direction of the wrist 14, there are three places (A1, A2, A3) on the back side of the hand from the thumb side, and three places on the palm side from the thumb side ( B1, B2, B3) were set. This embodiment corresponds to the position A3 in FIG.

  FIG. 15 shows that each of the four attachments performed a plurality of tap operations such that the subject 60 applied impacts of the same strength with the thumb, the index finger, the middle finger, the ring finger, and the little finger at each attachment position in FIG. It is the characteristic view which showed the average value and the standard deviation of the sound pressure in the case. From the measurement result of FIG. 15, a part with high sound pressure is selected with any finger. As a result, in the present embodiment, A3 in FIG. 14 (on the total extensor extensor extending from the ulnar styloid process to the flexor) is selected, but it may be another part. Further, not only the measurement result of FIG. 15 but also an optimal site may be selected depending on the difference in the subject, the number of subjects, the type of object, and the like.

  It should be noted that the specific part (vibration pulse input end) of the human body that is tapped on the object 60 is not limited to the finger 58 of the hand, and all parts of the human body can be the specific part. In this case, the body sound sensor 12 may be disposed through a holding body instead of the wristband 46, with a portion where a vibration pulse input from a specific portion is transmitted as body sound as an output end.

  (Modification "Reducing the burden on RNN")

  In the present embodiment, it is possible to limit the number of frequency components (label numbers i) applied to inputs in the RNN in the specific frequency extraction unit 36 (see FIG. 1).

  That is, as can be seen from the object type-feature amount table of FIG. 12, it can be seen that characteristic attenuation and peaks exist in the period of 50 Hz to 100 Hz belonging to the specific frequency regardless of the type of the object 60.

  Therefore, the specific frequency extraction unit 36 further narrows down the specific frequency (50 Hz to 300 Hz) to 100 Hz or less, whereby the burden of calculation processing in the specific frequency extraction unit 36 can be reduced.

  In addition, although it demonstrated using the numerical value which does not interfere with the technique of an indication above, the technique of an indication is not limited to the numerical value used for said description.

  All documents, patent applications and technical standards mentioned in this specification are to the same extent as if each individual document, patent application and technical standard were specifically and individually described to be incorporated by reference. Incorporated by reference in the book.

Regarding the above embodiment, the following additional notes are disclosed.
(Appendix 1)

  A detection unit that can be attached to a human body and detects frequency information transmitted from the human body;

  A frequency analysis time width necessary for analyzing a frequency distribution including a specific frequency component that is assumed to be a contact between a predetermined specific part of a human body and an object, and the frequency analysis time width is changed in time series. An analysis unit for analyzing the frequency distribution of the frequency information detected by the detection unit by setting transition widths necessary for maintaining the detection accuracy of the specific frequency to be independent resolutions,

  Based on the frequency distribution analyzed by the analysis unit, an extraction unit that extracts the specific frequency;

In the extraction unit, an output unit that outputs an operation signal indicating that a contact operation has been performed by the specific part by extracting the specific frequency;
An information input device.

(Appendix 2)
Based on the feature quantity of the specific frequency extracted by the extraction unit, a specifying unit that specifies the object that the specific part is in contact with, and
The information input device according to appendix 1, which has:

(Appendix 3)
The specific part is

  The information input device according to supplementary note 2, further comprising a storage unit in which a correlation table between a specific frequency and a feature amount is stored for each of a plurality of different objects.

(Appendix 4)
The information input device according to supplementary note 2 or supplementary note 3, wherein the feature amount includes sound pressure information.

(Appendix 5)
The information input device according to any one of supplementary notes 1 to 4, wherein the transition time width is 1/10 or less of the analysis time width.

(Appendix 6)
The information input device according to any one of supplementary notes 1 to 5, wherein the specific frequency belongs to a low frequency range of 20 Hz to 500 Hz, preferably 50 Hz to 300 Hz, more preferably 50 Hz to 100 Hz, in an audible frequency band.

(Appendix 7)
Any one of appendix 1 to appendix 6 in which the analysis in the analysis unit is a Fourier transform, the transition time width is a time width of a clipping window in the Fourier transform, and the transition time width is a transition width in the Fourier transform The information input device according to one.

(Appendix 8)
In the frequency distribution analyzed by the analysis unit by the extraction unit, the error between the predicted value of the feature quantity displaced before and after the frequency component sampled continuously in time and the actual measurement value continues for a predetermined period. The information input device according to any one of Supplementary Note 1 to Supplementary Note 7, wherein the specific frequency is extracted based on frequency information sampled in a time range that is equal to or less than a threshold value.

(Appendix 9)
Any one of Supplementary Note 1 to Supplementary Note 8, wherein the specific part is a fingertip, and the detection unit is a body-conducting sound sensor that is attached to a wrist part on a total finger extensor extending from the ulnar styloid process to the radius. The information input device described in 1.

(Appendix 10)
The start time of the frequency information analyzed by the analysis unit is S, where the feature quantity of the signal at time t is S, and the signal before the predetermined time from the time t, or the signal of the average value of the signal before the predetermined time is N. The information input device according to any one of supplementary notes 1 to 9, which is determined based on a ratio.

(Appendix 11)
A frequency analysis time width having a resolution of one period or more of a specific frequency assumed to be a contact between a predetermined specific part of a human body and an object, and a time width necessary for maintaining the detection accuracy of the specific frequency The transition time width with the resolution is set independently, and the frequency distribution of the frequency information at the time of contact between the specific part and the object detected by the detection unit attached to the human body is analyzed,

  A specific frequency extracting method including extracting the specific frequency based on the analyzed frequency distribution.

(Appendix 12)
The specific frequency extraction method according to claim 11, wherein the object that the specific part contacts is specified based on the extracted characteristic amount of the specific frequency.

(Appendix 13)
The specific frequency extraction method according to supplementary note 12, wherein the object is specified based on a correlation table between a specific frequency and a feature amount stored in advance for each of the plurality of different objects.

(Appendix 14)
The specific frequency extraction method according to supplementary note 2 or supplementary note 3, wherein the feature amount includes sound pressure information.

(Appendix 15)
The specific frequency extraction method according to any one of Supplementary Note 11 to Supplementary Note 14, wherein the transition time width is 1/10 or less of the analysis time width.

(Appendix 16)
The specific frequency extraction method according to any one of Supplementary Note 11 to Supplementary Note 15, wherein the specific frequency belongs to a low frequency range of 20 Hz to 500 Hz or less, preferably 50 Hz to 300 Hz, more preferably 50 Hz to 100 Hz in an audible frequency band.

(Appendix 17)
The analysis is a Fourier transform, the transition time width is a time width of a cut-out window in the Fourier transform, and the transition time width is a transition width in the Fourier transform. Specific frequency extraction method.

(Appendix 18)
In the analyzed frequency distribution, the error between the predicted value of the feature quantity displaced before and after the frequency component sampled continuously in time and the actual measurement value are continuously below the threshold value for a predetermined period. The information input device according to any one of supplementary notes 11 to 17, which extracts the specific frequency based on frequency information sampled in a time range.

(Appendix 19)
Any one of Supplementary notes 1 to 18, wherein the specific part is a fingertip, and contact with the object is detected by a body-conducting sound sensor attached to a wrist part on a total extensor extensor extending from the ulnar styloid process to the radius. The specific frequency extraction method according to claim 1.

(Appendix 20)
The start time of the frequency information to be analyzed is based on the S / N ratio, where S is the characteristic amount of the signal at time t, and N is the signal before the predetermined time from the time t or the average signal of the signal before the predetermined time. The specific frequency extraction method according to any one of Supplementary Note 11 to Supplementary Note 19, which is determined by:

(Appendix 21)
A frequency analysis time width having a resolution of one period or more of a specific frequency assumed to be a contact between a predetermined specific part of a human body and an object, and a time width necessary for maintaining the detection accuracy of the specific frequency The transition time width with the resolution is set independently, and the frequency distribution of the frequency information at the time of contact between the specific part and the object detected by the detection unit attached to the human body is analyzed,

Extracting the specific frequency based on the analyzed frequency distribution;
A specific frequency extraction processing program for causing a computer to execute processing including the above.

10 Information Input Device 12 Body Conduction Sensor 18 Communication Cable 20 Device Body 22 Input I / F
24 output I / F
26 Operation Information Receiving Device 28 Detection Signal Capture Unit 30 Signal Extraction Unit 32 Set Value Storage Unit 34 Frequency Analysis Unit 36 Specific Frequency Extraction Unit 38 Operation Presence Determination Unit 40 Object Type Determination Unit 42 Object Type-Feature Quantity Table Storage Unit 44 Operation Information output section 46 Wristband section 48 Sound collection section 50 Opening section 52 Sound collection element 54 Communication pipe 56 Sound collection microphone 70 CPU
72 RAM
74 ROM
76 I / O
78 bus

Claims (6)

A detection unit that can be attached to a human body and detects frequency information transmitted from the human body;
A frequency analysis time width necessary for analyzing a frequency distribution including a specific frequency component that is assumed to be a contact between a predetermined specific part of a human body and an object, and the frequency analysis time width is changed in time series. An analysis unit for analyzing the frequency distribution of the frequency information detected by the detection unit by setting transition widths necessary for maintaining the detection accuracy of the specific frequency to be independent resolutions,
Based on the frequency distribution analyzed by the analysis unit, an extraction unit that extracts the specific frequency;
In the extraction unit, an output unit that outputs an operation signal indicating that a contact operation has been performed by the specific part by extracting the specific frequency;
An information input device. Based on the feature quantity of the specific frequency extracted by the extraction unit, a specifying unit that specifies the object that the specific part is in contact with, and
The information input device according to claim 1. The specific part is
The information input device according to claim 2, further comprising a storage unit in which a correlation table between a specific frequency and a feature amount is stored for each of the plurality of different objects.   The information input device according to claim 2, wherein the feature amount includes sound pressure information. A frequency analysis time width having a resolution of one period or more of a specific frequency assumed to be a contact between a predetermined specific part of a human body and an object, and a time width necessary for maintaining the detection accuracy of the specific frequency The transition time width with the resolution is set independently, and the frequency distribution of the frequency information at the time of contact between the specific part and the object detected by the detection unit attached to the human body is analyzed,
A specific frequency extracting method including extracting the specific frequency based on the analyzed frequency distribution. A frequency analysis time width having a resolution of one period or more of a specific frequency assumed to be a contact between a predetermined specific part of a human body and an object, and a time width necessary for maintaining the detection accuracy of the specific frequency The transition time width with the resolution is set independently, and the frequency distribution of the frequency information at the time of contact between the specific part and the object detected by the detection unit attached to the human body is analyzed,
Extracting the specific frequency based on the analyzed frequency distribution;
A specific frequency extraction processing program for causing a computer to execute processing including the above.