JP6533141B2 - Vibration analysis system, user interface system and vibration analysis data generation method - Google Patents

Description

  The present invention relates to technology for applying vibration to a user interface by analyzing the waveform of the vibration.

  Patent Document 1 discloses a technique of operating a mobile phone by tapping the earphone microphone main body connected to the mobile phone or the vicinity thereof with a finger. Specifically, the vibration generated when hitting is detected by an acceleration sensor built in the earphone microphone. The detection signal is amplified and then only specific frequency components are selected.

  Then, one pulse of a predetermined width is generated for one hitting operation. A code string is generated according to the interval and the number of pulses generated by successive striking. Specifically, “0” is assigned if the pulse rise interval is less than or equal to the first time, and “1” is assigned if the pulse rise interval is longer than the first time and less than or equal to the second time. The code string generated by this is used as an operation command of the mobile phone. For example, the code string “1101” is assigned to the off-hook command, and when the code string “1101” is sent to the mobile phone, the mobile phone goes into the off-hook state.

Unexamined-Japanese-Patent No. 2003-143683 U.S. Pat. No. 7,035,744

  As described above, according to Patent Document 1, a command is generated according to the interval and the number of tapping, and the mobile phone is operated. At that time, the tapping operation is judged by the presence or absence of the vibration. For this reason, the detection of vibration is used only to determine the presence or absence of vibration.

  On the other hand, the present invention aims to provide a technique for applying vibration to a user interface by analyzing the waveform of the vibration. In particular, in the present invention, vibration generated by rubbing a first object (e.g., an input receiving member) with a second object (e.g., a user's finger) is used.

According to an aspect of the present invention, there is provided a time-frequency analysis unit that generates time-frequency distribution data representing the vibration generated by rubbing the first object with the second object in a predetermined time-frequency expression, The predetermined time frequency expression has a specification capable of revealing the difference in frequency components of the vibration in the shape of the time frequency distribution, which is determined based on the difference in combination of the first object and the second object. Performing an index determination process of determining an index spot which is a point serving as an index of a distribution shape from among a plurality of points constituting the time frequency distribution provided by the time frequency distribution data, and performing one or more index spots A vibration analysis system is provided, further comprising an index generator that generates an index profile including frequency component values of

  According to the above aspect, the vibration (abrasion vibration) generated by rubbing the first object with the second object is managed by a new data format called an index profile.

  The index profile is configured using an index spot which is a point serving as an index of the distribution shape in the time frequency distribution of rubbing vibration. Therefore, according to the index profile, it is possible to manage the rubbing vibration with a smaller amount of data than in the case of using data of the entire range of the time frequency distribution. For this reason, for example, when comparing two rubbing vibrations, it is possible to reduce the calculation time, memory capacity, and the like.

  Further, the data of the time frequency distribution which is the origin of the index spot is generated by a predetermined time frequency expression. In particular, the predetermined time-frequency representation has a specification that can reveal differences in the frequency components of the rubbing vibration in the shape of the time-frequency distribution, which is determined based on the difference in combination of the first object and the second object. doing. Therefore, it is possible to manage the difference of the rubbing vibration by the expression position of the index spot (specifically, the frequency component of the index spot), that is, by the index profile including the frequency component value of the index spot.

  The objects, features, aspects and advantages of the present invention will become more apparent from the following detailed description and the accompanying drawings.

It is a figure which shows the example of a vibration waveform. It is a figure which shows the frequency distribution of the waveform of FIG. It is a figure which shows the time frequency distribution obtained by performing STFT (Short Time Fourier Transform) with respect to the waveform of FIG. It is a figure which shows the time frequency distribution obtained by performing CWD (Choi-Williams Distribution) with respect to the waveform of FIG. FIG. 2 is a hardware block diagram for explaining an electric device according to the first embodiment. FIG. 1 is a functional block diagram for explaining a first example of a vibration analysis system according to a first embodiment. FIG. 7 is a diagram showing an example of an analysis period in the first embodiment. It is a figure which shows time frequency distribution (CWD) about analysis period AP1 in FIG. It is a figure which shows time frequency distribution (CWD) about analysis period AP2 in FIG. It is a figure which shows time frequency distribution (CWD) about analysis period AP3 in FIG. It is a figure which shows time frequency distribution (CWD) about analysis period AP4 in FIG. 7 is a flowchart illustrating a time frequency analysis unit according to Embodiment 1. FIG. 5 is a flowchart illustrating an index generation unit according to the first embodiment. FIG. 5 is a diagram for explaining a first example of an index spot in the first embodiment. FIG. 7 is a diagram for explaining a second example of the index spot in the first embodiment. FIG. 7 is a diagram for explaining a third example of the index spot in the first embodiment. It is a figure explaining the 4th example of an index spot about Embodiment 1. FIG. It is a figure explaining the 5th example of an index spot about Embodiment 1. FIG. FIG. 7 is a diagram for describing a first example of an index profile according to the first embodiment. FIG. 7 is a diagram for describing a second example of an index profile according to the first embodiment. FIG. 5 is a schematic view showing an index spot included in an index profile on a time frequency plane according to the first embodiment. 3 is a flowchart illustrating a user interface system according to the first embodiment. FIG. 7 is a schematic view showing an index spot included in an index profile on a time frequency plane in the case of Embodiment 1 (when rubbing is performed slowly). FIG. 7 is a flowchart illustrating speed determination processing according to the first embodiment. FIG. 7 is a diagram for describing a third example of an index profile according to the first embodiment. FIG. 21 is a diagram for describing a fourth example of the index profile in the second embodiment. FIG. 21 is a diagram for describing a fifth example of the index profile in the second embodiment. FIG. 16 is a flowchart illustrating an intensity determination process according to the second embodiment. FIG. 18 is a hardware block diagram for explaining a user interface system according to a third embodiment. FIG. 17 is a diagram for explaining the difference in index spots for input receiving members having different attributes in the third embodiment. FIG. 18 is a functional block diagram for explaining a second example of a vibration analysis system according to a third embodiment. It is a figure explaining a database about Embodiment 3. FIG. FIG. 21 is a diagram for explaining the relationship between the rubbing direction and the index spot for the input receiving member having different attributes according to the third embodiment. FIG. 21 is a diagram for explaining the relationship between the rubbing direction and the index spot for the input receiving member having different attributes according to the third embodiment. FIG. 21 is a diagram for explaining the relationship between the rubbing direction and the index spot for the input receiving member having different attributes according to the third embodiment. FIG. 21 is a diagram for explaining the relationship between the rubbing direction and the index spot for the input receiving member having different attributes according to the third embodiment. FIG. 21 is a functional block diagram for explaining a third example of a vibration analysis system according to a fourth embodiment. FIG. 21 is a diagram for explaining a storage medium in which an index profile is stored according to a fourth embodiment. FIG. 18 is a diagram for explaining a storage medium in which a database of index profiles is stored according to a fourth embodiment. FIG. 21 is a functional block diagram for explaining a fourth example of a vibration analysis system according to a fourth embodiment.

<Summary>
Before describing specific embodiments of the present invention, an outline of the present invention will be described.

  When an object is rubbed with a finger (ie, moved while the finger is in contact with an object), vibrations occur. This rubbing vibration differs depending on the attribute (material, unevenness of the surface, etc.) of the object to be rubbed. This can be understood, for example, from the experience that different sounds (more specifically, sounds with different frequencies) can be heard as the finger is moved on a sandpaper with different eye roughness.

  In view of this point, the inventor of the present application has conceived of determining the difference in the rubbing vibration and using it for the user interface.

  FIG. 1 shows an example of a waveform chart of vibration generated when an object is rubbed with a finger in a straight line and in one direction. FIG. 1 is a waveform diagram in the time domain, showing the time change of the vibration intensity. That is, in FIG. 1, the horizontal axis indicates time, and the vertical axis indicates the intensity of vibration.

Note that the numerical values on the vertical axis in FIG. 1 are derived from the output signal level of the microphone that captures the vibration, and are arbitrarily normalized according to the general notation for the intensity. Further, the numerical values on the horizontal axis are derived from the sampling when the output signal of the microphone is analog-to-digital converted (AD conversion), and are normalized by the sampling period. Here, since the sampling frequency is 16 kHz, the sampling period is 1/16 kHz = 62.5 μs. Therefore, the value on the horizontal axis can be converted to real time by multiplying by 62.5 μs. For example, 1.0 × 10 ⁴ × 62.5 μs = 0.625 s.

  FIG. 2 shows a waveform chart in which frequency is analyzed by performing FFT (Fast Fourier Transform) on the entire waveform of FIG. FIG. 2 is a waveform diagram in the frequency domain, showing the frequency distribution. That is, in FIG. 2, the horizontal axis indicates frequency, and the vertical axis indicates distribution intensity (for example, spectral density). The numerical value on the horizontal axis indicates the actual frequency, and the numerical value on the vertical axis is arbitrarily normalized.

  Unlike FIG. 1 or FIG. 2, the rubbing vibration has no clear feature (formant etc.) unlike voice and is close to noise such as white noise and pink noise. In other words, it is difficult to extract the information that characterizes the rubbing vibration.

  Therefore, the inventor of the present application further considered using analysis using time-frequency representation, that is, using time-frequency analysis. The time frequency expression is also called "TFR", and the time frequency analysis is also called "TFA".

  First, FIG. 3 shows a time frequency distribution obtained by performing STFT (Short Time Fourier Transform), which is a typical TFA, on the waveform of FIG. In FIG. 3, two horizontal axes crossing each indicate time and frequency, and a vertical axis indicates distribution intensity (for example, spectral density). The numerical values on the time axis are normalized by the sampling period as in FIG. 1, and the numerical values on the frequency axis are normalized by the sampling frequency. The values on the vertical axis are arbitrarily normalized.

  According to FIG. 3, it appears that there is a large spectral density in the low frequency region. However, since the resolution is small, the characteristic component can hardly be found. In other words, even with STFT, only characteristics close to normal white noise can be seen. The low resolution of the STFT is because there is a trade-off between time and frequency, in other words, an uncertainty principle.

  Various methods have been proposed to improve the resolution, and WVD (Wigner-Ville Distribution) is known as one of the methods. WVD also has the advantage of being able to reduce the operation cost as well as STFT. However, WVD tends to generate an interference component due to the content of the operation. The interference component may be called an interference wave, a cross term, or an artifact.

  From the results of the STFT and WVD studies, we have found that time-frequency representations with higher resolution than STFTs and smaller interference components than WVD are useful, and CWD (also known as such time-frequency representations). We came to the finding that Choi-Williams Distribution) could be used.

  FIG. 4 shows a waveform diagram in which the entire waveform of FIG. 1 is analyzed by CWD. Similar to the STFT of FIG. 3, two horizontal axes and a vertical axis in the time frequency distribution of FIG. 4 respectively indicate time, frequency and distribution intensity. According to FIG. 4, it can be seen that a large spectral density is concentrated in a certain area, and that their spectral density changes along the time axis. This is considered to be the occurrence of temporal change (Doppler shift) in the basic vibration frequency of the object to be rubbed.

  As described above, the frequency of the rubbing vibration differs depending on the attribute of the object to be rubbed. For this reason, the frequency component (more practically, a frequency range with a certain width) where a large spectral density appears in the time frequency distribution by CWD depends on the attribute of the object to be rubbed. In particular, according to CWD based on the above-mentioned findings, the difference in the expression position can be revealed as compared with STFT and WVD. Therefore, by comparing the distribution shape of the time frequency distribution by CWD, it is possible to accurately determine whether the attribute of the object is the same, in other words, whether or not the object having the same attribute is rubbed.

  Here, the simplest way is to compare the entire range of the time frequency distribution.

  However, in the present invention, a new concept of index spot is introduced and used. The index spot is a point serving as an index of the distribution shape in the time frequency distribution (here, the distribution obtained by CWD). In other words, the index spot is a representative point that characterizes the distribution shape. For example, peak points can be used as index spots. The index spots will be described later. Comparing the distribution of index spots can reduce the amount of data to be compared compared to comparing the entire range of time frequency distribution. As a result, calculation time, memory capacity and the like can be reduced.

  Hereinafter, the present invention will be more specifically described through Embodiments 1 to 4 and their modifications.

Embodiment 1
<Configuration outline>
FIG. 5 shows a hardware block diagram for explaining the electric device 10 according to the first embodiment. The type, application, and the like of the electric device 10 are not particularly limited. According to the example of FIG. 5, the electric device 10 includes the main processing unit 11, the storage unit 12, the information input unit 13, the information output unit 14, the external connection unit 15, and the user interface system 20. There is. Although these elements 11-15, 20 are connected to the bus 16 in FIG. 5, other connection forms may be adopted.

  The main processing unit 11 performs general processing relating to the electric device 10. The main processing unit 11 includes, for example, one or more microprocessors, and executes a program stored in the storage unit 12. As a result, the main processing unit 11 operates as various functional blocks. That is, various functions are realized by software. However, some or all of the functions may be realized by a dedicated circuit, in other words, hardware.

  The storage unit 12 is a storage medium for storing various types of information (program, data, etc.). The storage unit 12 is formed of, for example, a semiconductor memory (hereinafter also referred to as a memory), and may include a hard disk drive. As a memory, a random access memory (RAM), a read only memory (ROM), a rewritable non-volatile memory, or the like can be used. The RAM not only stores various types of information, but also provides a working area for the main processing unit 11 to execute a program.

  The information input unit 13 receives an input of information from the user to the electric device 10. The information input unit 13 includes, for example, various buttons, a touch panel, a keyboard, a mouse, a microphone, or a combination thereof. The information output unit 14 outputs information from the electrical device 10 to the user. The information input unit 13 is configured by, for example, a display device that visually outputs information, an audio device that acoustically outputs information, or a combination thereof.

  The external connection unit 15 connects the electric device 10 and an external device by wired communication or wireless communication. The electric device 10 and the external device may be directly connected, or may be connected via the Internet or the like. When an external storage medium (semiconductor memory, hard disk drive, optical disk, etc.) is connected as an external device, the external storage medium is used as part or all of the storage unit 12 and a storage unit 52 described later. Is also possible.

  The user interface system 20 includes an input receiving member 30, a vibration sensor 40, and a vibration analysis system 50.

  The input receiving member 30 is an object for the user to rub with a finger. The attributes of the input receiving member 30 are designed to generate unique vibrations. For example, the concavo-convex shape of the surface is designed to generate a characteristic vibration. For example, a part or all of the housing of the electric device 10 can be used as the input receiving member 30. Alternatively, a member attached to the housing of the electric device 10 or a member provided apart from the housing of the electric device 10 may be used as the input reception member 30.

  The vibration sensor 40 is arranged to be able to capture the vibration generated by rubbing the input receiving member 30. More specifically, the vibration sensor 40 includes a vibration capture device (microphone, acceleration sensor, etc.), and the vibration capture device is arranged to be able to capture vibration. Vibration capture devices convert physical vibrations into electrical signals. A plurality of vibration capturing devices may be provided.

  The vibration sensor 40 converts an analog electrical signal output from the vibration capturing device into a digital electrical signal that can be processed by the vibration analysis system 50. Then, the vibration sensor 40 outputs the converted digital signal, in other words, digital data of the vibration waveform. Therefore, the vibration sensor 40 also includes an amplifier, an analog-to-digital converter (AD converter), a filter, and the like.

  Unlike the example of FIG. 5, the vibration sensor 40 may be connected to the vibration analysis system 50 via the bus 16.

  The vibration analysis system 50 analyzes the output signal of the vibration sensor 40. In the example of FIG. 5, the vibration analysis system 50 includes a processing unit 51 and a storage unit 52. The processing unit 51 and the storage unit 52 may be referred to as an analysis processing unit 51 and an analysis storage unit 52.

  The processing unit 51 performs the later-described process related to vibration analysis. The processing unit 51 is configured of a microprocessor specialized for vibration analysis. That is, various functions provided by the processing unit 51 are realized by hardware. However, part or all of the functions of the processing unit 51 may be realized by causing a general-purpose microprocessor to execute a program, in other words, software.

  The storage unit 52 is a storage medium used by the processing unit 51 for vibration analysis. The storage unit 52 can be configured in the same manner as the storage unit 12. Although the storage unit 52 is illustrated as hardware separate from the storage unit 12 in FIG. 5, a partial storage area of the storage unit 12 may be used as the storage unit 52.

  The configuration of FIG. 5 can be changed as appropriate. For example, one or more of the information input unit 13, the information output unit 14, and the external connection unit 15 may be omitted if not required. Conversely, elements not shown in FIG. 5 may be added.

Vibration Analysis System 50
A functional block diagram of the vibration analysis system 50 is shown in FIG. According to the example of FIG. 6, the vibration analysis system 50 includes a time frequency analysis unit 110, an index generation unit 120, and a determination unit 130.

<Time Frequency Analysis Unit 110>
The output signal of the vibration sensor 40 (see FIG. 5) is input to the time frequency analysis unit 110, whereby the time frequency analysis unit 110 acquires waveform data D1 of the captured vibration. Then, the time frequency analysis unit 110 generates time frequency distribution data (which may be called time frequency analysis data) D2 expressing the captured vibration by CWD from the vibration waveform data D1. In the example of FIG. 6, the time frequency analysis unit 110 includes a pre-processing unit 111 and a CWD processing unit 112.

  The preprocessing unit 111 performs CWD preprocessing. That is, since CWD handles an analysis signal (Analytic Signal) which is a kind of complex signal, the pre-processing unit 111 converts the output signal of the vibration sensor 40 which is a real signal (Real part Signal) into an analysis signal. For such signal conversion, for example, a known Hilbert transform represented by the following equation (1) is used. However, signal conversion by another algorithm may be used.

The CWD processing unit 112 obtains a time frequency distribution by CWD for the analysis signal output from the preprocessing unit 111. CWD is represented, for example, by the well-known following Formula (2). As the analysis signal z (s) of equation (2), x _a (n) of equation (1) is applied. In equation (2), t represents time and ν represents frequency.

Here, following the general method, the square of the magnitude of CW _Z (t,)) obtained by equation (2), so-called spectral density, is taken as the distribution intensity of each point of the time frequency distribution. However, various values correlated with CW _Z (t,)) (for example, values obtained by normalizing the spectral density by a predetermined method) may be used as the distribution intensity.

  In addition, the various parameters in Formula (1), (2) shall be adjusted by prior simulation etc. so that a reference | standard value may be referred to and it is suitable for manifestation of an index spot, for example.

  The time frequency analysis unit 110 discretely sets an analysis period for one rubbing operation, and generates time frequency distribution data D2 only for the analysis period. FIG. 7 shows an example in which an analysis period AP (more specifically, analysis periods AP1 to AP4) is set to the vibration waveform data D1 of FIG. Further, FIG. 8 shows a time frequency distribution provided by the time frequency distribution data D2 obtained for the analysis period AP1 in FIG. Similarly, time frequency distributions for analysis periods AP2 to AP4 in FIG. 7 are respectively shown in FIGS. In the time frequency distribution of FIGS. 8 to 11, the horizontal axis and the vertical axis indicate time and frequency, respectively, and the numerical values of the horizontal axis and the vertical axis are normalized with the sampling period and the sampling frequency, respectively. Further, in FIGS. 8 to 11, the distribution intensity is indicated by shading unlike in FIG. That is, the higher the distribution intensity, the darker the color is.

  FIG. 12 shows a flowchart of an operation example of the time frequency analysis unit 110. According to the time-frequency analysis process S210 of FIG. 12, the time-frequency analysis unit 110 waits for the start of the rubbing operation (i.e., the input operation), and when the start of the rubbing operation is detected The system 50 shifts from the sleep mode to the analysis mode (step S211). For example, when the time frequency analysis unit 110 detects that the intensity of the output signal of the vibration sensor 40 exceeds a predetermined analysis mode start threshold, the time frequency analysis unit 110 determines that the rubbing operation is started.

  In the analysis mode, the time frequency analysis unit 110 starts the analysis period AP (step S212S), and for the vibration waveform data D1 sent from the vibration sensor 40 until the analysis period AP times out (step S212E). Pre-processing (step S213) and CWD processing (step S214) are performed. The vibration waveform data D1 may be accumulated during the analysis period AP, and pre-processing and CWD processing may be performed on the accumulated data after the analysis period AP ends.

  After that, after a predetermined waiting time, the time frequency analysis unit 110 returns to the step S212S and starts the next analysis period AP. However, if the rubbing operation is completed during the waiting time (step S215), the time frequency analysis unit 110 shifts from the analysis mode to the pause mode without starting the next analysis period AP. That is, the time-frequency analysis process S210 ends. For example, when the time frequency analysis unit 110 detects that the state in which the intensity of the output signal of the vibration sensor 40 falls below a predetermined analysis mode end threshold continues for a predetermined time, the time frequency analysis unit 110 determines that the rubbing operation is completed. Determine.

  The length of the analysis period AP and the interval of the analysis period AP are assumed to be set in advance. Here, it is assumed that the lengths of the respective analysis periods AP are the same, and the intervals of the analysis periods AP are the same. However, it is not limited to this example. Also, the number of analysis periods AP for one rubbing operation is not limited to four illustrated in FIG. For example, depending on the set value of the length of the analysis period AP, the rubbing speed, etc., only one analysis period AP may be set for the entire one rubbing operation.

  The analysis period AP may be always set to one analysis period AP for the entire rubbing operation without setting the analysis period AP discretely. However, particularly when the rubbing operation continues for a long time, the discrete analysis period AP can reduce the amount of data to be calculated. As a result, calculation time, memory capacity and the like can be reduced.

<Index generation unit 120>
FIG. 13 shows a flowchart of an operation example of the index generation unit 120. According to the index generation process S220 of FIG. 13, the index generation unit 120 performs an index determination process S221 and an index profile generation process S222.

<Indicator determination processing S221>
The index generation unit 120 acquires the time frequency distribution data D2 generated by the time frequency analysis unit 110. Then, the index generation unit 120 determines an index spot based on the time frequency distribution data D2 by the index determination processing S221. As described above, the index spot is a point serving as an index of the distribution shape in the time frequency distribution provided by the time frequency distribution data D2.

  14 to 17 show conceptual views of index spots. In FIG. 14 to FIG. 17, the distribution intensity is indicated by a bar graph, and the index spot SP is indicated by a black circle. In the lower part of FIG. 14 to FIG. 17, a diagram in which the index spot SP is projected on a time frequency plane, in other words, a diagram in which the index spot SP is represented by a time component and a frequency component is shown.

  According to the first example shown in FIG. 14, the index spot SP is a peak point in the time frequency distribution. For example, the index generation unit 120 sequentially sets the points of interest in the time frequency distribution by sequentially changing the combination of the values of the time t and the frequency ν. When the index generation unit 120 detects that the distribution intensity of the attention point is larger than the distribution intensity of the predetermined range surrounding the attention point, the index generation unit 120 determines the attention point as a peak point, that is, the index spot SP. .

  According to the second example shown in FIG. 15, the index spot SP is a point at which the distribution intensity exceeds a predetermined threshold in the time frequency distribution. For example, the index generation unit 120 sequentially sets the points of interest in the same manner as described above, and when it is detected that the distribution intensity of the points of interest exceeds a predetermined threshold, the point of interest is determined as the index spot SP.

  The third example shown in FIG. 16 is a combination of the first and second examples. That is, in the third example, the index spot SP is a peak point having a distribution intensity exceeding a predetermined threshold in the time frequency distribution. In FIG. 16, the index spot SP is determined by first extracting a peak point and comparing the extracted distribution point of the peak point with a predetermined threshold. Conversely, points where the distribution intensity exceeds a predetermined threshold may be extracted first, and peak points may be extracted for the extracted points. Note that the index spots SP to be extracted may differ depending on the processing order.

  Here, referring to FIGS. 8 to 11, the time frequency distribution data D2 can be visualized as an image. In view of this, the inventor of the present application has conceived of determining the index spot SP by treating the time frequency distribution data D2 as image data and applying an image processing technique. For example, edge extraction processing (also called edge enhancement processing) can be used. Note that edges may be understood as contours or boundaries.

  As an example of the edge extraction process, a process of extracting a portion where a pixel value greatly changes as an edge is known. The 4th example which applied this is shown in FIG. In the fourth example, the distribution point of the point of interest is determined as the index spot SP when the amount of change exceeding the predetermined threshold value is compared with the distribution intensity of points in a predetermined range surrounding the point of interest. Note that the rate of change may be used instead of the amount of change.

  The edge extraction process can be performed, for example, by the index generation unit 120 applying an edge extraction filter to the time frequency distribution data D2. The processing content of the edge extraction filter is not limited to the above fourth example. For example, among various edge extraction filters known in the image processing field, a filter having a desired specification can be selected.

  Here, the index determination process of the first to fourth examples can be generalized as a process of extracting a point satisfying a predetermined extraction condition as the index spot SP (referred to as a first index determination process). . On the other hand, the first to fourth examples can be classified as follows according to the specific contents of the above-mentioned predetermined extraction conditions.

  That is, the extraction condition of the first example is a condition of being a peak point (referred to as a first condition). The extraction condition of the second example is a condition that the distribution intensity exceeds a predetermined threshold (referred to as a second condition). The extraction conditions of the third example require that both the first condition and the second condition be satisfied. The extraction conditions of the first to third examples are summarized as conditions including at least one of the first condition and the second condition (referred to as a first extraction condition).

  The extraction condition of the fourth example is a condition (referred to as a second extraction condition) that it is a point extracted by performing edge extraction processing on the time frequency distribution data D2.

  By the way, you may give up several index spots SP. Specifically, the index determination process is a process of extracting points satisfying the predetermined extraction condition as candidate points for the index spot SP, and determining points representative of a plurality of candidate points as the index spot SP (second It may be called an index determination process). For example, any of the extraction conditions of the first to fourth examples can be used as the extraction conditions of the candidate points.

  This is shown as a fifth example in FIG. In addition, in FIG. 18, in order to avoid the complication of drawing, only the time frequency plane similar to the lower stage of FIGS. 14-17 is shown. According to FIG. 18, it is assumed that eleven candidate points SQ (indicated by white circles) are extracted, and the average point of these candidate points SQ is determined as the representative point, that is, the index spot SP. More specifically, the index generation unit 120 calculates the average value of the frequency components of the 11 candidate points SQ, calculates the average value of the time components and the distribution intensity in the same manner, and specifies the combination by the combination of these average values. To be the index spot SP.

  The representative points are not limited to the average points. For example, the representative point may be a median point (median point). The median point is a point specified by a combination of medians of components of the plurality of candidate points SQ.

  In addition, conditions for which candidate points SQ should be given may be defined in advance. For example, one representative point, that is, one index spot SP is determined for each section of a predetermined time width and frequency width.

  The specific content of the index determination process is not limited to the above example.

<Indicator profile generation processing S222>
In the index profile generation process S222 (see FIG. 13), the index generation unit 120 generates index profile data D3 as vibration analysis data based on the index spot SP determined in the index determination process S221. In addition, indicator profile data may be called an indicator profile.

  FIG. 19 shows a first example of the configuration of the index profile D3. The index profile D3 includes data of one or more index spots SP. According to the example of FIG. 19, the index profile D3 includes the frequency component value and the time component value of each index spot SP. The frequency component value may be given by the actual frequency value of the index spot SP, or may be given by a value obtained by converting the actual frequency value by predetermined normalization. The same applies to time component values. In the example of FIG. 19, a number for identifying the index spot SP is recorded, but this information may be omitted.

  FIG. 20 shows a second example of the configuration of the index profile D3. As described above, the time frequency distribution data D2 can be grasped as image data. In view of this, the index profile D3 can be configured in the same manner as the image data. Specifically, in the example of FIG. 20, the index profile D3 is configured by so-called bit map data in which the index spot SP is represented by “1” and the non-index spot SP is represented by “0”. In the bit map of FIG. 20, the left lowermost bit position is assumed to be closest to the origin of the time frequency plane, the horizontal direction corresponds to the time axis, and the vertical direction is the frequency axis. It shall correspond.

  In the example of FIG. 20, the index profile D3 also includes attribute data of bitmap data. Specifically, the index profile D3 includes frequency component related data and time component related data.

  The frequency component related data includes, for example, frequency range data indicating what frequency range the bit map data relates to, and frequency interval data indicating what frequency interval each bit of the bit map data is specified to. including.

  Similarly, the time component related data indicates, for example, time range data indicating what time range bitmap data relates to, and a time interval indicating what time interval each bit of the bitmap data is defined And interval data.

  The frequency component value and the time component value of the index spot SP can be calculated from the frequency component related data, the time component related data, and the position of the index spot SP in the bitmap data. That is, even in the bitmap data format, the index profile D3 can include the frequency component value and the time component value of the index spot SP.

  The bitmap data may be generated for the entire range (see FIGS. 8 to 11) of the time frequency distribution data D2 generated by the time frequency analysis unit 110, or includes the expression position of the index spot SP. It may be generated for a limited range. According to the latter, the data size can be reduced.

  Incidentally, in view of the contents of the index profile D3 in FIG. 19 and FIG. 20, in the second index determination processing (a point representative of a plurality of candidate points is determined as the index spot SP. See FIG. 18). The calculation of the intensity distribution value may be omitted.

  The index profile D3 includes data of index spots SP obtained in one rubbing operation (that is, in the period from the start to the end of the rubbing operation) in one index profile D3. In this case, when a plurality of analysis periods AP are set for one rubbing operation (see FIG. 7), index spots obtained for the plurality of analysis periods AP are recorded in one index profile D3. In addition, the time component value of an index spot shall be expressed on the basis of a uniform origin (for example, the start time of the 1st analysis period AP) with respect to all the analysis periods AP. On the other hand, when only one analysis period AP is set for one rubbing operation, the index profile D3 is configured by the index spots SP of the one analysis period AP.

  On the other hand, the index profile D3 may be generated for each analysis period AP. However, for example, when the analysis period AP is short, the number of index spots SP recorded in one index profile D3 decreases. It is more preferable that a certain number of index spots SP be included in the index profile D3 in order to ensure the accuracy in the determination processing described later. In view of this point, it is more preferable that the index profile D3 include the index spot SP in the whole of one rubbing operation as described above.

<Discrimination unit 130>
Returning to FIG. 6, the determination unit 130 acquires the index profile D3 generated by the index generation unit 120 as a determination target. Then, the determination unit 130 determines whether or not the index profile D3 to be discriminated (also referred to as the discrimination target profile D3) has the same identity as the registered index profile D3 (also referred to as the reference profile D3). Do the processing. The reference profile D3 is generated in advance for the vibration generated when the input receiving member 30 is rubbed, and is stored in advance in the storage unit 52 (see FIG. 5) of the vibration analysis system 50.

  Here, FIG. 21 is a schematic view showing the index spot SP included in the index profile D3 on a time frequency plane. As can be seen from FIG. 21, the index spots SP included in the index profile D3 have similar frequency component values. This can also be understood from FIGS. In particular, the frequency at which the index spot SP is concentrated (more practically, a frequency range having a certain width) depends on the attribute of the input receiving member 30. This is because, as described above, the rubbing vibration that is the origin of the index spot SP differs depending on the attribute of the input receiving member 30.

  In view of this point, the determination unit 130 determines whether the frequency component value included in the determination target profile D3 is identical to the frequency component value included in the reference profile D3. That is, the identity of the frequency component values corresponds to the identity of the determination target profile D3 with the reference profile D3.

  Identification of identity is preferably more practical. That is, when the distribution of frequency component values included in the discrimination target profile D3 is similar to the distribution of frequency component values included in the reference profile D3 under the range of a predetermined tolerance, the two index profiles D3 are It is more practical to determine identity. Such discrimination can be realized, for example, by known similarity discrimination techniques or data matching techniques.

  If it is determined that the determination target profile D3 has the same identity as the reference profile D3, the determination unit 130 outputs the determination result information D4. The determination result information D4 can be embodied by, for example, predetermined data or a predetermined signal.

  When it is determined that the determination target profile D3 does not have the same identity as the reference profile D3, the determination result information D4 indicating that may be output. For example, the data content or signal level of the determination result information D4 may be made different depending on whether the determination target profile D3 has the same identity as the reference profile D3.

  As described above, the reference profile D3 relates to the vibration generated when the input receiving member 30 is rubbed. Therefore, the identification target profile D3 having the same identity as the reference profile D3 indicates that the input receiving member 30 has been rubbed. Therefore, the determination unit 130 may be understood to determine whether or not the input receiving member 30 has been rubbed.

<User Interface System 20>
A flowchart of an example of the overall operation of the user interface system 20 is shown in FIG. According to the input determination process S200 of FIG. 22, the time-frequency analysis unit 110 executes the above-described time-frequency analysis process S210 (see FIG. 12). Thus, from the vibration waveform data D1 sent from the vibration sensor 40, time frequency distribution data D2 is generated.

  Next, the index generation unit 120 executes the above-described index generation process S220 (see FIG. 13). Thereby, the index spot SP is determined based on the time frequency distribution data D2 (step S221), and the index profile D3 is generated from the obtained index spot SP (step S222).

  Then, in the index determination process S230, as described above, the index profile D3 generated by the index generation unit 120 determines that the index profile D3 generated by the index generation unit 120 is a registered index profile D3. To determine if it has the same identity as Thus, it is determined whether or not the input receiving member 30 has been rubbed.

  The determination result information D4 is sent to the main processing unit 11 (see FIG. 5) and used for processing by the main processing unit 11. When the determination result information D4 indicates that the input receiving member 30 has been rubbed, for example, the main processing unit 11 reduces the volume of the electric device 10. According to this example, the determination result information D4 functions as a volume adjustment command. The use of the determination result information D4 is not limited to the volume adjustment.

<Speed determination>
Here, FIG. 23 shows an index profile D3 in the case where the input receiving member 30 is slowly rubbed as compared with the example of FIG. As understood from comparison of FIG. 23 with FIG. 21, the slower the rubbing speed, the smaller the number of index spots SP included in the same time width. In view of this point, the determination unit 130 determines the relative speed of the rubbing operation according to the number of index spots SP included in a predetermined time.

  FIG. 24 shows a flowchart of an example of the speed determination process. According to the speed determination process S250 of FIG. 24, the determination unit 130 acquires the number of index spots SP per predetermined time for the determination target profile D3 (obtained from the index generation unit 120) (step S251). The length of the predetermined time is assumed to be set in advance. Further, the predetermined time can be calculated, for example, from the index spot SP having the earliest time component value. However, the setting position of the predetermined time is not limited to this example.

  Similarly, the determination unit 130 acquires, for the reference profile D3, the number of index spots SP having the same length per predetermined time (step S252). Steps S251 and S252 may be performed in the reverse order.

  Since the reference profile D3 is known before the start of the speed determination process S250, the number of index spots SP per predetermined time can be counted in advance for the reference profile D3. Also, the value counted in advance can be recorded in the reference profile D3. Therefore, step S252 may be executed by reading the number counted in advance from the reference profile D3.

  Next, the determination unit 130 compares the number of index spots SP acquired from each of the determination target profile D3 and the reference profile D3 (step S253).

  As a result of comparison, when there are more index spots SP acquired from the discrimination target profile D3, the discrimination unit 130 generates velocity discrimination result information indicating that the input velocity is faster than the reference velocity (step SS254). The input speed is the speed of the rubbing operation that is the source of the determination target profile D3. Also, the reference speed is the speed of the reference motion, and the reference motion is the rubbing motion that is the origin of the reference profile D3.

  On the other hand, when the index spot SP acquired from the determination target profile D3 is smaller, the determination unit 130 generates speed determination result information indicating that the input speed is slower than the reference speed (step SS255).

  When the index spots SP have the same number, the determination unit 130 generates speed determination result information indicating that the input speed is the same as the reference speed (step SS 256).

  Here, the comparison of the number of index spots SP in step S253 can be performed by strictly comparing the values acquired in steps S251 and S252. Alternatively, a predetermined tolerance range may be introduced. For example, when the difference in the number of indicator spots SP is within a predetermined value, it may be determined that the number of indicator spots SP is the same.

  The speed determination process S250 is executed in the index determination process S230 (see FIG. 22), but is preferably performed after it is determined that the determination target profile D3 has the same identity as the reference profile D3. It is because it is efficient. The speed determination result information is included in the determination result information D4 when it is determined that the determination target profile D3 has the same identity as the reference profile D3.

  The speed determination result information is, for example, reflected in the adjustment amount of the volume of the electric device 10. Specifically, when the speed determination result information indicates that the input speed is the same as the reference speed, the main processing unit 11 (see FIG. 5) lowers the volume with the reference adjustment amount. On the other hand, when the speed determination result information indicates that the input speed is faster than the reference speed, the main processing unit 11 reduces the volume with an adjustment amount larger than the reference adjustment amount. Conversely, when the speed determination result information indicates that the input speed is slower than the reference speed, the main processing unit 11 reduces the volume with an adjustment amount smaller than the reference adjustment amount.

  When the speed determination is not used, as shown in FIG. 25, the time component value may be omitted from the index profile D3.

<Effect>
According to the vibration analysis system 50, the rubbing vibration is managed in a new data format of the index profile D3.

  The index profile D3 is configured using an index spot SP which is a point serving as an index of the distribution shape in the time frequency distribution of rubbing vibration. Therefore, according to the index profile D3, it is possible to manage the rubbing vibration with a smaller amount of data than in the case of using the data of the whole range of the time frequency distribution. Therefore, calculation time, memory capacity, and the like can be reduced in the process of comparing two rubbing vibrations (see the index determination process S230).

  Further, the time frequency distribution data D2 used to determine the index spot SP is generated by the CWD. In particular, CWD has a specification that can reveal differences in frequency components of rubbing vibration in the shape of the time frequency distribution. Therefore, the difference in the rubbing vibration can be managed by the expression position of the index spot SP (specifically, the frequency component of the index spot SP), in other words, by the index profile D3 including the frequency component value of the index spot SP.

  Thus, since the rubbing vibration can be appropriately managed by the index profile D3, the user interface system 20 utilizing the rubbing vibration can be provided.

  Further, as described above, when the relative speed of the rubbing operation is also determined, more operations can be allocated to one rubbing operation as compared with the case where only the presence or absence of the rubbing operation is determined.

  Here, in the technology of Patent Document 1, the number and interval of tapping operations are used to operate the mobile phone. Since the tapping interval is determined by the tapping speed, it may appear to be common with the user interface system 20 in utilizing the speed of operation. However, in the technique of Patent Document 1, it is necessary to control the generation of "0" and "1" by changing the length of the tapping interval, that is, the tapping speed. Because of this, the user may feel bothersome because he has to hit while thinking about the difference in speed. This is all the more so if you hit it a lot.

  On the other hand, according to the user interface system 20, the input operation may be performed only once. Furthermore, it is not necessary to consciously change the rubbing speed during one rubbing operation. For this reason, compared with the technique of patent document 1, high convenience can be provided.

  By the way, in the above, CWD is used as a time-frequency expression whose resolution is higher than that of STFT and whose interference component is smaller than WVD. Other than CWD, there is a time-frequency representation with higher resolution than STFT and smaller interference components than WVD. For example, BJ (Born-Jordan), ZAM (Zhao-Atlas-Marks), BD (B-Distribution), MBD (Modified B-Distribution), and KCS (Kernels Compact Support) are also available. These time frequency representations are known from technical papers and the like. For example, KCS, which is expected to have better characteristics than CWD, is introduced in Patent Document 2.

Second Embodiment
FIG. 26 shows a fourth example of the configuration of the index profile D3. As can be seen from the comparison with FIG. 19, the index profile D3 of FIG. 26 includes the frequency component value, the time component value, and the distribution intensity value of each index spot SP. The distribution intensity value may be given as an analysis result value by CWD, or may be given as a value obtained by converting the analysis result value by a predetermined normalization.

  FIG. 27 shows a fifth example of the configuration of the index profile D3. As can be seen from the comparison with FIG. 20, in the bit map data of FIG. 27, the non-index spot SP is represented by “0” and the index spot SP is represented by a value other than “0”. There is. In particular, the numerical value of the index spot SP indicates a distribution intensity value, and eight levels of distribution intensity values are illustrated in FIG.

  When the index profile D3 also includes the distribution intensity value, the relative intensity of the rubbing operation can be determined by comparing the distribution intensity value included in the determination target profile D3 with the distribution intensity value included in the reference profile D3.

  FIG. 28 shows a flowchart of an example of the strength determination process. According to the strength determination process S270 of FIG. 28, the determination unit 130 compares the distribution strength value included in the determination target profile D3 with the distribution strength value included in the reference profile D3 (step S271). When the index profile D3 includes a plurality of distribution intensity values (i.e., a plurality of index spots SP), for example, the maximum values of the plurality of distribution intensity values are compared with each other. Instead of the maximum value, various values such as an average value, a median (median), etc. may be used.

  Note that, since the reference profile D3 is known before the start of the strength determination processing S270, for the reference profile D3, the distribution intensity value (maximum value etc.) used for comparison may be obtained in advance and recorded in the reference profile D3. it can.

  As a result of the comparison in step S271, when the distribution intensity value included in the determination target profile D3 is larger, the determination unit 130 generates intensity determination result information indicating that the input intensity is larger than the reference intensity (step SS272). . The input strength is the strength of the rubbing operation that is the source of the determination target profile D3. The reference strength is the strength of the reference movement, and the reference movement is the rubbing movement that is the origin of the reference profile D3.

  On the other hand, when the distribution strength value included in the determination target profile D3 is smaller, the determination unit 130 generates strength determination result information indicating that the input strength is smaller than the reference strength (step SS273).

  Further, when the distribution intensity value included in the determination target profile D3 is the same as the distribution intensity value included in the reference profile D3, the determination unit 130 determines intensity determination result information indicating that the input intensity is the same as the reference intensity. It generates (step SS 274).

  Here, in step S271, the distribution intensity values can be strictly compared. Alternatively, a predetermined tolerance range may be introduced. For example, when the difference between the distribution intensity values is within a predetermined value, it may be determined that the distribution intensity values are the same.

  The strength determination process S270 is performed in the index determination process S230 (see FIG. 22), but is preferably performed after it is determined that the determination target profile D3 has the same identity as the reference profile D3. It is because it is efficient. The strength determination result information is included in the determination result information D4 when it is determined that the determination target profile D3 has the same identity as the reference profile D3.

  The strength determination result information is, for example, reflected in the adjustment amount of the volume of the electric device 10. Specifically, when the strength determination result information indicates that the input strength is the same as the reference strength, the main processing unit 11 (see FIG. 5) lowers the volume with the reference adjustment amount. On the other hand, when the strength determination result information indicates that the input strength is larger than the reference strength, the main processing unit 11 lowers the volume with an adjustment amount larger than the reference adjustment amount. Conversely, when the strength determination result information indicates that the input strength is smaller than the reference strength, the main processing unit 11 reduces the volume with an adjustment amount smaller than the reference adjustment amount.

  As described above, by determining the strength of the rubbing operation, more operations can be allocated to one rubbing operation as compared with the case where only the presence or absence of the rubbing operation is determined. Further, as described in the first embodiment, higher convenience can be provided as compared with the technique of Patent Document 1.

  Note that both the strength determination process and the speed determination process may be used. In that case, the rubbing operation can be classified into four types according to the combination of strength and speed. As a result, up to four types of operation of the electric device 10 can be used properly in one rubbing operation.

  When the speed determination is not used, the time component value may be omitted from the index profile D3 of FIG. 26 (see FIG. 25).

Embodiment 3
FIG. 29 shows a hardware block diagram of the user interface system 20B according to the third embodiment. The user interface system 20B is applicable to the electric device 10 in place of the user interface system 20 (see FIG. 5).

  Referring to FIG. 29, the user interface system 20B includes two types of input receiving members 30. The two types of input receiving members 30 may be distinguished by reference numerals 30B and 30C. The case where three or more types of input receiving members 30 are provided can be sufficiently understood from the following description.

  The attributes of the two types of input receiving members 30B and 30C are adjusted such that different vibrations occur. For example, by providing portions of the casing of the electric device 10 with different uneven shapes on the surface, those portions can be used as the input receiving members 30B and 30C. Due to the difference in the attributes, index spots SP having different frequency components appear as illustrated in FIG.

  In the user interface system 20B, the vibration sensor 40 is disposed so as to capture both the vibration generated by rubbing the input receiving member 30B and the vibration generated by rubbing the input receiving member 30C.

  The user interface system 20B includes a vibration analysis system 50B in place of the vibration analysis system 50 described above. The vibration analysis systems 50 and 50B are similar in hardware configuration but differ in function. FIG. 31 shows a functional block diagram of the vibration analysis system 50B. According to the example of FIG. 31, the vibration analysis system 50B has a configuration in which the database 140 is added to the vibration analysis system 50 of FIG. The database 140 is assumed to be stored in advance in the storage unit 52 of the vibration analysis system 50B.

  The structural example of the database 140 is shown in FIG. As shown in FIG. 32, the database 140 includes the reference profile D3 for the input receiving member 30B (ie, the index profile D3 for the vibration generated when rubbing the input receiving member 30B) and the reference for the input receiving member 30C. And the profile D3. Identification information is given to each of the two reference profiles D3. That is, two reference profiles D3 are managed by the database 140 in a distinguishable manner by the identification information. Here, the identification information is a 3-digit number, but other types of identification information may be adopted.

  The vibration analysis system 50B basically operates in the same manner as the first and second embodiments. However, the determination unit 130 collates the determination target profile D3 acquired from the index generation unit 120 with the reference profile D3 in the database 140 in the index determination process S230 (see FIG. 22).

  More specifically, the determination unit 130 uses, as a search condition, having the same identity as the determination target profile D3 (that is, having the same identity as the frequency component value included in the determination target profile D3). Search the reference profile D3. When the reference profile D3 satisfying the search condition is found, the determination unit 130 acquires identification information of the found reference profile D3 and outputs the identification information as the determination result information D4.

  Here, when the reference profile D3 satisfying the search condition is not found, the determination unit 130 does not output the determination result information D4. However, the discrimination result information D4 indicating that the reference profile D3 satisfying the search condition is not found may be output.

  The determination result information D4 is used in the main processing unit 11 (see FIG. 5). Specifically, when the determination result information D4 is "0001" (that is, when it is determined that the input receiving member 30B has been rubbed), for example, the main processing unit 11 reduces the volume of the electric device 10. On the other hand, when the determination result information D4 is "0002" (ie, when it is determined that the input receiving member 30C has been rubbed), for example, the main processing unit 11 increases the volume of the electric device 10.

  In the above, the case where two types of input reception members 30B and 30C were rubbed independently was explained. On the other hand, when the two types of input receiving members 30B and 30C are continuously rubbed, index spots SP having different frequency components appear in time series. Furthermore, the order of expression depends on the order of rubbing. This is shown in FIGS. FIG. 33 shows a case where the input receiving member 30B is rubbed first and then the input receiving member 30C is rubbed, as indicated by thick arrows. FIG. 34 shows the case where rubbing is performed in the reverse order to that of FIG. 33, in other words, in the reverse direction.

  Here, in FIGS. 33 to 34, the input receiving members 30B and 30C are arranged without a gap. In this case, the input receiving members 30B and 30C can be formed as an integral member. Moreover, it becomes easy to rub continuously. In addition, the direction and order of arrangement are not limited to the examples of FIGS.

  On the other hand, the input receiving members 30B and 30C may be separated. That is, as described in step S215 of FIG. 12, for example, when the state in which the intensity of the output signal of the vibration sensor 40 falls below the predetermined analysis mode end threshold continues for a predetermined time, it is determined that the rubbing operation is ended. In other words, even if the intensity of the output signal of the vibration sensor 40 falls below the predetermined analysis mode end threshold, it is not determined that the rubbing operation has ended if the duration is shorter than the predetermined time. In view of this, when the finger is quickly moved from one input receiving member 30 to the other input receiving member 30, the operation of continuously rubbing the two input receiving members 30 is recognized as one rubbing operation. . Even if the finger is separated from the input receiving member 30 during the movement of the finger, the same applies.

  According to the example of FIGS. 33 to 34, different reference profiles D3 are created according to the difference in the rubbing direction (in other words, the rubbing order), and the two types of reference profiles D3 are registered in the database 140. Good.

  Alternatively, reference profiles D3 for the input receiving member 30B and the input receiving member 30C may be separately created, and the two types of reference profiles D3 may be registered in the database 140. In such a case, it is possible to determine time-series changes in frequency components of the index spot SP by partially checking the determination target profile D3 with the database 140. That is, the rubbing direction can be identified.

  Although the rubbing operation in one direction is shown in FIGS. 33 and 34, as shown in FIGS. 35 and 36, it is also possible to determine the rubbing operation in both directions. FIG. 35 shows a case where the input receiving member 30B is rubbed so as to pass through the input receiving member 30C and return to the input receiving member 30B. FIG. 36 shows the case of reciprocating in the reverse order of FIG.

  Even when the input receiving members 30B and 30C are arranged without a gap, an operation of rubbing only one of the input receiving members 30 is also possible. Further, according to the three or more types of input receiving members 30, not only a linear array, but also an array such as a matrix or a circle graph can be adopted. Furthermore, by utilizing a part or all of such three or more types of input receiving members 30, it is possible to employ a rubbing operation of various trajectories such as circular, U-shaped, Z-shaped, and N-shaped.

  By the way, in the above, the input receiving member 30 is rubbed with a finger. However, the input tool for rubbing the input receiving member 30 is not limited to the finger. For example, a pen can be used as an input tool. The pen may be a pen dedicated to the user interface system 20 or 20B, or may be a general-purpose pen.

  Here, even if the input receiving member 30 is the same, if the attribute of the input tool is different, the generated vibrations are different. This can be understood, for example, from the experience that different sounds are heard when moving a finger on a paper with the same eye roughness and when moving a pen harder than the finger. Similarly, pens with different attributes (eg, material) will generate different vibrations.

  In view of this point, only one input receiving member 30 may be provided, and a reference profile D3 (for example, a reference profile D3 for a finger and a pen) for each type of input tool may be registered in the database 140. Alternatively, a plurality of types of input receiving members 30 may be provided, and a reference profile D3 for each type of input tool may be registered in the database 140 for some or all of the input receiving members 30.

  As described above, by making the plurality of reference profiles D3 database, it is possible to determine more rubbing operations. As a result, more operations can be realized.

  The input receiving member 30 is referred to as a first object, the input tool is referred to as a second object, and the input receiving members 30 having different attributes correspond to different first members, and the input devices having different attributes are different. It corresponds to the second member. In this case, the frequency components of the vibration generated by rubbing the first object with the second object can be generalized as different depending on the combination of the first object and the second object. That is, the difference in frequency component of the rubbing vibration is determined based on the difference in combination of the first object and the second object.

  The difference in frequency components of the rubbing vibration is determined by the difference in frequency components of the index spot SP. For this reason, resolution is required for the index spot SP with respect to frequency components. Regarding this point, according to the time frequency representation (CWD, BJ, ZAM, BD, MBD, KCS, etc.) whose resolution is higher than STFT and the interference component is smaller than WVD, the time frequency distribution that is the origin of index spot SP In the shape of the above, it is possible to provide a specification that can make the difference in the frequency component of the rubbing vibration apparent.

  The combination of the first object and the second object is determined in advance in the specifications of the user interface systems 20 and 20B. Therefore, the resolution of the frequency component of the index spot SP (in other words, the resolution of the time frequency distribution from which the index spot SP originates) may be one predetermined combination or a plurality of predetermined combinations. It is sufficient if it can be determined. Such performance can be realized by the above time frequency representation (CWD, BJ, ZAM, BD, MBD, KCS, etc.).

Fourth Preferred Embodiment
FIG. 37 shows a functional block diagram of a vibration analysis system 50C according to the fourth embodiment. The vibration analysis system 50C has a configuration obtained by removing the determination unit 130 from the vibration analysis system 50 of FIG. The vibration analysis system 50C can be realized by the same hardware configuration as the vibration analysis system 50.

  For example, the vibration analysis system 50C can be used as a dedicated system for generating the reference profile D3 (in other words, a system separate from the user interface systems 20 and 20B). Specifically, the vibration analysis system 50C acquires the vibration waveform data D1 from the vibration sensor 40 or the storage unit in which the vibration waveform data D1 is stored in advance, and generates the index profile D3 similarly to the vibration analysis system 50. .

  Then, the vibration analysis system 50C records the generated index profile D3 on the storage medium 60 as shown in FIG. Alternatively, a plurality of index profiles D3 may be recorded on the storage medium 60 as the database 140 as shown in FIG.

  The storage medium 60 is, for example, a storage medium constituting the storage unit 52 (see FIG. 5) of the vibration analysis system 50C.

  Alternatively, the storage medium 60 may be a storage medium externally attached to the vibration analysis system 50C. In this case, the index profile D3 may be recorded on the storage medium 60 without passing through the storage unit 52, or may be transferred from the storage unit 52 to the storage medium 60. The external storage medium 60 can be used, for example, as an original medium for supplying the same reference profile D3 to a plurality of systems.

  Here, the configuration of the analysis system 50C (see FIG. 37) is also included in the vibration analysis systems 50 and 50B (see FIGS. 5 and 31). Therefore, the vibration analysis system 50, 50B may be provided with a function of generating the reference profile D3.

  Here, an example in which the vibration analysis system 50B (see FIG. 31) is provided with the function is shown in FIG. According to the vibration analysis system 50D of FIG. 40, the index generation unit 120 can output the index profile D3 to the database 140. More specifically, in the case of the determination mode, the index generation unit 120 supplies the generated index profile D3 to the determination unit 130. On the other hand, in the case of the registration mode, the index generation unit 120 registers the generated index profile D3 in the database 140. The switching between the determination mode and the registration mode is instructed by the user using the information input unit 13 (see FIG. 5) or an input unit (not shown) of the user interface system 20 or 20B.

  The vibration analysis systems 50C, 50D and the storage medium 60 are useful in the task of installing the reference profile D3 or its database 140 in the user interface system 20, 20B. In addition, it is possible to use also in the work which adds or updates standard profile D3.

<Modification of Embodiments 1 to 4>
In the above, various functions of the vibration analysis systems 50 and 50B to 50D are accommodated in one device (for example, the electric device 10). However, various functions of the vibration analysis systems 50 and 50B to 50D may be distributed to, for example, a plurality of devices connected by a communication network.

  Although the present invention has been described in detail, the above description is an exemplification in all aspects, and the present invention is not limited thereto. It is understood that countless variations not illustrated are conceivable without departing from the scope of the present invention.

20, 20B User interface system 30, 30B, 30C Input receiving member 40 Vibration sensor 50, 50B to 50D Vibration analysis system 60 Storage medium 110 Time frequency analysis unit 120 Index generation unit 130 Discrimination unit 140 Database AP, AP1 to AP4 Analysis period D1 Vibration waveform data D2 Time frequency distribution data D3 Index profile (vibration analysis data)
D4 Discrimination result information SP Index spot SQ Candidate point S210 Time frequency analysis processing S220 Index generation processing S221 Index determination processing S222 Index profile generation processing S230 Discrimination processing S250 Speed discrimination processing S270 Intensity discrimination processing

Claims (14)

A time-frequency analysis unit that generates time-frequency distribution data representing the vibration generated by rubbing the first object with the second object in a predetermined time-frequency expression,
The predetermined time-frequency expression has a specification capable of eliciting the difference in frequency components of the vibration in the shape of the time-frequency distribution, which is determined based on the difference in combination of the first object and the second object. And
An index determination process is performed to determine an index spot, which is a point serving as an index of a distribution shape, from a plurality of points constituting the time frequency distribution provided by the time frequency distribution data, and one or more of the index spots A vibration analysis system, further comprising: an index generation unit that generates an index profile including frequency component values. The vibration analysis system according to claim 1, wherein
The vibration analysis system, wherein the index profile further includes at least one of a time component value and a distribution intensity value of the one or more index spots. The vibration analysis system according to claim 1 or 2, wherein
The indicator determination process is
First index determination processing of extracting a point satisfying a predetermined extraction condition as the index spot;
Or
Vibration analysis system which is a second index determination process of extracting the point satisfying the predetermined extraction condition as a candidate point for the index spot and determining a point representing a plurality of candidate points as the index spot . The vibration analysis system according to claim 3, wherein
The predetermined extraction condition is
A first extraction condition including at least one of a first condition of being a peak point and a second condition of being a distribution intensity value exceeding a predetermined threshold value;
Or
The vibration analysis system which is the 2nd extraction condition of being a point extracted by performing edge extraction processing to the time frequency distribution data. The vibration analysis system according to any one of claims 1 to 4, wherein
The vibration analysis system, wherein the time frequency analysis unit discretely sets an analysis period for one rubbing operation, and generates the time frequency distribution data only for the analysis period. The vibration analysis system according to claim 5, wherein
The vibration analysis system, wherein the index generation unit includes, in one index profile, data of the index spot obtained for a plurality of analysis periods in the one rubbing operation. The vibration analysis system according to any one of claims 1 to 6, wherein
It is determined whether or not the frequency component value included in the determination target profile which is the index profile to be determined is identical to the frequency component value included in the reference profile which is the registered index profile. A vibration analysis system further comprising a determination unit. The vibration analysis system according to claim 7, wherein
The determination unit compares the number of the index spots included in the determination target profile per predetermined time with the number of the index spots per predetermined time included in the reference profile to obtain the determination target profile. Vibration analysis system to determine the relative speed of the corresponding rubbing action. The vibration analysis system according to claim 7 or 8, wherein
The indicator profile further includes distribution intensity values of the one or more indicator spots,
The determination unit determines the relative intensity of the rubbing operation corresponding to the determination target profile by comparing the distribution intensity value included in the determination target profile with the distribution intensity value included in the reference profile. Do,
Vibration analysis system. The vibration analysis system according to any one of claims 7 to 9, wherein
It further comprises a database in which a plurality of reference profiles are managed in a distinguishable manner by identification information,
The determination unit searches the plurality of reference profiles using the same frequency component value included in the determination target profile as a search condition, and acquires the identification information of the reference profile that satisfies the search condition. Do,
Vibration analysis system. The vibration analysis system according to any one of claims 1 to 10, wherein
A vibration analysis system, wherein the resolution of the predetermined time frequency representation is higher than STFT (Short Time Fourier Transform), and the interference component of the predetermined time frequency representation is smaller than WVD (Wigner-Ville Distribution). At least one type of input receiving member,
A vibration sensor arranged to capture a vibration generated by rubbing the at least one type of input receiving member with an input tool;
A time frequency analysis unit that acquires waveform data of the vibration from the vibration sensor and generates time frequency distribution data representing the vibration in a predetermined time frequency expression from the waveform data;
The predetermined time frequency expression has a specification capable of eliciting, in the shape of a time frequency distribution, a difference in frequency components of the vibration, which is determined based on a difference in combination of the input receiving member and the input tool.
An index profile including frequency component values of one or more index spots is performed by performing an index determination process of determining an index spot which is a point serving as an index of distribution shape in the time frequency distribution provided by the time frequency distribution data. An index generation unit to generate
It is determined whether or not the frequency component value included in the determination target profile which is the index profile to be determined is identical to the frequency component value included in the reference profile which is the registered index profile. A user interface system further comprising a determination unit. A user interface system according to claim 12, wherein
It further comprises a database in which a plurality of reference profiles are managed in a distinguishable manner by identification information,
The determination unit searches the plurality of reference profiles using the same frequency component value included in the determination target profile as a search condition, and acquires the identification information of the reference profile that satisfies the search condition. Do,
User interface system. (A) generating time frequency distribution data representing the vibration generated by rubbing the first object with the second object in a predetermined time frequency expression,
The predetermined time-frequency expression has a specification capable of eliciting the difference in frequency components of the vibration in the shape of the time-frequency distribution, which is determined based on the difference in combination of the first object and the second object. And
(B) Among the plurality of points constituting the time frequency distribution provided by the time frequency distribution data, an index determination process is performed to determine an index spot which is a point serving as an index of the distribution shape, and one or more A method of generating vibration analysis data, further comprising the step of generating an index profile including frequency component values of the index spot.