JP7495728B2 - Vibration control device, vibration control program, and vibration control method - Google Patents

Description

The technology described in this specification relates to a vibration control device, a vibration control program, and a vibration control method.

In recent years, vibration feedback has become increasingly sophisticated in fields such as smartphones, game consoles, virtual reality (VR) devices, and robot operation support. Specifically, vibration devices that can provide a realistic sense of touch by reproducing vibrations over a wide frequency range have been developed.

Hideto Takenouchi, Nan Cao, Hikaru Nagano, Masashi Konyo, Satoshi Tadokoro, the 2017 IEEE/SICE International Symposium on System Integration, "Extracting Haptic Information from High-Frequency Vibratory signals Measured on a Remote Robot to Transmit Collisions with Environments", pp. 968-973, December 2017. Nan Cao, Hikaru Nagano, Masashi Konyo, Shogo Okamoto, Satoshi Tadokoro, "A Pilot Study: Introduction of Time-Domain Segment to Intensity-Based Perception Model of High-Frequency Vibration", June 2018 Nan Cao, Hikaru Nagano, Masashi Konyo, Satoshi Tadokoro, the 27th IEEE International Symposium on Robot and Human Interactive Communication, "Sound reduction of vibration feedback by perceptually similar modulation", August 2018 Sliman Bensmaia, Mark Hollins, Jeffrey Yau, Attention, "Vibrotactile intensity and frequency information in the Pacinian system: A psychophysical model", Perception, & Psychophysics, Vol. 67, No. 5, pp 828-84, July 2005

However, when attempting to present high-frequency vibrations, for example, at or above 300 Hz, problems with the device, perception sensitivity, and auditory noise can occur. Because the amplitude of the vibrator is small in the high-frequency band, it is not easy to generate both high-frequency and low-frequency vibrations in devices that utilize resonance. In addition, humans have a peak at 200-300 Hz and their perception sensitivity weakens for vibration frequencies above that, so sufficient amplitude is required to perceive the vibration. Furthermore, when the frequency exceeds about 300 Hz, the vibration becomes audible as sound. For example, if vibrations are generated in combination with music or movie content, they may be perceived as noise that interferes with the sound source of the music or movie content.

In one aspect, the technology described herein aims to generate vibrations in the high frequency range that are easily perceptible by humans.

In one aspect, a vibration control device that controls vibrations by a vibration device using a signal includes a time division control unit that divides the signal into predetermined time intervals, and an energy control unit that converts the waveform of the signal for each of the predetermined time intervals divided by the time division control unit, and the energy control unit adjusts the energy of a signal in a specific frequency band among the frequency components of the signal to convert the waveform, while converting signals outside the specific frequency band into a waveform having a different frequency from that of the signal while maintaining the energy.

One aspect is that it can generate vibrations in the high frequency range that are easily perceived by humans.

1 is a block diagram illustrating a schematic configuration example of a vibration generating system according to an embodiment. 2 is a graph showing waveforms of a signal before and after conversion by the vibration control device shown in FIG. 1 . 1 is a graph showing the discriminability of vibration by humans. 4 is a sample waveform of vibration used in a forced three-choice discrimination experiment conducted to determine the discriminability shown in the graph in FIG. 3. 4 is a graph showing signal waveforms before and after conversion for each segment by the vibration control device shown in FIG. 1 . 13 is a graph showing the amplitude threshold _Tf used in calculating the corrected energy. 13 is a graph showing the exponent threshold b used in calculating the corrected energy; 2 is a diagram illustrating the use of a window function in the vibration control device shown in FIG. 1 . 2 is a graph illustrating an example of synthesis of a low frequency and a high frequency in the vibration control device shown in FIG. 1 . 4 is a graph showing a specific example of a signal waveform before and after conversion by the vibration control device shown in FIG. 1 . 2 is a block diagram illustrating an example of a functional configuration of an Intensity Segmentation Method (ISM) unit in the vibration control device shown in FIG. 1 . FIG. 2 is a block diagram for explaining a first embodiment of a vibration waveform generation process in the vibration control device shown in FIG. 1 . FIG. 12 is a block diagram illustrating details of the energy control process shown in FIG. 11 . FIG. 12 is a block diagram for explaining a process of separating low-frequency components in the energy control process shown in FIG. 11 as a second embodiment of the process of generating a vibration waveform in the vibration control device shown in FIG. 1 . 11A to 11C are graphs illustrating an example of generating vibration according to ISM without enhancing the waveform. 6A to 6C are graphs illustrating a first example of emphasizing and separating high frequency components of 3000 Hz or higher from a sound source. 6A to 6C are graphs illustrating a second example of emphasizing and separating high frequency components of 3000 Hz or higher from a sound source. 11A to 11C are graphs illustrating an example of emphasizing and separating low-frequency components below 1000 Hz from a sound source. FIG. 12 is a block diagram illustrating a first modified example of the energy control process shown in FIG. 11 . FIG. 12 is a block diagram illustrating a second modified example of the energy control process shown in FIG. 11 . FIG. 12 is a block diagram illustrating details of the energy synthesis process shown in FIG. 11 . 12 is a block diagram illustrating the details of the generation process of the corrected vibration waveform shown in FIG. 11. 2 is a block diagram showing an example of the configuration of a DAC when a plurality of vibration devices are used in the vibration generating system shown in FIG. 1 . 2 is a block diagram showing an example of the configuration of a DAC when a single vibration device is used in the vibration generating system shown in FIG. 1 .

The following describes the embodiments with reference to the drawings. However, the embodiments described below are merely examples, and are not intended to exclude the application of various modified examples or techniques not explicitly stated in the embodiments. In other words, the present embodiment can be implemented with various modifications without departing from the spirit of the present invention.

In addition, each figure does not necessarily include only the components shown in the figure, but may include other components. In the following figures, parts with the same reference numerals indicate the same or similar parts unless otherwise specified.

[A] Embodiment FIG. 1 is a block diagram that illustrates a schematic configuration example of a vibration generating system 100 according to an embodiment.

The vibration generating system 100 includes a vibration control device 1, a digital analog converter (DAC) 2, a high-frequency vibration 31, a low-frequency vibration 32, an earphone (L) 41, and an earphone (R) 42.

DAC2 may be called Universal Serial Bus (USB) audio, and converts the digital signal input from the vibration control device 1 into an analog signal. DAC2 then outputs the converted analog signal to high-frequency vibration 31, low-frequency vibration 32, earphone (L) 41, and earphone (R) 42. Note that an amplifier (not shown) for driving high-frequency vibration 31, low-frequency vibration 32, earphone (L) 41, and earphone (R) 42 may be provided downstream of DAC2.

The low-frequency vibration 32 shown in FIG. 1 is an example of a first vibration device, which generates vibrations due to signal components below a predetermined frequency. The high-frequency vibration 31 shown in FIG. 1 is an example of a second vibration device, which generates vibrations due to signal components above a predetermined frequency. Note that the low-frequency vibration 32 does not have to be provided in the vibration generating system 100. In that case, vibrations due to signal components below a predetermined frequency may be generated from the high-frequency vibration 31, or vibrations due to signal components below a predetermined frequency may not be generated in the vibration generating system 100.

The predetermined frequency that separates the signal components of the vibration output from the low-frequency vibration 32 and the signal components of the signal output from the high-frequency vibration 31 may be a frequency in the range of 80 Hz to 400 Hz.

The earphone (L) 41 generates a sound from the stereo sound source that is input to the human left ear. The earphone (R) 42 generates a sound from the stereo sound source that is input to the human left ear. The earphone (L) 41 and the earphone (R) 42 do not have to be provided in the vibration generating system 100. The earphone (L) 41 and the earphone (R) 42 may be shared to generate a monaural sound source. Furthermore, the vibration generating system 100 may be provided with a speaker instead of the earphone (L) 41 and the earphone (R) 42, and may output a sound source of three or more channels.

The vibration control device 1 includes a Central Processing Unit (CPU) 11, a memory 12, and a storage device 13.

The vibration control device 1 in one example of this embodiment may convert acoustic information such as music, movies, and voice into a haptic signal. When the frequency exceeds about 300 to 400 Hz, the vibration becomes audible as sound, resulting in noise. For this reason, in conventional vibration experience devices for music, movies, and the like, a low-pass filter at about several hundred Hz is often applied to cut off the high frequency band. On the other hand, the vibration control device 1 in one example of this embodiment converts the waveform of the high frequency band into another frequency in the low frequency band and outputs it.

The vibration control device 1 in one example of this embodiment may modulate the high-frequency vibrations generated when the robot comes into contact with an object to a frequency band that can be perceived by humans. By transmitting the vibrations generated when the robot comes into contact with an object to a remote operator, it is possible to grasp the strength of the collision with the object and the state of friction. A robot that holds a metal casing, such as a construction robot, may generate vibrations in a band that cannot be perceived by humans when it comes into contact with an object. Therefore, the vibration control device 1 in one example of this embodiment modulates the frequency band of the output signal.

Furthermore, the vibration control device 1 in one example of this embodiment may be applied to chairs, suits, headsets, etc. that include a vibration device.

Memory 12 is a storage device that includes Read Only Memory (ROM) and Random Access Memory (RAM).

The storage device 13 is a device that stores data in a readable and writable manner, and may be, for example, a hard disk drive (HDD), a solid state drive (SSD), or a storage class memory (SCM). The storage device 13 stores the generated teacher data, learning models, etc.

The CPU 11 is a processing device that performs various controls and calculations, and realizes various functions by executing an operating system (OS) and programs stored in the memory 12. That is, as shown in FIG. 1, the CPU 11 may function as a frequency removal control unit 111, a time division control unit 112, an energy control unit 113, and a signal output unit 114.

The CPU 11 is an example of a computer, and illustratively controls the operation of the entire vibration control device 1. The device for controlling the operation of the entire vibration control device 1 is not limited to the CPU 11, and may be, for example, any one of an MPU, DSP, ASIC, PLD, FPGA, or dedicated processor. The device for controlling the operation of the entire vibration control device 1 may also be a combination of two or more of a CPU, MPU, DSP, ASIC, PLD, FPGA, and dedicated processor. Note that MPU is an abbreviation for Micro Processing Unit, DSP is an abbreviation for Digital Signal Processor, and ASIC is an abbreviation for Application Specific Integrated Circuit. Also, PLD is an abbreviation for Programmable Logic Device, and FPGA is an abbreviation for Field Programmable Gate Array.

The frequency removal control unit 111 removes the first signal component having a frequency below a predetermined frequency.

The time division control unit 112 divides the second signal components other than the first signal components removed by the frequency removal control unit 111 at predetermined time intervals.

The energy control unit 113 converts the waveform of the second signal component while maintaining the energy of the second signal component for each predetermined time divided by the time division control unit 112.

The signal output unit 114 outputs the first signal component removed by the frequency removal control unit 111 in addition to the second signal component whose waveform has been converted by the energy control unit 113.

Figure 2 is a graph showing the waveforms of the signal before and after conversion by the vibration control device 1 shown in Figure 1.

Taking into account the human perception characteristics of high-frequency vibrations, the system focuses on the vibration energy that is correlated with human perception characteristics in the high-frequency band, rather than on the waveform itself, and replaces the waveform with another waveform that has the same vibration energy, making it possible to change the frequency band. In the example shown in Figure 2, the waveform indicated by symbol A1 is converted to the waveform indicated by symbol A2.

By dividing any continuous vibration signal into appropriate time intervals that take into account human perception characteristics and converting each divided segment into vibration energy, it is possible to convert the signal into any signal waveform while maintaining the same tactile sensation felt by humans, or to enable the perception of high frequency bands that are difficult to sense.

By appropriately selecting the frequency of the converted vibration, it is possible to drive the transducer efficiently according to its response range, reduce auditory noise, and convert the sound into any sound source.

It is said that humans can only perceive vibrations up to about 1 kHz. For this reason, vibrations above 1 kHz are often ignored. On the other hand, it is known that even with vibrations above 1 kHz, if the amplitude is an amplitude modulated wave whose amplitude fluctuates within a band that can be sensed by humans, the envelope components can be perceived.

On the other hand, a vibration energy model (see, for example, Reference 4) is known as a human perception characteristic for high-frequency vibrations of about 100 Hz or more. From this, it is known that vibrations cannot be differentiated even if the carrier frequency of an amplitude-modulated wave is replaced while maintaining the high-frequency vibration energy (see, for example, References 2 and 3). However, even if the vibration energy is maintained, as mentioned above, there are cases where the envelope components of the vibration can be perceived as differences in tactile information, and the range of perception has not been investigated. Furthermore, although Reference 2 devisees a method of converting signals based on vibration energy in a time-division manner, it does not consider a method of maintaining low-frequency components.

Figure 3 is a graph showing the discrimination ability of vibrations by humans. Figure 4 is a sample waveform of vibrations used in a forced three-choice discrimination experiment conducted to determine the discrimination ability shown in the graph shown in Figure 3.

Based on a conventionally known vibration energy model (see, for example, Reference 4), the graph shown in Figure 3 is obtained by investigating human perceptual discrimination characteristics while maintaining vibration energy. Symbols B1 and B2 in Figure 4 show the same waveform, while symbol B3 in Figure 4 shows a different waveform. The subjects are asked to compare the constant amplitude motion shown by symbols B1 and B2 in Figure 4 with the amplitude modulated stimulus shown by symbol B3, and to answer which is the amplitude modulated wave. In Figure 3, the correct answer rate obtained in a forced three-choice discrimination experiment is expressed as Sensitivity (d': d-prime), which is a discrimination performance index based on signal detection theory, and when d' is 1 or less, the correct answer rate falls below about 60%.

According to the graph in Figure 3, the upper limit of the frequency at which the envelope components can be distinguished is approximately 80 to 125 Hz. In addition, it is not necessary to preserve the envelope components above this upper limit of frequency, and this shows that if the carrier frequency of the amplitude modulated wave is replaced while maintaining the vibration energy, the stimuli cannot be distinguished.

As mentioned above, even if the vibration energy is maintained, if the energy fluctuates in the low-frequency range, the fluctuations may be perceived as differences in tactile information, but the range of perception has not been investigated. Therefore, based on the discovery that the upper limit of perceptible low-frequency fluctuations is approximately 80 to 125 Hz, we will convert the vibration energy while maintaining the low-frequency components using two measures (see measures [1] and [2] below).

Figure 5 is a graph showing the signal waveforms before and after conversion for each segment by the vibration control device 1 shown in Figure 1.

Humans' high frequency perception is based on vibrational energy rather than the waveform itself, so if the vibrational energy is maintained, the same sensation will be felt. However, if the vibrational energy fluctuations occur below about 80-125 Hz, it is necessary to reproduce the vibrational energy fluctuations.

Therefore, in one example of this embodiment, as a means of maintaining the fluctuation of vibration energy below a predetermined frequency (e.g., about 80 to 125 Hz), the vibration is time-divided, for example, in a range of about 80 to 200 Hz, the vibration energy is obtained for each segment, and it is replaced with a vibration having a different carrier frequency.

In the example shown in FIG. 5, the original vibration signal shown with reference symbol C1 and the converted signal shown with reference symbol C2 are converted so that the energy of the converted signal is the same as the energy of the original vibration signal within the same time segment.

The width of the time division (in other words, the division width) should be set to a level that can express energy fluctuations of 80 to 125 Hz or less (in other words, to a level where the peaks of the fluctuations are aligned) (Measure [1]). The frequency of the division width may be 80 to 125 Hz or more, but if the division width is made too short, the accuracy of estimating vibration energy with a period longer than the division width will deteriorate. Therefore, by using the following measure [2], the waveform of vibrations whose energy cannot be estimated is output as is.

Furthermore, components below a predetermined frequency may be extracted and presented as such as a vibration stimulus (measure [2]). The predetermined frequency may be 80-125 Hz or higher, but components above the predetermined frequency may be represented by the energy control unit 113 of the second signal component. This allows for flexibility in frequency selection. However, if the predetermined frequency is set too high, noise problems may occur or a wideband vibration device may be required.

According to the above measures [1] and [2], the specified frequency may be about 80 to 400 Hz. 400 Hz is the upper limit from the viewpoint of noise issues and the performance of the vibration device.

Setting the specified frequency also involves the selection of the carrier frequency when converting vibrations. Since the peak vibration frequency at which human perception improves is around 200 to 250 Hz, a practical carrier frequency that increases sensitivity without causing noise is around 150 to 400 Hz. The carrier frequency may be a constant multiple of the division width. In addition, multiple different carrier frequencies may be used, and may include a high frequency range of 400 Hz or higher.

In addition, the predetermined frequency that separates low and high frequencies does not necessarily have to match the frequency of the division width for calculating energy.

According to Reference 4, the corrected energy, which is vibration energy corrected to improve human perceptibility, can be expressed as follows:

Ａは、分離された基底信号ｇ_ｋの振幅である。Ｔ_ｆは、振幅閾値であり、周波数ｆの信号においてヒトが感じられる最小の振幅である。ｂ_ｆは、指数値であり、周波数ｆの信号における非線形特性である。

A is the amplitude of the separated base signal g _k . T _f is the amplitude threshold, which is the minimum amplitude that humans can sense in a signal of frequency f. b _f is the exponent value, which is the nonlinear characteristic in a signal of frequency f.

FIG. 6 is a graph showing the amplitude threshold _Tf used in calculating the correction energy.

As shown in FIG. 6, the amplitude threshold varies depending on the frequency. In the range of approximately 10 ² to 10 ³ Hz, humans can sense even relatively small amplitudes, but in other ranges, humans can only sense relatively large amplitudes.

FIG. 7 is a graph showing the exponent values _bf used in calculating the correction energy.

The exponent values _bf in FIG. 7 are an example in which values obtained by linearly interpolating the exponent values _bf below 400 Hz that have been reported in the past are used.

Figure 8 is a diagram explaining the use of window functions in the vibration control device 1 shown in Figure 1.

As shown by reference D1, a high-frequency signal H(t) is input. As shown by reference D2, the high-frequency signal H(t) is divided into frames as signals h _i , h i+1 , h i+2 , ... for each frame i, _i+1 , i ₊₂ , .... As shown by reference D3, the signal h of each divided frame is separated into a plurality of basis signals g ₁ , g ₂ , g ₃ .... As shown by reference D4, scalar values E _i , E i+1 , E _{i +2 ,} ... obtained by combining the correction energies of all basis signals g ₁ , g ₂ , g ₃ ... are output based on the frequencies f ₁ , f ₂ , f ₃ ... of the basis signals g ₁ , g ₂ , g 3 .... As shown by reference D5, the scalar values _Ei , _Ei+1 , _Ei+2 , ... of the vibration energy calculated in each frame i are converted into vibration waveforms having the same vibration energy but different carrier frequencies, and windowing processing using a window function is performed on the amplitudes _ai (t), _ai+1 (t), _ai+2 (t), ... of the waveforms. As shown by reference D6, frame synthesis is performed on the 1st to Nth frames, and the amplitude A(t) of the vibration waveform is output. As shown by reference D7, a second vibration waveform _S2 (t) having a carrier frequency such that the amplitude is A(t) is output.

Figure 9 is a graph that illustrates an example of combining low and high frequencies in the vibration control device 1 shown in Figure 1.

The second vibration waveform _S2 (t) shown in E1, which is generated from the high-frequency signal H(t) using the window function in Fig. 8, is synthesized with the first vibration waveform S1(t) shown in E2, which _is the low-frequency signal L(t) output as is, to output a synthesized waveform _S1 (t)+ _S2 (t) shown in E3.

Figure 10 is a graph showing a specific example of a signal waveform before and after conversion by the vibration control device 1 shown in Figure 1.

In FIG. 10, the waveform of the violin sound before conversion (see symbol F1) and the waveform after conversion (see symbol F2) are shown by amplitude over time.

When using conventional tactile vibrations to detect high-frequency vibrations, such as those of a violin, loud auditory noise is generated, and applying a low-pass filter eliminates the vibrations that humans can perceive. Therefore, the correction energy is calculated so that the waveform becomes a single wavelength with a low-frequency carrier frequency over time.

Figure 11 is a block diagram illustrating an example of the functional configuration of the ISM section 101 in the vibration control device 1 shown in Figure 1.

The ISM unit 101 functions as a time division control unit 112, an energy control unit 113, an energy vibration conversion unit 114a, and a vibration generation unit 114b. In this embodiment, the ISM unit 101 controls the vibration including high-frequency components caused by the high-frequency vibration 31 using a signal. Note that, based on the human perception characteristics of vibration energy, the high-frequency components of the signal X(t) are expected to be about 100 Hz or higher, but they may be converted to low-frequency components below 100 Hz. This makes it possible to emphasize the low-frequency components. The methods of controlling vibration based on the time division of energy according to the present invention are collectively referred to as ISM.

The time division control unit 112 divides the vibration signal X(t) into N frames in time, and inputs the time-divided i-th frame signal h _i to the energy control unit 113. The number of frames N may be determined by a predetermined period and an overlap rate of the windowing process.

The energy control unit 113 calculates a corrected energy e _i for the signal h _i of the i-th frame, and inputs the calculated corrected energy to the energy vibration conversion unit 114 a.

The energy vibration conversion section 114a generates a signal A(t) by combining the corrected energies e ₁ to e _N of the 1st to Nth frames, and inputs the signal A(t) to the second vibration generation section 114b.

The vibration generating unit 114b outputs a signal waveform S(t) based on the synthesized signal A(t).

[B] Example of Operation A first embodiment of the vibration waveform generation process in the vibration control device 1 shown in FIG. 1 will be described with reference to the block diagram (steps S1 to S7) shown in FIG.

The signal removal unit 111a and the low-pass filter 111b shown in FIG. 12 correspond to the frequency removal control unit 111 shown in FIG. 1. The energy vibration conversion unit 114a, the second vibration generation unit 114b, and the first vibration generation unit 114c shown in FIG. 12 correspond to the signal output unit 114 shown in FIG. 1.

The signal removal unit 111a removes components below a predetermined frequency from the acquired pre-conversion signal X(t) to generate a high-frequency signal H(t) and inputs it to the time division control unit 112 (step S1).

The time division control unit 112 divides the high frequency signal H(t) into N frames and inputs the time-divided i-th frame signal h _i to the energy control unit 113 (step S2). The number of frames N may be determined by a predetermined period and an overlap rate of the windowing process.

The energy control unit 113 calculates a corrected energy e _i for the signal h _i of the i-th frame, and inputs the calculated corrected energy to the energy vibration conversion unit 114 a (step S3).

The energy vibration conversion section 114a generates a signal A(t) by combining the corrected energies e ₁ to e _N of the 1st to Nth frames, and inputs the signal A(t) to the second vibration generation section 114b (step S4).

The second vibration generating unit 114b outputs a second vibration waveform S ₂ (t) based on the synthesized signal A(t) (step S5).

On the other hand, the low-pass filter 111b inputs the low-pass signal L(t) obtained by filtering out components below a predetermined frequency from the acquired pre-conversion signal X(t) to the first vibration generating unit 114c (step S6).

The first vibration generating unit 114c outputs a first vibration waveform S ₁ (t) based on the low-frequency signal L(t) (step S7).

Next, the details of the energy control process shown in step S3 of FIG. 12 will be explained with reference to the block diagram (steps S11 to S14) shown in FIG. 13.

As shown in FIG. 13, the energy control unit 113 functions as a basis signal separation control unit 113a, a frequency calculation unit 113b, an energy correction parameter calculation unit 113c, and a correction energy calculation unit 113d.

The basis signal separation control unit 113a separates the time-divided i-th frame signal h _i , which is an input signal, into a plurality of basis signals g, and inputs the separated k-th basis signal g _k to the frequency calculation unit 113b (step S11). For example, the signals may be separated by short-time Fourier analysis, wavelet analysis, Empirical Mode Decomposition (EMD) method, or the like.

The frequency calculation unit 113b calculates the frequency f _k of the k-th basis signal g _k by, for example, discrete Fourier analysis or Hilbert Spectrum analysis, and inputs the frequency f k to the energy correction parameter calculation unit 113c (step S12).

The energy correction parameter calculation unit 113c calculates the exponent value _bk and the amplitude threshold value _Tk described with reference to FIG. 6 based on the frequency _fk , and inputs them to the corrected energy calculation unit 113d (step S13).

The corrected energy calculation unit 113d calculates the corrected energy _Ipc for each basis signal _gk based on the exponent value _bk and the amplitude threshold _Tk in accordance with the formula shown in Equation 1, and outputs a scalar value _ej obtained by adding up the corrected energies of all the basis signals _gk (step S14).

Next, as a second example of the vibration waveform generation process in the vibration control device 1 shown in FIG. 1, the process of separating low-frequency components in the energy control process shown in FIG. 11 will be explained according to the block diagram (steps S101 to S105) shown in FIG. 14.

As shown in FIG. 14, the energy control unit 113 functions as a basis signal separation control unit 113a, a frequency calculation unit 113b, an energy correction parameter calculation unit 113c, and a correction energy calculation unit 113d, and may also have a function of separating low-frequency components to a low-frequency component synthesis unit 113g.

The basis signal separation control unit 113a separates the time-divided i-th frame signal h _i , which is an input signal, into a plurality of basis signals g, and inputs the separated k-th basis signal g _k to the frequency calculation unit 113b (step S101). For example, the signals may be separated by short-time Fourier analysis, wavelet analysis, EMD method, or the like.

The frequency calculation unit 113b calculates the frequency f _k of the k-th basis signal g _k by, for example, discrete Fourier analysis or Hilbert Spectrum analysis, and inputs the frequency f k to the energy correction parameter calculation unit 113c (step S102).

The energy correction parameter calculation unit 113c calculates the exponent value _bk and the amplitude threshold value _Tk described with reference to FIG. 6 based on the frequency _fk , and inputs them to the corrected energy calculation unit 113d (step S103).

The corrected energy calculation unit 113d calculates the corrected energy _Ipc for each base signal _gk based on the exponent value _bk and the amplitude threshold _Tk in accordance with the formula shown in Equation 1, and outputs a scalar value _ej obtained by adding up the corrected energies of all the base signals _gk (step S104).

The low-frequency component synthesis unit 113g synthesizes a base signal having a frequency f _k of the base signal g _k that is lower than a predetermined frequency, thereby generating a low-frequency component L(t) (step S105).

For a sound source that contains signals in multiple frequency bands, it may be desirable to emphasize the vibration energy of a specific frequency band and present it as vibration. In such a case, the energy control units 1131 and 1132 are modified examples that are applied when adjusting the energy of a base signal present in a predetermined frequency band to convert the waveform, as described below with reference to Figs. 15 to 20.

Figure 15 (a) to (c) are graphs that explain an example of generating vibrations according to ISM without emphasizing the waveform. Figure 15 shows a band corresponding to the waveform of the cymbal (drum) of the high frequency components from a piano trio piece, and bands corresponding to the waveforms of the piano and bass. In Figure 15 (a) to (c), the horizontal axis shows time [s], and the vertical axis shows frequency [Hz], with darker spectra indicating greater power and lighter spectra indicating less power.

Figure 15 (a) shows the distribution of the sound source spectrum, with the waveform of the cymbal, which is a high-frequency component, indicated by a dashed line, and the waveforms of the piano and bass, which are low-frequency components, indicated by a dashed line.

Figure 15(b) shows the spectral distribution (centered at 200 Hz) after conversion using ISM. In Figure 15(b), the cymbals, piano, and bass are all extracted as intensities due to the effect of ISM.

Figure 15 (c) shows an example of converting the signal to a signal using the representative frequency of the base signal, rather than converting to a signal with a frequency of 200 Hz based on the intensity. This makes it possible to visualize which frequency bands have been emphasized.

Figure 16 (a) to (c) are graphs explaining a first example of emphasizing and separating high frequency components from a sound source. Figure 16 shows an example of emphasizing and separating the high frequency components of cymbals (drums) from a piano trio piece of music. In Figure 16 (a) to (c), the horizontal axis represents time [s] and the vertical axis represents frequency [Hz], with darker spectra indicating greater power and lighter spectra indicating less power.

Figure 16 (a) shows the distribution of the sound source spectrum, with the waveform of the cymbal, which is a high-frequency component, indicated by a dashed line, and the waveforms of the piano and bass, which are low-frequency components, indicated by a dashed line.

Figure 16(b) shows the spectral distribution (centered at 200 Hz) after ISM conversion. In Figure 16(b), only the intensity above 3000 Hz is increased by +20 dB (100 times).

Figure 16(c) shows an example of converting the signal to a signal using the representative frequency of the base signal, rather than converting to a signal with a frequency of 200 Hz based on the intensity. This makes it possible to visualize which frequency bands are emphasized. In Figure 16(c), the power of the cymbal spectrum is increased.

Figure 17 (a) to (c) are graphs explaining a second example of emphasizing and separating high frequency components from a sound source. Figure 17 shows an example of emphasizing and separating the high frequency components of cymbals (drums) from a piano trio piece of music. In Figure 17 (a) to (c), the horizontal axis represents time [s] and the vertical axis represents frequency [Hz], with darker spectra indicating greater power and lighter spectra indicating less power.

Figure 17 (a) shows the distribution of the sound source spectrum, with the waveform of the cymbal, which is a high-frequency component, indicated by a dashed line, and the waveforms of the piano and bass, which are low-frequency components, indicated by a dashed line.

Figure 17(b) shows the spectral distribution (centered at 200 Hz) after ISM conversion. In Figure 17(b), the intensity above 3000 Hz is increased by +20 dB (100 times), while the intensity below 1000 Hz is decreased by -10 dB (1/10 times).

Figure 17(c) shows an example of converting the signal to a signal using the representative frequency of the base signal, rather than converting to a signal with a frequency of 200 Hz based on the intensity. This makes it possible to visualize which frequency bands are emphasized. In Figure 17(c), the power of the cymbal spectrum is increased.

Figure 18 (a) to (c) are graphs that explain an example of emphasizing and separating low-frequency components from a sound source. Figure 18 shows an example of emphasizing and separating the low-frequency components of piano and bass from a piano trio piece of music. In Figure 18 (a) to (c), the horizontal axis represents time [s] and the vertical axis represents frequency [Hz], with darker spectra indicating greater power and lighter spectra indicating less power.

Figure 18 (a) shows the distribution of the sound source spectrum, with the waveform of the cymbal, which is a high-frequency component, indicated by a dashed line, and the waveforms of the piano and bass, which are low-frequency components, indicated by a dashed line.

Figure 18(b) shows the spectral distribution (centered at 200 Hz) after ISM conversion. In Figure 18(b), the intensity below 1000 Hz is increased by +10 dB (10 times).

Figure 18 (c) shows an example of converting the signal to a signal using the representative frequency of the base signal, rather than converting to a signal with a frequency of 200 Hz based on the intensity. This makes it possible to visualize which frequency bands have been emphasized. In Figure 18 (c), the power of the piano and bass spectra has increased.

The first modified example of the energy control process shown in FIG. 11 will be explained with reference to the block diagram shown in FIG. 19 (steps S41 to S45).

As shown in FIG. 19, the energy control unit 1131 functions as a gain calculation unit 113e in addition to the basis signal separation control unit 113a, frequency calculation unit 113b, energy correction parameter calculation unit 113c, and correction energy calculation unit 113d shown in FIG. 13.

The basis signal separation control unit 113a separates the time-divided i-th frame signal h _i , which is an input signal, into a plurality of basis signals g, and inputs the separated k-th basis signal g _k to the frequency calculation unit 113b (step S41). For example, the signals may be separated by short-time Fourier analysis, wavelet analysis, EMD method, or the like.

The frequency calculation unit 113b calculates the frequency f _k of the k-th basis signal g _k by, for example, discrete Fourier analysis or Hilbert Spectrum analysis, and inputs the frequency f k to the energy correction parameter calculation unit 113c (step S42).

The energy correction parameter calculation unit 113c calculates the exponent value _bk and the amplitude threshold value _Tk described with reference to FIG. 6 based on the frequency _fk , and inputs them to the corrected energy calculation unit 113d (step S43).

The gain calculation unit 113e outputs a gain value G _k for each predetermined frequency band according to the frequency f _k of the calculated base signal g _k (step S44). When it is desired to emphasize the energy, G _k is set to >1, and when it is desired to suppress the energy, 0≦G _k <1 is set. The adjustment of the energy by the emphasis or suppression may be performed for one frequency band or for multiple frequency bands. Moreover, the adjustment of the energy may be performed for the entire frequency band input to the energy control unit 1131.

The corrected energy calculation unit 113d calculates a gain-adjusted corrected energy _Ipc for each of the separated base signals _gk in accordance with the following formula 2 for the amplitude A of the base signals _gk , and outputs a scalar value _ej obtained by adding up the corrected energies of all the base signals _gk (step S45).

The second modified example of the energy control process shown in FIG. 11 will be explained with reference to the block diagram (steps S51 to S55) shown in FIG. 20.

As shown in FIG. 20, the energy control unit 1132 functions as a gain calculation unit 113e and a signal source identification unit 113f in addition to the basis signal separation control unit 113a, frequency calculation unit 113b, energy correction parameter calculation unit 113c, and correction energy calculation unit 113d shown in FIG. 13.

The basis signal separation control unit 113a separates the time-divided i-th frame signal h _i , which is an input signal, into a plurality of basis signals g, and inputs the separated k-th basis signal g _k to the frequency calculation unit 113b (step S51). For example, the signals may be separated by short-time Fourier analysis, wavelet analysis, EMD method, or the like.

The frequency calculation unit 113b calculates the frequency f _k of the k-th basis signal g _k by, for example, discrete Fourier analysis or Hilbert Spectrum analysis, and inputs the frequency f k to the energy correction parameter calculation unit 113c (step S52).

The energy correction parameter calculation unit 113c calculates the exponent value _bk and the amplitude threshold value _Tk described with reference to FIG. 6 based on the frequency _fk , and inputs them to the corrected energy calculation unit 113d (step S53).

The signal source identification unit 113f estimates identification candidates from the input signals h _i and the history of h _i based on the set signal features, identifies which signal source the basis signal g _k belongs to, and outputs the identification result as an ID (identifier) or the like (step S54). The signal source identification unit 113f may prepare a classifier in advance by machine learning or the like. For example, the features of many musical instruments may be learned by deep learning, and a group of candidates (e.g., piano, bass, drums) may be estimated as to which musical instrument is included in the current input signal h _i (or the history of each of the multiple input signals h _i when the input signal h _i is too short), and it may be identified as to which musical instrument the basis signal g _k belongs to.

The gain calculation unit 113e outputs a gain value G _k for each predetermined frequency band according to the ID identified by the signal source identification unit 113f (step S55). When it is desired to emphasize energy, G _k is set to >1, and when it is desired to suppress energy, 0≦G _k <1 is set. The adjustment of energy by emphasis or suppression may be performed for one frequency band or for multiple frequency bands. Moreover, the adjustment of energy may be performed for the entire frequency band input to the energy control unit 1132.

The corrected energy calculation unit 113d calculates a gain-adjusted corrected energy _Ipc for each of the separated base signals _gk in accordance with the formula shown in Equation 2 for the amplitude A of the separated base signals _gk , and outputs a scalar value _ej obtained by adding up the corrected energies of all the base signals _gk (step S56).

Next, the details of the energy synthesis process shown in step S4 of FIG. 11 will be explained with reference to the block diagram (steps S21 to S23) shown in FIG. 21.

The energy vibration conversion unit 114a functions as an energy equivalent conversion unit 1141a, a windowing processing unit 1142a, and a frame synthesis unit 1143a.

As shown in FIG. 21, the energy equivalent conversion unit 1141a converts the scalar value e _i of the vibration energy calculated for each frame i into a vibration waveform having an equivalent vibration energy but a different carrier frequency, and outputs the amplitude a _i (t) of the waveform to the windowing processing unit 1142a (step S21).

The windowing processor 1142a performs windowing processing on the input amplitude a _i (t) of each frame i using the window function shown in FIG. 8, and inputs the processing result to the frame synthesizer 1143a (step S22).

The frame synthesis unit 1143a performs frame synthesis on the input from the windowing processing unit 1142a for the 1st to Nth frames, and outputs the amplitude A(t) of the vibration waveform (step S23).

Next, the details of the process for generating the corrected vibration waveform shown in step S5 of FIG. 11 will be described with reference to the block diagram shown in FIG. 22 (steps S31 and S32).

As shown in FIG. 22, the second vibration generating unit 114b functions as an amplitude vibration converting unit 1141b and a waveform output unit 1142b. The second vibration generating unit 114b has an input signal A(t) and outputs a sine wave having a carrier frequency. The generated waveform may be phase controlled so that the vibrations are smoothly connected.

The amplitude vibration conversion unit 1141b converts the input amplitude A(t) into vibration (step S31).

The waveform output section 1142b outputs a sine wave S ₂ (t) having a carrier frequency such that the amplitude becomes A(t) (step S32).

[C] Effects According to the vibration control device 1, the signal control program, and the signal control method in one example of the embodiment, for example, the following operational effects can be achieved.

The time division control unit 112 divides the signal that controls the vibration by the vibration device into predetermined time intervals. The energy control units 1131 and 1132 convert the waveform of the second signal component for each predetermined time interval divided by the time division control unit 112. The energy control units 1131 and 1132 convert the waveform of a signal in a specific frequency band by adjusting the energy, while converting the waveform of a signal other than the specific frequency band while maintaining the energy. In addition, the energy control units 1131 and 1132 may convert the waveform of a signal extracted based on a specific signal feature by adjusting the energy, while converting a signal not extracted by the specific feature into a waveform having a different frequency from the signal while maintaining the energy. This makes it possible to generate vibrations in a high-frequency band that are easily perceived by humans, and to output vibrations corresponding to any signal source by emphasizing or suppressing them. Then, it is possible to adjust the energy according to an individual's sensitivity to vibration and preferences. In addition, it is possible to suppress the generation of auditory noise caused by vibrations in a high-frequency band.

The energy control units 1131 and 1132 perform adjustments by multiplying the energy of a signal in a specific frequency band by a gain value determined according to the specific frequency band. This makes it easy to emphasize or suppress vibrations corresponding to any signal source.

The gain value may be determined based on other characteristics of the signal, such as ID, rather than just frequency. This makes it easy to emphasize or suppress vibrations corresponding to a particular signal source.

The frequency of the predetermined division period of the time division control unit 112 is an arbitrary value with a lower limit of 80 Hz. This allows the signal components in the high frequency band to be efficiently extracted.

The signal output unit 114 outputs a first signal component that does not undergo waveform conversion by the energy control units 1131 and 1132 for signals below a predetermined frequency contained in the vibration, and a second signal component that undergoes waveform conversion by the energy control units 1131 and 1132. This allows the signal components in the low-frequency band that are not subject to conversion to be output as is to the vibration device.

The predetermined frequency for separating the first and second signal components is in the range of 80 Hz to 400 Hz. This allows the waveform of the high frequency components to be appropriately converted.

For the first signal component output by the signal output unit 114, vibration is generated by the low-frequency vibration 32 of the multiple vibration devices. For the second signal component output by the signal output unit 114, vibration is generated by the high-frequency vibration 31 of the multiple vibration devices. This allows a person to realistically experience high-frequency and low-frequency band vibrations. Note that the low-frequency vibration 32 does not have to be provided in the vibration generating system 100. In that case, vibration due to a signal component below a predetermined frequency may be generated from the high-frequency vibration 31, and vibration due to a signal component below a predetermined frequency does not have to be generated in the vibration generating system 100.

[D] Others The disclosed technology is not limited to the above-described embodiments, and can be modified in various ways without departing from the spirit of the embodiments. The configurations and processes of the embodiments can be selected as necessary, or can be combined as appropriate.

The vibration generating system 100 shown in FIG. 1 is equipped with a high-frequency vibration 31 and a low-frequency vibration 32, but is not limited to this. The number of vibrations equipped in the vibration generating system 100 can be changed in various ways.

Figure 23 is a block diagram showing an example configuration of DAC2 when multiple vibration devices 310, 320 are used in the vibration generating system 100 shown in Figure 1.

In the example shown in FIG. 23, the DAC 2 shown in FIG. 1 functions as a high-frequency gain adjuster 21a, a low-frequency gain adjuster 21b, a high-frequency vibration device drive circuit 22a, and a low-frequency vibration device drive circuit 22b. Also, the high-frequency vibration 31 and the low-frequency vibration 32 shown in FIG. 1 function as a high-frequency vibration device 310 and a low-frequency vibration device 320, respectively.

The high-frequency gain adjuster 21a outputs the second vibration waveform _S2 (t) input from the vibration control device 1 to the high-frequency vibration device 310 via the high-frequency vibration device drive circuit 22a. The low-frequency gain adjuster 21b outputs the first vibration waveform _S1 (t) input from the vibration control device 1 to the low-frequency vibration device 320 via the low-frequency vibration device drive circuit 22b.

Figure 24 is a block diagram showing an example of the DAC configuration when a single vibration device is used in the vibration generating system 100 shown in Figure 1.

In the example shown in FIG. 24, the DAC 2 shown in FIG. 1 functions as the high-frequency gain adjuster 21a, the low-frequency gain adjuster 21b, and the vibration device drive circuit 22. Also, the high-frequency vibration 31 and the low-frequency vibration 32 shown in FIG. 1 function as the vibration device 30.

The high-frequency gain adjuster 21 a and the low-frequency gain adjuster 21 b output the second vibration waveform S ₂ (t) and the first vibration waveform S ₁ (t), respectively, input from the vibration control device 1 to a common vibration device 30 via a common vibration device driving circuit 22.

100: vibration generating system 1: vibration control device 11: CPU
101: ISM unit 111: Frequency elimination control unit 111a: Signal elimination unit 111b: Low-pass filter 111d: Corrected energy calculation unit 112: Time division control unit 113, 1131, 1132: Energy control unit 113a: Basis signal separation control unit 113b: Frequency calculation unit 113c: Energy correction parameter calculation unit 113d: Corrected energy calculation unit 113e: Gain calculation unit 113f: Signal source identification unit 113g: Low-frequency component synthesis unit 114: Signal output unit 114a: Energy vibration conversion unit 114b: Second vibration generation unit 114c: First vibration generation unit 1141a: Energy equivalent conversion unit 1142a: Window processing unit 1143a: Frame synthesis unit 1141b: Amplitude vibration conversion unit 1142b: Waveform output unit 12: Memory 13: Storage device 2: DAC
21a: High-frequency gain adjuster 21b: Low-frequency gain adjuster 22: Vibration device drive circuit 22a: High-frequency vibration device drive circuit 22b: Low-frequency vibration device drive circuit 30: Vibration device 31: High-frequency vibration 32: Low-frequency vibration 310: High-frequency vibration device 320: Low-frequency vibration device

Claims (12)

A vibration control device that controls vibration by a vibration device using a signal,
a time division control unit that divides the signal into predetermined time intervals;
an energy control unit that converts a waveform of the signal for each of the predetermined times divided by the time division control unit;
Equipped with
the energy control unit adjusts the energy of a signal in a specific frequency band among the frequency components of the signal to convert the waveform, while converting a signal outside the specific frequency band into a waveform having a different frequency from that of the signal while maintaining the energy of the signal;
Vibration control device. A vibration control device that controls vibrations caused by a vibration device using a signal,
a time division control unit that divides the signal into predetermined time intervals;
an energy control unit that converts a waveform of the signal for each of the predetermined times divided by the time division control unit;
Equipped with
the energy control unit adjusts the energy of a signal extracted based on a specific signal feature amount from among the signals to convert the waveform, while converting a signal not extracted based on the specific feature amount into a waveform having a different frequency from that of the signal while maintaining the energy of the signal.
Vibration control device. The energy control unit performs the adjustment by multiplying the energy of the signal in the specific frequency band by a gain value determined according to the specific frequency band.
The vibration control device according to claim 1 or 2. The gain value is determined according to an identifier defined for each of the specific frequency bands.
The vibration control device according to claim 3 . The predetermined division period of the time division control unit is a frequency having a lower limit of 80 Hz.
The vibration control device according to any one of claims 1 to 4. A vibration control device as described in any one of claims 1 to 5, further comprising a signal output unit that outputs a first signal component whose waveform is not converted by the energy control unit for a signal having a frequency below a predetermined frequency contained in the vibration, and a second signal component whose waveform is converted by the energy control unit. The vibration control device according to claim 6, wherein the predetermined frequency for separating the first signal component from the second signal component is in the range of 80 Hz to 400 Hz. With respect to the first signal component output by the signal output unit, vibration is generated by a first vibration device among the plurality of vibration devices,
With respect to the second signal component output by the signal output unit, vibration is generated by a second vibration device among the plurality of vibration devices.
The vibration control device according to claim 6 or 7. A computer controls the vibration of the vibration device by signals.
Dividing the signal into predetermined time segments;
converting a waveform of the signal for each of the divided predetermined times;
Execute the process,
The processing of converting the waveform is performed by adjusting the energy of a signal in a specific frequency band among the frequency components of the signal, while converting a signal outside the specific frequency band into a waveform having a different frequency from that of the signal while maintaining the energy of the signal outside the specific frequency band.
Vibration control program. A computer controls the vibration of the vibration device by signals.
Dividing the signal into predetermined time segments;
converting a waveform of the signal for each of the divided predetermined times;
Execute the process,
The processing of converting the waveform includes converting the waveform of a signal extracted based on a specific signal feature from among the signals by adjusting the energy, and converting the waveform of a signal not extracted based on the specific feature into a waveform having a different frequency from that of the signal while maintaining the energy of the signal.
Vibration control program. A vibration control method for controlling vibration by a vibration device using a signal, comprising:
Dividing the signal into predetermined time segments;
converting the waveform of the signal for each of the divided predetermined times;
The processing of converting the waveform is performed by adjusting the energy of a signal in a specific frequency band among the frequency components of the signal, while converting a signal outside the specific frequency band into a waveform having a different frequency from that of the signal while maintaining the energy of the signal outside the specific frequency band.
Vibration control methods. A computer controls the vibration of the vibration device by signals.
Dividing the signal into predetermined time segments;
converting a waveform of the signal for each of the divided predetermined times;
Execute the process,
The processing of converting the waveform includes converting the waveform of a signal extracted based on a specific signal feature from among the signals by adjusting the energy, and converting the waveform of a signal not extracted based on the specific feature into a waveform having a different frequency from that of the signal while maintaining the energy of the signal.
Vibration control methods.

Images