JPWO2013150648A1 - Vibration signal generating apparatus and method, computer program, recording medium, and sensory sound system - Google Patents

Abstract

The vibration signal generation device (10) includes a generation unit (11) that generates a vibration signal from the acoustic signal, and a calculation unit that calculates a vibration change point that is a point on the time axis where the temporal variation of the acoustic signal satisfies a predetermined variation condition. (12) and correction means (13) for performing correction processing for correcting the vibration signal so that the amplitude of the vibration signal immediately before the vibration change point is smaller than the amplitude of the vibration signal immediately after the vibration change point; ) Based on the characteristics of the harmonic component included in the acoustic signal at the vibration change point, it is determined whether or not to perform the correction process at the vibration change point, and (ii) the correction process is performed when it is determined to perform the correction process. And control means (14) for controlling the correction means to perform.

Description

  The present invention relates to a vibration signal generation apparatus and method, a computer program, a recording medium, and a body sensation sound system for generating a vibration signal having a relatively low frequency supplied to an electro-mechanical vibration converter employed in, for example, a body sensation sound apparatus. Related to the technical field.

  As this type of body sensation acoustic device, for example, an acoustic signal input to drive a transducer that converts an electrical signal into mechanical vibration is equalized according to switching of a mode switch (here, a cutoff frequency and an amplification). A body sensation acoustic device has been proposed (see Patent Document 1). Further, as this type of body sensation sound apparatus, a body sensation sound apparatus that emphasizes a portion (attack portion) where the rising of an acoustic signal is abrupt has been proposed (see Patent Document 2).

Japanese Patent Laid-Open No. 2004-23488 Japanese Patent No. 3579639

  However, the body sensation acoustic devices described in Patent Documents 1 and 2 have a technical problem in that depending on the input acoustic signal, it is not always possible to provide a user with a feeling of discomfort or a comfortable body sensation. For example, in the technique described in Patent Document 2, a portion where the rise of the acoustic signal is suddenly (attack portion) is uniformly emphasized, so that, for example, in a portion where the vocal sound extends in accordance with the accompaniment of the back, There is a possibility that a bodily sensation vibration that cuts off the vocal sound is provided to the user in accordance with the accompaniment of the back. Such bodily sensation vibrations that cut off vocal sounds are not always comfortable for the user.

  The present invention has been made in view of the above problems, for example, and provides a vibration signal generation apparatus and method, a computer program, a recording medium, and a body acoustic system that can provide a user with more suitable body vibration. Is an issue.

  A vibration signal generation apparatus for solving the above-described problem generates a vibration signal composed of a frequency component in a vibration frequency band that is narrower than the audible band from an acoustic signal including a frequency component in the audible band. Generating means for calculating, a calculating means for calculating a vibration change point that is a point on a time axis in which time fluctuation of the acoustic signal satisfies a predetermined fluctuation condition, and an amplitude of the vibration signal immediately before the vibration change point is the vibration change Correction means for performing correction processing for correcting the vibration signal so as to be smaller than the amplitude of the vibration signal immediately after the point; and (i) based on the characteristics of the harmonic component included in the acoustic signal at the vibration change point. Determining whether or not to perform the correction process at the vibration change point; and (ii) controlling the correction unit to perform the correction process when it is determined to perform the correction process. And a control means.

  A vibration signal generation method for solving the above problem generates a vibration signal composed of a frequency component in a vibration frequency band that is a frequency band narrower than the audible band from an acoustic signal including a frequency component in the audible band. A generating step of calculating, a calculating step of calculating a vibration change point that is a point on a time axis in which time variation of the acoustic signal satisfies a predetermined variation condition, and an amplitude of the vibration signal immediately before the vibration change point is the vibration change A correction step of performing a correction process for correcting the vibration signal so as to be smaller than the amplitude of the vibration signal immediately after the point; and (i) based on the characteristics of the harmonic component included in the acoustic signal at the vibration change point. Determining whether or not to perform the correction process at the vibration change point; and (ii) controlling the correction process to perform the correction process when it is determined to perform the correction process. And a control process.

  A computer program for solving the above-described problems causes a computer to function as the above-described vibration signal generation device.

  A recording medium for solving the above problem stores the above-described computer program.

  A body sensation sound system for solving the above problems is a body sensation sound system comprising a terminal device, a server device, and an electro-mechanical vibration conversion device connected to each other via a network. And music information indicating a list of the plurality of music data, and the terminal device receives the music information via the network, and accepting means capable of accepting user input. A display means for acquiring and displaying to the user, and a signal for identifying one piece of music data among a plurality of pieces of music data indicated by the music information in accordance with the user input received by the receiving means And a first communication means for transmitting the music specifying signal to the server device via the network, wherein the server device specifies the music specifying signal. From an acoustic signal corresponding to one piece of music data specified by the number and including a frequency component in an audible band, from a frequency component in a vibration frequency band that is a narrower frequency band than the audible band A generating means for generating a vibration signal, a calculating means for calculating a vibration change point that is a point on the time axis in which the time fluctuation of the acoustic signal satisfies a predetermined fluctuation condition, and the vibration signal immediately before the vibration change point. Correction means for performing correction processing for correcting the vibration signal so that the amplitude is smaller than the amplitude of the vibration signal immediately after the vibration change point; and (i) a harmonic component included in the acoustic signal at the vibration change point. And (ii) if it is determined to perform the correction process, the correction process is performed so that the correction process is performed. Further comprising a second communication means for transmitting the mechanical vibration converter - and control means for controlling the correction means, the vibration signal which the correction processing has been performed, via the network, the electricity.

  The effect | action and other gain of this invention are clarified from the form for implementing demonstrated below.

It is a block diagram which shows the structure of the vibration signal generation apparatus of a present Example. It is a flowchart which shows the flow of operation | movement of the vibration signal generation apparatus of a present Example. It is a graph which shows the vibration change point on an audio signal. It is a spectrum figure which shows an example of the calculation method of the change point candidate of a present Example. It is a wave form diagram which shows the aspect of the mask process of a present Example. It is a wave form diagram which shows the aspect of the vibration signal after mask processing is performed, and the vibration signal before mask processing is performed. It is a wave form diagram which shows an example of the mask used by a mask process. It is a wave form diagram which shows the modification of mask processing. For audio signals including vocals, cymbals, and pianos, masks for vibration signals and all vibration change points when mask processing is performed for some vibration change points selected according to the harmonic frequency. It is a wave form diagram which shows each of the vibration signal when a process is performed. It is a block diagram which shows the structure of the acoustic experience system of 1st Example. It is a block diagram which shows the structure of the acoustic experience system of 2nd Example. It is a block diagram which shows the structure of the acoustic experience system of 3rd Example. It is a block diagram which shows the structure of the acoustic experience system of 4th Example.

  Hereinafter, description will be made on each embodiment of the vibration signal generating apparatus and method, the computer program, the recording medium, and the body sensation sound system.

(Embodiment of vibration signal generation device)
<1>
The vibration signal generation device according to the present embodiment generates a vibration signal including a frequency component in a vibration frequency band that is a frequency band narrower than the audible band, from an acoustic signal including a frequency component in the audible band. Calculating means for calculating a vibration change point that is a point on the time axis in which the time variation of the acoustic signal satisfies a predetermined change condition; and the amplitude of the vibration signal immediately before the vibration change point is immediately after the vibration change point Correction means for performing correction processing for correcting the vibration signal to be smaller than the amplitude of the vibration signal, and (i) the vibration based on the characteristics of the harmonic component included in the acoustic signal at the vibration change point. Determining whether or not to perform the correction process at a change point; and (ii) a control unit that controls the correction unit to perform the correction process when it is determined to perform the correction process. Obtain.

  According to the vibration signal generation device of the present embodiment, the generation unit generates a vibration frequency band that is a frequency band narrower than the audible band from an acoustic signal including a frequency component within an audible band (for example, 20 Hz to 20000 Hz). A vibration signal which is a signal composed of the frequency components is generated. Note that various known methods (for example, a method using a low-pass filter disclosed in Patent Document 1 and Patent Document 2) can be applied to the generation method of the vibration signal. Omitted.

  The “vibration frequency band” is typically set as a frequency band (for example, 60 Hz to 130 Hz, etc.) that can be appropriately converted into mechanical vibration by the electro-mechanical vibration converter. Such a “vibration frequency band” is preferably set as appropriate according to the performance of the target electromechanical vibration converter.

  Subsequent to, before or after the generation of the vibration signal by the generation unit, the calculation unit calculates a vibration change point, which is a point on the time axis where the temporal variation of the acoustic signal satisfies a predetermined variation condition.

  Here, the “predetermined variation condition” is a broad meaning indicating a condition that can be arbitrarily set for time variation such as the amplitude of the acoustic signal (more specifically, the amplitude of the frequency component of the acoustic signal). . As such a “predetermined variation condition”, for example, (i) the amount of change in the amplitude of the frequency component of the acoustic signal is greater than a predetermined value, and (ii) from other vibration change points, the minimum frequency in the vibration frequency band As an example, a condition that the distance is longer than a predetermined period (for example, 0.02 second when the lowest frequency is 50 Hz) (that is, a certain period has elapsed from the previous vibration change point) is taken as an example. According to this example, the “vibration change points” are separated from each other by a time period in which the amplitude change amount of the frequency component of the acoustic signal is larger than a predetermined amount and determined according to the lowest frequency of the vibration frequency band. It means a point on the time axis. The “point where the amount of change in the amplitude of the frequency component of the acoustic signal is larger than a predetermined amount” corresponds to a so-called “sound generation position” in the acoustic signal. Accordingly, the “vibration change point” can be rephrased as a sounding position “separated from each other for a period determined according to the lowest frequency of the vibration frequency band” among the sounding positions. The “sound generation position” refers to the timing at which one sound is emitted by an instrument that emits the one sound in a music composed of a plurality of continuous sounds on the time axis.

  The calculation means may calculate such a “vibration change point” as follows, for example. The calculation means first calculates the sum of the respective powers of the frequency components constituting the acoustic signal at one time by performing a fast Fourier transform on the acoustic signal. After that, the calculating means calculates the one time on condition that the rate of change of the sum of power at one time with respect to the sum of power at another time different from the one time is larger than a predetermined amount. Candidate for vibration change point. Subsequently, the calculation means determines, as vibration change points, points that are separated from each other by a period determined according to the lowest frequency of the vibration frequency band among a plurality of vibration change point candidates.

  Thereafter, the correction unit performs a correction process on the vibration signal generated by the generation unit according to the vibration change point calculated by the calculation unit. More specifically, the correcting means determines that the amplitude of the vibration signal immediately before the vibration change point (that is, the signal component of the vibration signal immediately before the vibration change point) is the vibration immediately after the one vibration change point. The vibration signal is corrected so as to be smaller than the amplitude of the signal (that is, the signal component of the vibration signal immediately after the vibration change point). When a plurality of vibration change points (for example, the first vibration change point and the second vibration change point) are calculated, the correction unit (i) determines the amplitude of the vibration signal immediately before the first vibration change point. Is smaller than the amplitude of the vibration signal immediately after the first vibration change point, and (ii) the amplitude of the vibration signal immediately before the second vibration change point is the vibration signal immediately after the second vibration change point. The vibration signal is corrected so as to be smaller than the amplitude of.

  In the present embodiment, in particular, the correction unit performs the correction process while being controlled by the control unit. More specifically, when the control unit determines that the correction process is performed at the vibration change point, the correction unit performs the correction process at the vibration change point (that is, the vibration signal immediately before the vibration change point). Correction processing for correcting the vibration signal is performed so that the amplitude is smaller than the amplitude of the vibration signal immediately after the vibration change point). On the other hand, when the control unit determines that the correction process is not performed at the vibration change point, the correction unit does not perform the correction process at the vibration change point (that is, the amplitude of the vibration signal immediately before the vibration change point). Is not performed so as to correct the vibration signal so that it becomes smaller than the amplitude of the vibration signal immediately after the vibration change point).

  In order to determine whether or not the correction process is performed at the vibration change point, the control means includes an overtone component included in the acoustic signal at the vibration change point (that is, the signal component of the acoustic signal at the time corresponding to the vibration change point). Refer to the characteristics of. For example, the control unit may determine that the correction process is performed at the vibration change point when the characteristic of the harmonic component included in the acoustic signal at the vibration change point satisfies a predetermined characteristic. On the other hand, for example, when the characteristic of the harmonic component included in the acoustic signal at the vibration change point does not satisfy a predetermined characteristic condition, the control unit may determine not to perform the correction process at the vibration change point.

  The “harmonic component” here means a signal component having a frequency twice as high as a signal component having a predetermined reference frequency, a signal component having a frequency three times,..., K (however, k is a broad meaning meaning a signal component having a frequency that is an integer of 2 or more times. In order to identify such overtone components, the reference frequency is set in advance or appropriately set according to the performance of the target electro-mechanical vibration converter, the condition of the sensible vibration that the user wants to experience, and the like. It is preferable. For example, the frequency of the scale of the acoustic signal at the vibration change point may be set as the reference frequency, as will be described in detail later. Further, the “characteristic of the harmonic component” has a broad meaning indicating an arbitrary index that can directly or indirectly specify the harmonic component included in the acoustic signal. Examples of such “characteristics of a harmonic component” include, for example, the strength (in other words, power) of a harmonic component described later, the presence / absence of a harmonic component, and the like.

  Here, according to the inventor's research, the following matters have been found. That is, conventionally, an acoustic signal is often supplied to an electromechanical vibration converter as it is. Then, since processing such as amplification is performed on all of the supplied acoustic signals, there is a possibility that the vibration experienced by the user may be little or no inflection depending on the acoustic signals. On the other hand, an apparatus that extracts only a relatively low frequency component from an acoustic signal using, for example, a low-pass filter and supplies the extracted component to an electro-mechanical vibration converter has also been proposed. However, it is not sufficient in terms of vivid bodily sensation vibration. However, according to the vibration signal generation device of the present embodiment, the correction unit is configured so that the amplitude of the vibration signal immediately before the vibration change point is smaller than the amplitude of the vibration signal immediately after the one vibration change point. Correct the vibration signal. For this reason, the vibration signal generation device according to the present embodiment can provide the user with a relatively large change in vibration (that is, a vivid sensory vibration) before and after the vibration change point.

  In addition, according to the vibration signal generation device of the present embodiment, the correction unit targets at least some vibration change points selected according to the characteristics of the overtone component among all the vibration change points calculated by the calculation unit. As shown in FIG. Here, according to the research by the inventors of the present application, for example, the acoustic signal in which the vocal sound extends in accordance with the accompaniment of the back includes a relatively large amount of harmonic components of the vocal sound. Is known. If correction processing is performed uniformly for all the vibration change points calculated by the calculation means for such an acoustic signal, not only the sensory vibration that matches the vocal sound but also the accompaniment of the back. The user is provided with the perceived vibration. However, the sensory vibration that matches the accompaniment of the back may be provided to the user as a sensory vibration that cuts off the voice of the vocal. However, according to the vibration signal generation device of the present embodiment, the correction unit performs the correction process at some vibration change points selected according to the characteristics of the overtone component among all the vibration change points calculated by the calculation unit. (Ie, the correction process is turned on). In other words, the correction unit does not have to perform correction processing on some other vibration change points that are not selected according to the characteristics of the harmonic component among all the vibration change points calculated by the calculation unit (that is, The correction process may be turned off). Therefore, for example, the correcting means is a part of other vibration changing points that are not selected in accordance with the characteristics of the harmonic component among all the vibration changing points calculated by the calculating means (that is, vibration changing points according to the accompaniment). While the correction processing is not performed on the subject, some vibration change points selected according to the characteristics of the overtone component among all the vibration change points calculated by the calculation means (that is, vibration changes according to the vocal sound) Correction processing can be performed on the point). Therefore, the vibration signal generation device according to the present embodiment provides the user with a sense of vibrant vibration that matches the voice of the vocal, even if an acoustic signal in which the voice of the vocal extends in accordance with the accompaniment of the back is used. Can do. In other words, the vibration signal generation device of the present embodiment provides little or no bodily sensation vibration that cuts off the vocal sound to the user according to the accompaniment of the back. Needless to say, the same effect can be obtained even if an arbitrary acoustic signal is used in addition to the acoustic signal in which the vocal sound is extended in accordance with the accompaniment of the back. Therefore, the vibration signal generation device according to the present embodiment can provide the user with more suitable body vibration.

  Note that in order not to provide the user with bodily sensation vibration that cuts off the vocal sound in accordance with the accompaniment of the back, only the vocal sound is extracted from the acoustic signal and the vibration change point is based only on the extracted vocal sound. It may be sufficient to calculate. However, in order to extract only the voice of vocals, it is necessary to perform complicated and advanced processing, which is very difficult in practice. However, this embodiment is very useful in practice because correction processing can be performed based on harmonic components that can be extracted relatively easily by paying attention to the frequency.

<2>
In another aspect of the vibration signal generating device of the present embodiment, the control means corresponds to the harmonic component included in the acoustic signal at the vibration change point and the scale of the acoustic signal at the vibration change point. Whether to perform the correction processing is determined based on the characteristics of the harmonic component.

  According to this aspect, the control means can preferably determine whether or not to perform the correction process based on the characteristics of the harmonic component corresponding to the scale of the acoustic signal of the vibration signal. For example, when the scale of the acoustic signal at a certain vibration change point is G4, the control means has a harmonic component corresponding to the scale (G4) (that is, a signal component having a frequency twice that of the scale (G4). , Based on the characteristics of a signal component having a frequency three times that of the scale (G4),..., A signal component having a frequency k times that of the scale (G4) (where k is an integer of 2 or more). It is possible to preferably determine whether or not to perform the correction process. Therefore, the various effects described above are suitably realized.

<3>
In another aspect of the vibration signal generation device of the present embodiment, the correction unit performs a predetermined filtering process on the vibration signal in a period between two vibration change points that are continuous on the time axis. The correction processing is performed so that the amplitude of the vibration signal immediately before the vibration change point is smaller than the amplitude of the vibration signal immediately after the vibration change point, and the control means is continuous on the time axis. Based on the characteristics of the harmonic component included in the acoustic signal at each of two vibration change points, the filtering process is performed on the vibration signal in a period between the two vibration change points that are continuous on the time axis. Decide whether to do it.

  According to this aspect, the correction unit performs the above-described correction process relatively easily by performing the predetermined filtering process on the vibration signal in the period between two vibration change points that are continuous on the time axis. be able to. Even when such filtering processing is performed, the control means performs correction processing (that is, based on the characteristics of the harmonic components included in the respective acoustic signals at the two vibration change points that are continuous on the time axis). It is possible to suitably determine whether or not to perform a filtering process.

<4>
In another aspect of the vibration signal generation device of the present embodiment, the control means (i) when the intensity of the harmonic overtone component included in the acoustic signal at the vibration change point is equal to or greater than a predetermined threshold, (Ii) When the intensity of the harmonic component included in the acoustic signal at the vibration change point is not equal to or greater than a predetermined threshold value, it is determined to perform the correction process.

  According to this aspect, for example, when an acoustic signal in which the vocal sound is extended in accordance with the accompaniment of the back is taken as an example, for example, the correcting means is the harmonic component of all the vibration change points calculated by the calculating means. Some vibration change points (ie, vocals) whose intensity (eg, power) is equal to or greater than a predetermined threshold (ie, the intensity of the harmonic component is relatively high or the harmonic component is relatively large) Correction processing can be performed at a vibration change point matched to the sound). In other words, the correcting means has the intensity of the harmonic component that is not greater than or equal to the predetermined threshold value among all the vibration change points calculated by the calculating means (that is, the intensity of the harmonic component is relatively low or the harmonic component is relatively large). Correction processing is not performed at some other vibration change points that are not included (or not included at all) (that is, vibration change points that match the accompaniment). Therefore, as described above, the vibration signal generation device of the present embodiment can provide the user with more suitable body vibration.

<5>
In another aspect of the vibration signal generating device of the present embodiment, the correcting means attenuates the amplitude of the vibration signal immediately before the vibration change point, thereby reducing the amplitude of the vibration signal immediately before the vibration change point. The correction processing for correcting the vibration signal so as to be smaller than the amplitude of the vibration signal immediately after the vibration change point is performed.

  According to this aspect, it is possible to provide the user with the above-described preferable body vibration relatively easily.

<6>
In another aspect of the vibration signal generation device of the present embodiment, the correction means amplifies the amplitude of the vibration signal immediately after the vibration change point, so that the amplitude of the vibration signal immediately before the vibration change point is increased. The correction processing for correcting the vibration signal so as to be smaller than the amplitude of the vibration signal immediately after the vibration change point is performed.

  According to this aspect, it is possible to provide the user with the above-described preferable body vibration relatively easily.

(Embodiment of vibration signal generation method)
<7>
The vibration signal generation method of the present embodiment generates a vibration signal composed of a frequency component in a vibration frequency band, which is a frequency band narrower than the audible band, from an acoustic signal including a frequency component in the audible band. A calculation step of calculating a vibration change point that is a point on the time axis in which the time variation of the acoustic signal satisfies a predetermined variation condition, and the amplitude of the vibration signal immediately before the vibration change point is immediately after the vibration change point A correction process for correcting the vibration signal so as to be smaller than the amplitude of the vibration signal, and (i) the vibration based on the characteristics of the harmonic component included in the acoustic signal at the vibration change point. Determining whether or not to perform the correction process at a change point, and (ii) a control process for controlling the correction process to perform the correction process when it is determined to perform the correction process. Obtain.

  According to the vibration signal generation method of the present embodiment, various effects that can be enjoyed by the above-described vibration signal generation device of the present embodiment can be suitably enjoyed.

  Incidentally, in response to the various aspects adopted by the vibration signal generation apparatus of the present embodiment described above, the vibration signal generation method of the present embodiment can also adopt various aspects.

(Embodiment of computer program)
<8>
The computer program according to the present embodiment causes the computer to function as the vibration signal generating device according to the present embodiment described above (including various aspects thereof).

  According to the computer program of the present embodiment, various effects that can be enjoyed by the vibration signal generation device of the present embodiment described above can be suitably enjoyed.

  Incidentally, in response to the various aspects adopted by the vibration signal generation apparatus of the present embodiment described above, the computer program of the present embodiment can also adopt various aspects.

(Embodiment of recording medium)
<9>
The recording medium of the present embodiment stores the above-described computer program of the present embodiment (including various aspects thereof).

  According to the recording medium of the present embodiment, various effects that can be enjoyed by the vibration signal generation device of the present embodiment described above can be suitably enjoyed.

  Incidentally, in response to the various aspects adopted by the vibration signal generating apparatus of the present embodiment described above, the recording medium of the present embodiment can also adopt various aspects.

(Embodiment of bodily sensation acoustic system)
<10>
The body sensation vibration system of this embodiment is a body sensation sound system including a terminal device, a server device, and an electromechanical vibration conversion device connected to each other via a network, and the server device includes a plurality of music data. And storing means for storing music information indicating a list of the plurality of music data, wherein the terminal device receives the music information via the network, receiving means capable of receiving user input, and The music which is a signal for specifying one piece of music data among the plurality of pieces of music data indicated by the music information in response to the input of the user received by the receiving means and the display means displayed to the user First communication means for transmitting a specific signal to the server device via the network, wherein the server device is characterized by the music specific signal. A vibration signal comprising a frequency component within a vibration frequency band that is a frequency band narrower than the audible band, from an acoustic signal corresponding to a piece of music data and including a frequency component within the audible band Generating means for calculating a vibration change point that is a point on the time axis in which the time variation of the acoustic signal satisfies a predetermined fluctuation condition, and the amplitude of the vibration signal immediately before the vibration change point is Correction means for performing correction processing for correcting the vibration signal so as to be smaller than the amplitude of the vibration signal immediately after the vibration change point; and (i) a characteristic of the harmonic component included in the acoustic signal at the vibration change point. And (ii) determining whether or not to perform the correction process at the vibration change point, and (ii) performing the correction process when the correction process is determined to be performed. And Gosuru control means, the vibration signal which the correction processing has been performed, via the network, the electro - further comprising a second communication means for transmitting the mechanical vibration converter.

  According to the body sensation sound system of the present invention, the terminal device, the server device, and the electro-mechanical vibration converter are connected to each other via a network such as the Internet or a LAN (Local Area Network). The terminal device includes an accepting unit and a first communication unit. The server device includes a storage unit, a generation unit, a correction unit, a control unit, and a second communication unit. For this reason, according to the body sensation sound system of this embodiment, the various effects which the vibration signal generation apparatus of this embodiment mentioned above can enjoy can be enjoyed suitably.

  Incidentally, in response to the various aspects employed by the vibration signal generation device of the present embodiment described above, the body sensation acoustic system of the present embodiment can also employ various aspects.

  Such an operation and other advantages of the present embodiment will be clarified from examples described below.

  As described above, the vibration signal generation device of this embodiment includes a generation unit, a calculation unit, a correction unit, and a control unit. The vibration signal generation method of this embodiment includes a generation step, a calculation step, a correction step, and a control step. The computer program of this embodiment causes a computer to function as the vibration signal generation device of this embodiment. The recording medium of this embodiment stores the computer program of this embodiment. The body sensation sound system of this embodiment includes a server device including a generation unit, a calculation unit, a correction unit, and a control unit, a terminal device, and an electro-mechanical vibration conversion device. Accordingly, it is possible to provide the user with a more suitable body vibration.

  Hereinafter, an example is described based on a drawing.

(1) Embodiment of Vibration Signal Generation Device First, the vibration signal generation device 10 of this embodiment will be described with reference to FIGS. 1 to 9.

(1-1) Configuration of Vibration Signal Generation Device First, the configuration of the vibration signal generation device 10 of the present embodiment will be described with reference to FIG. FIG. 1 is a block diagram illustrating a configuration of a vibration signal generation device 10 according to the present embodiment. In FIG. 1, for convenience of explanation, only members that are directly related to the present invention are shown, and other members are not shown.

  As shown in FIG. 1, the vibration signal generation device 10 of the present embodiment includes a vibration signal generation unit 11, a vibration change point calculation unit 12, a vibration signal strength correction unit 13, a vibration signal strength control unit 14, and a signal. And an input unit 15. The vibration signal generation unit 11, the vibration change point calculation unit 12, the vibration signal strength correction unit 13, and the vibration signal strength control unit 14 may be physically configured by hardware such as an electronic circuit or on the CPU. It may be logically realized by operating software. When the vibration signal generation unit 11, the vibration change point calculation unit 12, the vibration signal strength correction unit 13, and the vibration signal strength control unit 14 are logically realized by software operating on the CPU, the software (that is, the computer) The computer can function as the vibration signal generation device 10 by loading a recording medium storing a program) into the computer or by downloading the software to the computer via a communication line or the like.

  The signal input unit 15 includes music data stored in a recording medium (not shown) (for example, a flash memory, a hard disk drive, and an optical disk), music data input via a microphone (not shown), Accepts input of music data etc. acquired via the network. As a result, an audio signal corresponding to the music data is input to the vibration signal generation device 10.

  The vibration signal generation unit 11 is a specific example of “generation means”, and from an audio signal including a frequency component of an audible band, from a frequency component in a vibration frequency band that is a frequency band narrower than the audible band. A vibration signal that is a signal is generated. Note that the vibration signal generation unit 11 is typically a low-pass filter that passes only signal components in the vibration frequency band (in other words, cuts signal components in other frequency bands other than the vibration frequency band).

  The vibration change point calculation unit 12 is a specific example of “calculation means” and calculates a vibration change point of an audio signal corresponding to music data input via the signal input unit 15.

  The vibration signal strength correction unit 13 is a specific example of “correction means”, and the vibration change point calculation unit 12 calculates the amplitude of the vibration signal immediately before the vibration change point calculated by the vibration change point calculation unit 12. The vibration signal generated by the vibration signal generation unit 11 is corrected so as to be smaller than the amplitude of the vibration signal immediately after the vibration change point.

  The vibration signal strength control unit 14 is a specific example of “control means”, and (i) whether to perform correction processing at the vibration change point based on the characteristics of the harmonic component included in the acoustic signal at the vibration change point. And (ii) controls the vibration signal strength correction unit 13 to perform the correction process when it is determined to perform the correction process.

  The specific operations of the vibration signal generation unit 11, the vibration change point calculation unit 12, the vibration signal strength correction unit 13, and the vibration signal strength control unit 14 will be described in detail below (see FIGS. 2 to 9). For the sake of description, detailed description here is omitted.

(1-2) Operation of Vibration Signal Generation Device Next, the operation of the vibration signal generation device 10 of this embodiment will be described with reference to FIGS. 2 and 3. FIG. 2 is a flowchart showing an operation flow of the vibration signal generation device 10 of the present embodiment. FIG. 3 is a graph showing vibration change points on the audio signal.

  As shown in FIG. 2, the vibration signal generation unit 11 includes frequency components within the vibration frequency band from an audio signal (see FIG. 3A) corresponding to music data input via the signal input unit 15. A vibration signal (see FIG. 3B) is generated (step S101).

  For example, the vibration frequency band may be automatically set by the vibration signal generation unit 11. Alternatively, the vibration frequency band may be manually set by the user of the vibration signal generation device 10. However, the vibration frequency band is preferably a frequency band that can be appropriately converted into mechanical vibration by an electro-mechanical vibration converter 40 (see FIGS. 10 to 13) described later. Therefore, the vibration frequency band is preferably set as appropriate according to the performance of the target electro-mechanical vibration converter 40. An example of such a vibration frequency band is a frequency band in the range of 60 Hz to 130 Hz.

  Note that the amplitude of the vibration signal generated by the vibration signal generation unit 11 is determined according to the amplitude of the audio signal input via the signal input unit 15. For example, the amplitude of the vibration signal generated by the vibration signal generation unit 11 is proportional to or similar to the amplitude of the audio signal input via the signal input 15.

  Following the processing of step S101 described above, before or after or in parallel, the vibration change point calculation unit 12 transmits an audio signal corresponding to the music data input via the signal input unit 15 (FIG. 3A). Change point candidates (see the broken line in FIG. 3C) are calculated (step S201). At this time, it is preferable that the vibration change point calculation unit 12 calculates a plurality of change point candidates. Alternatively, the vibration change point calculation unit 12 may calculate a single change point candidate depending on conditions defined by a calculation method described later. Hereinafter, for convenience of explanation, the description will be given using an example in which the vibration change point calculation unit 12 calculates a plurality of change point candidates. The vibration change point calculation unit 12 calculates the “change point candidate” as a point on the time axis (that is, “time”).

  Here, a method for calculating a change point candidate of an audio signal will be described with reference to FIG. FIG. 4 is a spectrum diagram illustrating an example of a method for calculating change point candidates according to this embodiment.

  First, the vibration change point calculation unit 12 performs, for example, fast Fourier transform (FFT) on the audio signal. As a result, the vibration change point calculation unit 12 can calculate the FFT power of each of a plurality of frequency components constituting the audio signal at a certain time (here, time t) as shown in FIG. .

  Subsequently, the vibration change point calculation unit 12 calculates the sum of the FFT powers of the plurality of frequency components constituting the audio signal. At this time, the vibration change point calculation unit 12 includes a part of the frequency components included in the extracted frequency band that is a specific frequency band among the plurality of frequency components included in the audio signal (for example, the frequency f2 from the frequency f2 indicated by shading). After extracting f3), it is preferable to calculate the sum of the FFT powers of the extracted partial frequency components.

  At this time, the extraction frequency band that defines some frequency components extracted by the vibration change point calculation unit 12 may be automatically set by the vibration signal calculation unit 12. Alternatively, the extraction frequency band may be manually set by the user of the vibration signal generation device 10. However, it is preferable that the extracted frequency band includes a frequency component corresponding to a part or a musical instrument for which intermittent bodily sensation vibration is not to be provided by mask processing corresponding to a vibration change point described later. For example, you do not want to provide bodily sensation vibration according to the accompaniment of the back where the vocal sound is stretched (that is, the bodily sensation vibration that cuts off the vocal sound where the vocal sound is stretched) In the case of not wanting to provide according to the performance of the back), a band from 200 Hz to 800 Hz corresponding to vocal voice may be set as the extraction frequency band. Or, for example, you don't want to provide vibrations in response to the chorus of the back where the trumpet is playing (that is, the trumpet is cut off where the trumpet is playing). When it is not desired to provide vibration according to the chorus of the back), a band from 200 Hz to 1 kHz corresponding to the performance of the trumpet may be set as the extraction frequency band.

  Thereafter, the vibration change point calculation unit 12 calculates an increase rate of the sum P (t, f) of the FFT power at a certain time t. Specifically, the vibration change point calculation unit 12 uses the sum of FFT powers P (t−1, f) at the previous time t−1 as a reference, the sum P (t of FFT powers at a certain time t. , F) (for example, P (t, f) / P (t−1, f)) is calculated.

  Thereafter, the vibration change point calculation unit 12 calculates a change point candidate of the audio signal based on the calculated increase rate of the sum of FFT powers. Specifically, the vibration change point calculation unit 12 calculates, for example, a time t that satisfies the condition of P (t, f) / P (t−1, f)> N (t) as a change point candidate. N (t) is a degree of change (in other words, a threshold value) calculated from the sum P (t, f) of the FFT power at time t. The vibration change point calculation unit 12 calculates a plurality of change point candidates by continuously performing the processing described above.

  The vibration change point calculation unit 12 adds or instead of the rate of increase of the sum of the FFT powers, at a certain time t based on the amplitude A (t−1) of the audio signal at the previous time t−1. The rate of increase of the amplitude A (t) of the audio signal at (i.e., A (t) / A (t-1)) may be calculated. In this case, the vibration change point calculation unit 12 may calculate, for example, a time t that satisfies the condition of A (t) / A (t−1)> N ′ (t) as a change point candidate. N ′ (t) is the degree of change (in other words, a threshold value) calculated from the amplitude A (t) at time t.

  Returning to FIG. 2 again, the vibration change point calculation unit 12 selects a vibration change point (see a broken line in FIG. 3D) such that the mutual interval is equal to or greater than a specific interval from the calculated plurality of change point candidates. Extract (step S202). Here, the “specific interval” is set according to the lower limit value of the above-described vibration frequency band, for example. For example, when the lower limit value of the vibration frequency band is x (where x is a real number greater than 0) Hz, the specific interval may be set to 1 / x second. Specifically, for example, when the lower limit value of the vibration frequency band is 50 Hz, the specific interval may be set to 1/50 = 0.02 seconds.

  At this time, it is preferable that the vibration change point calculation unit 12 extracts a plurality of vibration change points. Alternatively, the vibration change point calculation unit 12 may extract only one vibration change point. In the following, for convenience of explanation, the description will be given using an example in which the vibration change point calculation unit 12 extracts a plurality of vibration change points.

  Subsequently, the vibration signal strength control unit 14 extracts each scale of the plurality of vibration change points calculated in step S202 (step S401). Specifically, the vibration signal strength control unit 14 extracts the scale of the acoustic signal at time t corresponding to the vibration change point (that is, the scale that appears in the acoustic signal at time t corresponding to the vibration change point). At this time, the vibration signal strength control unit 14 may extract a scale in which the FFT power is equal to or higher than a predetermined power (for example, an average value of FFT powers of all frequency components constituting the audio signal). That is, the vibration signal strength control unit 14 does not have to extract a scale in which the FFT power is less than the predetermined power.

  Thereafter, the vibration signal strength control unit 14 creates a power map of the scale extracted in step S401 for each vibration change point (step S402). Specifically, for example, the vibration signal strength control unit 14 performs the scale extracted in step S401 and overtones of the scale (that is, the second overtone, the third overtone, ..., k (where k is an integer of 2 or more)). A power map is generated in which each FFT power is distinguished for each frequency component.

  Thereafter, the vibration signal strength control unit 14 calculates the harmonic frequency for each vibration change point based on the scale power map created in step S402 (step S403). Note that the overtone frequency is the degree to which the overtone is included in the acoustic signal at time t corresponding to the vibration change point (in other words, the strength or intensity of the overtone contained in the acoustic signal at time t corresponding to the vibration change point). ) Is an arbitrary index.

  In step S403, the vibration signal strength control unit 14 calculates the harmonic frequency according to whether or not the second harmonic, the third harmonic, and the fourth harmonic are included in the acoustic signal at time t corresponding to the vibration change point. May be. For example, the vibration signal strength control unit 14 sets “2” when the second harmonic is included in the acoustic signal at the time t corresponding to the vibration change point and neither the third harmonic nor the fourth harmonic is included. May be calculated. For example, the vibration signal strength control unit 14 includes a harmonic frequency of “3” when the acoustic signal at time t corresponding to the vibration change point includes the second harmonic and the third harmonic and does not include the fourth harmonic. May be calculated. For example, the vibration signal strength control unit 14 calculates the harmonic frequency of “4” when the sound signal at the time t corresponding to the vibration change point includes any of the second harmonic, the third harmonic, and the fourth harmonic. May be.

  Alternatively, in step S403, the vibration signal strength control unit 14 responds to the FFT power of the scale of the acoustic signal at time t corresponding to the vibration change point and the FFT power of at least one of the second harmonic, the third harmonic, and the fourth harmonic of the scale. Thus, the harmonic frequency may be calculated. The “scale FFT power” referred to here means the sum of the FFT powers of the frequency bands constituting the scale. For example, if the scale of the acoustic signal at time t corresponding to the vibration change point is G5, the FFT power of the scale G5 is the FFT power in the frequency band (807.3 Hz to 855.3 Hz) corresponding to the scale G5. Become sum.

  For example, the FFT power of the scale of the acoustic signal at the time t corresponding to the vibration change point is P1, and the FFT power of the second harmonic of the scale of the acoustic signal at the time t corresponding to the vibration change point is P2, corresponding to the vibration change point. Assuming that the FFT power of the third harmonic of the scale of the acoustic signal at time t is P3, the vibration signal strength control unit 14 is derived from the formula (P2 × 1.2) / P1 + (P3 × 1.5) / P1. A harmonic frequency may be calculated. However, the vibration signal strength control unit 14 may calculate the harmonic frequency of “0” when the conditions of P2 / P1> 0.8 and the condition of P3 / P1> 0.6 are not satisfied. Good.

  Alternatively, for example, assuming that the FFT power of the fourth harmonic of the scale of the acoustic signal at time t corresponding to the vibration change point is P4, the vibration signal strength control unit 14 (P2 × 1.2) / P1 + (P3 × 1. 5) The harmonic frequency derived from the mathematical formula of /P1+(P4×1.9)/P1 may be calculated. However, the vibration signal strength control unit 14 determines that “P2 / P1> 0.7, P3 / P1> 0.5, and P4 / P1> 0.4” are not satisfied. A harmonic frequency of “0” may be calculated.

  In the above description, attention is focused on three types of overtones: second overtone, third overtone, and fourth overtone. However, the vibration signal strength control unit 14 may calculate the harmonic frequency based on the harmonics of the fifth harmonic or higher in addition to or instead of the second harmonic, the third harmonic, and the fourth harmonic. Alternatively, the vibration signal strength control unit 14 may calculate the harmonic frequency based on the 1 / k harmonic in addition to or instead of the second harmonic, the third harmonic,. That is, the vibration signal strength control unit 14 may calculate the harmonic frequency based on an arbitrary harmonic.

  In addition, in the above description, the vibration signal strength control unit 140 increases the degree that harmonics are included in the acoustic signal at the time t corresponding to the vibration change point (that is, the harmonic signal includes a relatively large number of harmonics). The higher the harmonic frequency is calculated, the higher the harmonic frequency is, or the higher the harmonic power is included. However, the vibration signal strength control unit 14 increases the degree that harmonics are included in the acoustic signal at the time t corresponding to the vibration change point (that is, the more harmonics are included). Alternatively, it is possible to calculate a harmonic overtone frequency that becomes smaller (as the harmonics having a relatively strong FFT power are included). In this case, in step S404, which will be described later, it is preferable that the vibration signal strength control unit 14 determines whether or not the harmonic frequency is equal to or less than a predetermined threshold value.

  In addition, the above-mentioned harmonic frequency is merely an example, and it goes without saying that the harmonic frequency calculated in other modes may be used.

  Thereafter, the vibration signal strength control unit 14 determines whether or not the harmonic frequency of each of the two vibration change points that are in succession (that is, continuous) on the time axis is equal to or greater than a predetermined threshold (step S404). At this time, the “predetermined threshold value” includes whether or not a relatively large number of overtones are included in each of the acoustic signals at two vibration change points that follow each other on the time axis, or overtones with relatively strong FFT power. It is preferable that a value that can suitably determine whether or not it is included is set.

  For example, when the harmonic frequency calculated according to whether or not each of the second harmonic, the third harmonic, and the fourth harmonic is included in the acoustic signal at time t corresponding to the vibration change point, the vibration signal strength control is used. The unit 14 may determine whether the harmonic frequency is 4 or more. The determination in this case corresponds to a determination as to whether or not all of the second harmonic, the third harmonic and the fourth harmonic are included in each of the two vibration change points which are in succession on the time axis. The predetermined threshold value “4” is merely an example. For example, a predetermined threshold value “2” may be used, a predetermined threshold value “3” may be used, and other predetermined threshold values may be used. May be used.

  Alternatively, for example, when the harmonic frequency calculated according to the scale of the acoustic signal at time t corresponding to the vibration change point and the FFT power of the harmonic of the scale is used, the vibration signal strength control unit 14 It may be determined whether or not is greater than 1. The determination in this case corresponds to a determination as to whether or not at least one of a second harmonic, a third harmonic, and a fourth harmonic having a relatively strong FFT power is included in each of two successive vibration change points on the time axis. To do. The predetermined threshold “1” is merely an example, and other predetermined thresholds may be used, for example.

  As a result of the determination in step S404, if it is determined that each harmonic overtone frequency at two vibration change points that are in succession on the time axis is greater than or equal to a predetermined threshold (step S404: Yes), It is assumed that each of the acoustic signals at the two vibration change points includes a relatively large number of overtones or a harmonic that has a relatively strong FFT power. In this case, the vibration signal strength control unit 14 of the vibration signal strength correction unit 13 does not perform mask processing described later (that is, turns off the mask processing) in a period between the two vibration change points. The operation is controlled (step S405).

  On the other hand, as a result of the determination in step S404, if it is determined that the harmonic overtone frequency of at least one of the two vibration change points on the time axis is not greater than or equal to the predetermined threshold value (step S404: No), It is assumed that the acoustic signals at the two adjacent vibration change points do not contain a relatively large number of overtones or do not contain overtones with a relatively strong FFT power. In this case, the vibration signal strength control unit 14 does not perform the operation of step S405. That is, the vibration signal strength control unit 14 controls the operation of the vibration signal strength correction unit 13 so as to perform mask processing (that is, to turn on mask processing) to be described later in the period between the two vibration change points. .

  Thereafter, the vibration signal strength correction unit 13 masks the vibration signal generated in step S101 so that the amplitude of the vibration signal immediately before the vibration change point is smaller than the amplitude of the vibration signal immediately after the vibration change point. Processing is performed (step S301). As a result, the vibration signal strength correction unit 13 corrects the vibration signal generated in step S101 (step S301). At this time, the vibration signal strength correction unit 13 corrects the vibration signal under the control of the vibration signal strength control unit 14. That is, the vibration signal strength correction unit 13 selectively performs mask processing on a vibration signal in a period between two vibration change points determined to perform (i) mask processing, while (ii) mask Mask processing is not performed on a vibration signal in a period between two vibration change points determined not to be processed.

  Here, the mask processing will be described with reference to FIGS. FIG. 5 is a waveform diagram showing an aspect of the mask processing of this embodiment. FIG. 6 is a waveform diagram showing the vibration signal before the mask process is performed and the vibration signal after the mask process is performed. FIG. 7 is a waveform diagram showing an example of a mask used in the mask process. FIG. 8 is a waveform diagram showing a modification of the mask process.

  The vibration signal strength correction unit 13 applies, for example, a cosine mask shown in FIGS. 5C and 5D to the vibration signal generated by the vibration signal generation unit 11 (see FIG. 5A). The used mask processing is performed. As a result, as shown in FIG. 5B, the vibration signal strength correction unit 13 determines the average amplitude of the vibration signal immediately before the vibration change point (see circle a1) as the average of the vibration signal immediately after the vibration change point. It can be made lower than the amplitude (see circle a2).

  In particular, in the present embodiment, as shown in FIG. 5C, mask processing is performed under the control of the vibration signal strength control unit 14. For this reason, the vibration signal strength correction unit 13 performs a vibration signal in a period between two vibration change points (vibration change point B and vibration change point C in FIG. 5C) determined not to perform the mask process. Is not masked. On the other hand, as shown in FIG. 5D, in the comparative example in which the mask process is performed without being controlled by the vibration signal strength control unit 14, the mask process is performed for all vibration change points.

  By such a mask process, for example, the vibration signal shown in FIG. 6A is corrected as shown in FIG. 6B, for example. FIG. 6A is an example of a vibration signal before mask processing is performed, and FIG. 6B is an example of a vibration signal after mask processing is performed.

  In the above description, an example in which a cosine type mask shown in FIG. 7A is used as a mask used in the mask process is used. Such a cosine mask may be used when, for example, a ballad audio signal is input to the vibration signal generation device 10 via the signal input unit 15. However, the mask used in the mask process is not limited to a cosine type mask. For example, a linear mask shown in FIGS. 7B and 7C may be used. Alternatively, a staircase type mask shown in FIG. 7D may be used. Alternatively, a bowl-shaped mask or the like shown in FIG. 7E may be used. Such a bowl-shaped mask may be used, for example, when a dance audio signal is input to the vibration signal generation device 10 via the signal input unit 15. Needless to say, any of the masks can make the amplitude of the vibration signal immediately before the vibration change point smaller than the amplitude of the vibration signal immediately after the vibration change point.

  Further, as shown in FIG. 8A, the terminal portion of the mask is more than the rear vibration change point (vibration change point B in FIG. 8A) of the two adjacent vibration change points. It may be located forward. In this case, when the mask process is performed on the vibration signal shown in FIG. 8B, the vibration signal shown in FIG. 8C is obtained.

  According to the vibration signal generation device 10 of the present embodiment described above, the following technical effects are realized.

  First, according to the inventor's research, the following matters have been found. That is, conventionally, an audio signal is often supplied to the electromechanical vibration converter 40 as it is. Then, the electromechanical vibration converter 40 performs processing such as amplification on all of the supplied audio signals. For this reason, depending on the audio signal, there is a possibility that the bodily sensation vibration provided to the user has little or no inflection. On the other hand, there has also been proposed a vibration signal generation device that extracts only a relatively low frequency component from an audio signal using, for example, a low-pass filter and supplies the extracted audio signal to the electromechanical vibration converter 40. However, it is not sufficient in terms of vivid bodily sensation vibration.

  However, according to the vibration signal generation device 10 of the present embodiment, the vibration signal strength correction unit 13 has the amplitude of the vibration signal immediately before the vibration change point smaller than the amplitude of the vibration signal immediately after the vibration change point. As described above, the vibration signal can be corrected. For this reason, the vibration signal generation device 10 of the present embodiment can provide the user with a relatively large change in vibration (that is, a vivid sensory vibration) before and after the vibration change point.

  In addition, according to the vibration signal generation device 10 of the present embodiment, the vibration signal strength correction unit 13 is at least one selected according to the harmonic frequency among all the vibration change points calculated by the vibration change point calculation unit 12. Mask processing can be performed on the vibration change point of the part. Here, according to the research by the inventors of the present application, for example, an audio signal in which the vocal sound is extended in accordance with the accompaniment of the back includes a relatively large number of harmonics of the vocal sound. It turns out. If all the vibration change points calculated by the vibration change point calculation unit 12 from such an audio signal are uniformly masked, not only the bodily sensation vibration matched to the vocal sound but also the back accompaniment. Is provided to the user. However, the sensory vibration that matches the accompaniment of the back may be provided to the user as a sensory vibration that cuts off the voice of the vocal. That is, vibrations that do not match the vocal voice growth may be provided to the user.

  However, according to the vibration signal generation device 10 of the present embodiment, the vibration signal strength correction unit 13 selects a part of all vibration change points calculated by the vibration change point calculation unit 12 according to the harmonic frequency. Mask processing can be performed on the vibration change point. In other words, the vibration signal strength correction unit 13 performs mask processing on other vibration change points that are not selected according to the harmonic frequency among all the vibration change points calculated by the vibration change point calculation unit 12. Not performed. Therefore, for example, the vibration signal strength correction unit 13 selects other vibration change points (that is, accompaniment) that are not selected according to the harmonic frequency among all the vibration change points calculated by the vibration change point calculation unit 12. While the mask process is not performed on the combined vibration change points), some vibration change points selected according to the harmonic frequency out of all the vibration change points calculated by the vibration change point calculation unit 12 (that is, The mask process can be performed on the vibration change point in accordance with the voice of the vocal). As a result, the user is provided with body vibration that matches the vocal voice growth (in other words, there is no sense of incongruity).

  Here, referring to FIG. 9, the mask processing is performed on the vibration signal and all vibration change points when mask processing is performed on some vibration change points selected according to the harmonic frequency. The vibration signal in the case of breakage will be described. FIG. 9 shows a vibration signal and all vibration change points when mask processing is performed on a part of vibration change points selected according to the harmonic frequency for audio signals including vocals, cymbals, and pianos. It is a wave form diagram which shows each of a vibration signal when a mask process is performed for object.

  As shown in FIG. 9, the vibration change point A is a vibration change point calculated in accordance with the vocal sounding position (that is, the beat of the vocal). The vibration change point B is a vibration change point calculated according to the cymbal sounding position (that is, the cymbal beat). The vibration change point C is a vibration change point calculated according to the sound generation position of the piano (that is, the beat of the piano). It is assumed that the vocal sound is continued while the cymbal performance and the piano performance are being performed. That is, the audio signal shown in FIG. 9 corresponds to a musical piece in which vocal sound is extended while accompaniment of a cymbal and a piano accompanies the back. Note that, as described above, the audio signal in which the vocal sound is extended contains a relatively large number of harmonics of the vocal sound. Accordingly, it is assumed that the harmonic frequency is equal to or higher than the predetermined threshold at any of the vibration change points A to C.

  In this case, as shown in the lower waveform diagram of FIG. 9, when the mask process is performed on all vibration change points, the vibration in the period between the vibration change points A and B is changed. Mask processing is performed on each of the signal and the vibration signal in the period between the vibration change point B and the vibration change point C. As a result, the user is provided with not only the body vibration that matches the vocal sound but also the body vibration that matches the back accompaniment. A user who is provided with such a body vibration may receive an impression that the body vibration according to the vocal voice is cut off or hindered by the body vibration according to the accompaniment of the back.

  On the other hand, as shown in the upper waveform diagram of FIG. 9, when mask processing is performed on a part of vibration change points selected according to the harmonic frequency, the vibration change point A and the vibration change point Mask processing is not performed on the vibration signal during the period between B and the vibration signal during the period between the vibration change point B and the vibration change point C. As a result, the user is provided with a body vibration that matches the voice of the vocal, while the user is not provided with a body vibration that matches the accompaniment of the back. A user who is provided with such a body vibration can suitably experience a body vibration having a sharpness in accordance with the growth of vocal voice.

  Thus, the vibration signal generation device 10 according to the present embodiment can provide the user with more suitable body vibration.

(2) Example of body sensation sound system Hereinafter, an example when the vibration signal generation device 10 of this embodiment is applied to a body sensation sound system will be described with reference to FIGS. 10 to 13.

(2-1) First Example of Sensory Sound System First, the sensory sound system 100 of the first example will be described with reference to FIG. FIG. 10 is a block diagram illustrating a configuration of the acoustic experience system according to the first example. In addition, the arrow in a figure has shown the flow of a signal (same in subsequent figures).

  As shown in FIG. 10, the body sensation sound system 100 according to the first embodiment includes a vibration signal generation device 10 and an electro-mechanical vibration converter 40.

  The signal input unit 15 transmits an audio signal supplied from the outside to the vibration signal generation unit 11, the vibration change point calculation unit 12, and the vibration signal strength control 14. The vibration change point calculation unit 12 calculates a vibration change point of the audio signal and transmits a signal indicating the calculated vibration change point to the vibration signal strength correction unit 13. In addition, the vibration change point calculation unit 12 transmits a signal indicating the calculated vibration change point to the vibration signal strength control unit 14. The vibration signal strength control unit 14 determines whether or not to perform mask processing on a vibration signal in a period between two adjacent vibration change points, and a signal indicating the determined result is corrected for vibration signal strength. To the unit 13.

  The vibration signal strength correction unit 13 performs predetermined processing on the vibration signal generated by the vibration signal generation unit 11 based on the received signal indicating the vibration change point and the signal indicating the determination result of whether or not to perform mask processing. Perform mask processing. Thereafter, the vibration signal strength correction unit 13 transmits a vibration signal subjected to mask processing to the electro-mechanical vibration converter 40. As a result, the electro-mechanical vibration converter 40 converts the vibration signal into mechanical vibration, thereby providing the user with a sense of body vibration.

(2-2) Second Example of the Sensory Sound System Next, a sensory sound system 200 of the second example will be described with reference to FIG. FIG. 11 is a block diagram showing the configuration of the acoustic experience system 200 of the second embodiment. In addition, below, the description which overlaps with the body sensation sound system 100 of 1st Example is abbreviate | omitted, and the same code | symbol is attached | subjected and shown to a common location on drawing, and only a different point is demonstrated.

  As shown in FIG. 11, the bodily sensation sound system 200 according to the second embodiment includes a vibration signal generation device 10, a user interface unit 20, a memory 30, an electromechanical vibration converter 40, and a delay circuit (Delay) 50. And an output terminal (Audio OUT) 60.

  The user interface unit 20 includes a display unit (not shown) that provides a user with, for example, a music selection screen and a mode selection screen, and buttons (not shown) for the user to perform an input operation.

  On the “music selection screen”, a list of one or a plurality of music data stored in advance in the memory 30 such as a flash memory is displayed.

  On the “mode selection screen”, names corresponding to a plurality of frequency bands (here, “vocal tracking”, “base tracking”, “frequency band designation”) are displayed. Note that the frequency band is set as a fixed value (so-called preset value) in advance by the manufacturer or the like for “vocal tracking” and “base tracking”. On the other hand, the “frequency band designation” allows the user to freely set the frequency band.

  When a user selects one piece of music data from the list of music data displayed on the music selection screen, a signal indicating the selected one piece of music data is transmitted from the user interface unit 20 to the memory 30. . Then, one piece of music data is transmitted from the memory 30 to the signal input unit 15 of the vibration signal generation device 10.

  On the other hand, when one name among the names corresponding to the plurality of frequency bands displayed on the mode selection screen is selected by the user, a signal indicating the frequency band corresponding to the selected one name is generated as a vibration signal. It is transmitted to the vibration change point calculation unit 12 of the device 10. The frequency band corresponding to the selected name corresponds to the “extracted frequency band” described above.

  In the vibration signal generation device 10, the signal input unit 15 transmits one piece of music data to the vibration signal generation unit 11, the vibration change point calculation unit 12, and the vibration signal strength control unit 14. The signal input unit 15 further transmits one piece of music data to the output terminal 60 via the delay circuit 50.

  The vibration change point calculation unit 12 that has received one piece of music data calculates a vibration change point based on the received one piece of music data and a signal indicating the extracted frequency band. That is, the vibration change point calculation unit 12 extracts a frequency component in the extracted frequency band from the frequency components included in one piece of music data, and changes the vibration based on the FFT power and the like related to the extracted frequency component. Calculate points. The vibration change point calculation unit 12 transmits a piece of music data and a signal indicating the calculated vibration change point to the vibration signal strength correction unit 13. In addition, the vibration change point calculation unit 12 transmits a signal indicating the calculated vibration change point to the vibration signal strength control unit 14.

  The vibration signal strength control unit 14 that has received one piece of music data determines whether or not to perform mask processing on a vibration signal in a period between two adjacent vibration change points, and shows the determined result. The signal is transmitted to the vibration signal strength correction unit 13.

  The vibration signal generation unit 11 that has received one piece of music data generates a vibration signal from the received one piece of music data. The vibration signal generation unit 11 generates and transmits a vibration signal to the vibration signal strength correction unit 13.

  The vibration signal strength correction unit 13 performs predetermined processing on the vibration signal generated by the vibration signal generation unit 11 based on the received signal indicating the vibration change point and the signal indicating the determination result of whether or not to perform mask processing. Mask processing is performed. Thereafter, the vibration signal strength correction unit 13 transmits a vibration signal subjected to mask processing to the electromechanical vibration converter 40. As a result, the electro-mechanical vibration converter 40 converts the vibration signal into mechanical vibration, thereby providing the user with a sense of body vibration.

  Here, since one piece of music data output from the output terminal 60 is delayed by a period caused by the delay circuit 50, the reproduction position of the one piece of music data and the vibration generated by the electro-mechanical vibration converter 40 are used. And synchronize. For this reason, the body sensation sound system 200 according to the second embodiment can provide a user with a sharp body sensation vibration while maintaining harmony with the music corresponding to one piece of music data.

(2-3) Third Example of the Sensory Sound System Next, a sensory sound system 300 of the second example will be described with reference to FIG. FIG. 12 is a block diagram illustrating a configuration of an acoustic experience system 300 according to the third embodiment. In the following description, the description overlapping the bodily sensation acoustic system 200 of the second embodiment from the bodily sensation acoustic system 100 of the first embodiment is omitted, and the same reference numerals are given to the common parts on the drawings, and the basics are shown. Only the differences will be described.

  As shown in FIG. 12, in the sensory sound system 300 of the third embodiment, the electro-mechanical vibration converter 40 has a sensory sound as a constituent element of the vibration unit 310 compared to the sensory sound system 200 of the second embodiment. It is different in that it is separated from the system 300.

  In the bodily sensation sound system 300 according to the third embodiment, the vibration signal masked by the vibration signal strength correction unit 13 is transmitted via a network (not shown) (for example, a wired network or a wireless network) by the operation of the communication unit 70. To the vibration unit 310. The vibration unit 310 includes a communication unit 311 that receives a vibration signal transmitted from the sensory acoustic system 300, an amplifier 312 that amplifies the received vibration signal, and an electro-mechanical vibration that converts the amplified vibration signal into mechanical vibration. And a converter 40. For this reason, the body sensation sound system 300 according to the third embodiment, in the same way as the body sensation sound system 200 according to the second embodiment, maintains the harmoniousness with the music corresponding to one piece of music data and produces a sharp body sensation vibration. Can be provided.

(2-4) Fourth Example of the Sensory Sound System Next, a sensory sound system 400 of the fourth example will be described with reference to FIG. FIG. 13 is a block diagram illustrating a configuration of the body sensation sound system 400 according to the fourth embodiment. In addition, while omitting the description overlapping the bodily sensation acoustic system 300 of the third embodiment from the bodily sensation acoustic system 100 of the first embodiment, common portions in the drawings are denoted by the same reference numerals, and are basically different. Only will be described.

  As shown in FIG. 13, the body sensory sound system 400 of the fourth embodiment includes a terminal device 410, a server device 420, and a vibration unit 430 that are connected to each other via a network 440.

  The user interface unit 20 of the terminal device 410 acquires music information indicating a list of one or more music data stored in the memory 30 of the server device 420 via the communication unit 412, the network 440, and the communication unit 422. . The user interface unit 20 displays the acquired music information to the user.

  When the user interface unit 20 receives an input indicating one piece of music data among the pieces of music data indicated by the piece of music information, the user interface unit 20 sends a music specifying signal for specifying one piece of music data to the server device 420. The data is transmitted via the network 440 and the communication unit 422.

  The control unit 421 of the server device 420 transmits one piece of music data specified by the received music specifying signal to the reproduction unit 411 of the terminal device 410 via the communication unit 422, the network 440, and the communication unit 412. . The control unit 421 further transmits one piece of music data to the vibration signal generation device 10.

  The control unit 421 sends the vibration signal subjected to the mask processing output from the vibration signal generation device 10 to the electro-mechanical vibration conversion unit 40 of the vibration unit 430, the communication unit 422, the network 440, and the communication unit 431. Send through. As a result, the electro-mechanical vibration converter 40 included in the vibration unit 430 converts the vibration signal into mechanical vibration, thereby providing the user with a vibrant body vibration.

  The present invention is not limited to the above-described embodiments, and can be appropriately changed without departing from the gist or concept of the invention that can be read from the claims and the entire specification. In addition, the computer program and the sensory sound system are also included in the technical scope of the present invention.

DESCRIPTION OF SYMBOLS 10 Vibration signal production | generation apparatus 11 Vibration signal production | generation part 12 Vibration change point calculation part 13 Vibration signal strength correction part 14 Vibration signal strength control part 15 Signal input part 20 User interface part 30 Memory 40 Electromechanical vibration converter 50 Delay circuit (Delay) )
60 Output terminal (Audio OUT)
DESCRIPTION OF SYMBOLS 70 Radio | wireless part 100 Body sensation sound system 200 Body sensation sound system 300 Body sensation sound system 310 Vibration unit 311 Communication part 312 Amplifier 400 Body sensation sound system 401 Terminal device 411 Playback part 412 Communication part 420 Server apparatus 421 Control part 422 Communication part 430 Vibration unit 431 Communication Department 440 Network

Claims (10)

Generating means for generating a vibration signal comprising a frequency component in a vibration frequency band, which is a frequency band narrower than the audible band, from an acoustic signal including a frequency component in the audible band;
Calculating means for calculating a vibration change point, which is a point on the time axis in which time fluctuation of the acoustic signal satisfies a predetermined fluctuation condition;
Correction means for performing correction processing for correcting the vibration signal so that the amplitude of the vibration signal immediately before the vibration change point is smaller than the amplitude of the vibration signal immediately after the vibration change point;
(I) Based on the characteristics of the harmonic component included in the acoustic signal at the vibration change point, it is determined whether or not to perform the correction process at the vibration change point, and (ii) determined to perform the correction process. And a control means for controlling the correction means so as to perform the correction processing when it is performed.   The control means performs the correction process based on characteristics of the harmonic component that is included in the acoustic signal at the vibration change point and that corresponds to the scale of the acoustic signal at the vibration change point. The vibration signal generation device according to claim 1, wherein whether or not the vibration signal is generated is determined. The correction means performs a predetermined filtering process on the vibration signal in a period between two vibration change points that are continuous on the time axis, so that the amplitude of the vibration signal immediately before the vibration change point is increased. Performing the correction process so as to be smaller than the amplitude of the vibration signal immediately after the vibration change point,
The control means, between the two vibration change points that are continuous on the time axis, based on the characteristics of the harmonic component included in the acoustic signal of each of the two vibration change points that are continuous on the time axis. The vibration signal generation device according to claim 1, wherein whether to perform the filtering process on the vibration signal in a period is determined.   The control means determines (i) that the correction processing is not performed when the intensity of the harmonic component included in the acoustic signal at the vibration change point is equal to or greater than a predetermined threshold, and (ii) the vibration change point The vibration signal generation device according to claim 1, wherein when the intensity of the harmonic component included in the acoustic signal is not equal to or greater than a predetermined threshold, the correction process is determined to be performed.   The correction means attenuates the amplitude of the vibration signal immediately before the vibration change point so that the amplitude of the vibration signal immediately before the vibration change point is larger than the amplitude of the vibration signal immediately after the vibration change point. The vibration signal generation device according to claim 1, wherein the correction processing for correcting the vibration signal to be small is performed.   The correction means amplifies the amplitude of the vibration signal immediately after the vibration change point, so that the amplitude of the vibration signal immediately before the vibration change point is larger than the amplitude of the vibration signal immediately after the vibration change point. The vibration signal generation device according to claim 1, wherein the correction processing for correcting the vibration signal to be small is performed. Generating a vibration signal composed of a frequency component in a vibration frequency band, which is a frequency band narrower than the audible band, from an acoustic signal including a frequency component in the audible band;
A calculation step of calculating a vibration change point that is a point on the time axis in which the time variation of the acoustic signal satisfies a predetermined variation condition;
A correction step of performing correction processing for correcting the vibration signal so that the amplitude of the vibration signal immediately before the vibration change point is smaller than the amplitude of the vibration signal immediately after the vibration change point;
(I) Based on the characteristics of the harmonic component included in the acoustic signal at the vibration change point, it is determined whether or not to perform the correction process at the vibration change point, and (ii) determined to perform the correction process. And a control step of controlling the correction step so as to perform the correction processing when it is performed.   A computer program for causing a computer to function as the vibration signal generation device according to claim 1.   A recording medium storing the computer program according to claim 8. A bodily sensation acoustic system comprising a terminal device, a server device, and an electro-mechanical vibration conversion device connected to each other via a network,
The server device has storage means for storing a plurality of music data and music information indicating a list of the plurality of music data,
The terminal device
Accepting means capable of accepting user input;
Display means for acquiring the music information via the network and displaying it to the user;
In response to the user input received by the receiving means, a music specifying signal, which is a signal for specifying one piece of music data among a plurality of pieces of music data indicated by the music information, is sent to the server via the network. First communication means for transmitting to the device,
The server device
From an acoustic signal corresponding to one piece of music data specified by the music specifying signal and including a frequency component within the audible band, within a vibration frequency band that is a frequency band narrower than the audible band Generating means for generating a vibration signal composed of frequency components;
Calculating means for calculating a vibration change point, which is a point on the time axis in which time fluctuation of the acoustic signal satisfies a predetermined fluctuation condition;
Correction means for performing correction processing for correcting the vibration signal so that the amplitude of the vibration signal immediately before the vibration change point is smaller than the amplitude of the vibration signal immediately after the vibration change point;
(I) Based on the characteristics of the harmonic component included in the acoustic signal at the vibration change point, it is determined whether or not to perform the correction process at the vibration change point, and (ii) determined to perform the correction process. Control means for controlling the correction means so as to perform the correction processing when
The sensory acoustic system, further comprising: a second communication unit that transmits the vibration signal subjected to the correction process to the electromechanical vibration converter via the network.

Images