JP5082760B2 - Sound control apparatus and program - Google Patents

Description

  The present invention relates to a technique for controlling sound according to voice input.

Conventionally, a technique for generating a sound corresponding to a phoneme of an input voice has been proposed. For example, Patent Literature 1 discloses a technique for outputting a rhythm sound corresponding to a phoneme identified by speech recognition for an input speech. That is, a voice pattern that correlates with the input voice is specified by voice recognition among a plurality of voice patterns registered in advance, and a rhythm sound corresponding to the voice pattern is output.
Japanese Patent Laid-Open No. 9-281968

  However, in the technique of Patent Document 1, speech recognition for input speech is essential. Therefore, there is a problem that a large-capacity storage device is required to store the voice pattern registered in advance by the user, and the processing load of voice recognition by the arithmetic processing device becomes excessive. In view of the above circumstances, an object of the present invention is to generate a sound according to the phoneme of an input voice without requiring voice recognition.

Using the relationship that the energy distribution (frequency spectrum) of the components of each band in the input speech differs depending on the phoneme, the sound control device according to the present invention uses the phoneme index value that changes according to the phoneme of the input speech. Is calculated based on the intensity of a component in a specific band of the input voice , sound selection means for selecting one of a plurality of sounds based on the phoneme index value, and a peak for detecting the peak value of the input voice Detecting means; threshold setting means for variably setting a threshold according to phoneme index value; pronunciation determining means for determining whether or not the peak value exceeds the threshold; and pronunciation determination means determining if the peak value exceeds the threshold In this case, there is provided data generation means for generating sound data indicating the generation of the sound selected by the sound selection means.

In the above configuration, since the phoneme index value that is the index of the phoneme of the input speech is calculated based on the strength of a specific component of the input speech, the speech recognition of the input speech is not necessary in principle. Therefore, there is an advantage that a large-capacity storage device for storing the voice pattern is not necessary, and the processing load for discriminating phonemes is reduced. In addition, since the threshold value to be compared with the peak value is variably set for determining whether or not the selection sound can be generated by the sound selection means, the frequency of pronunciation according to the peak value of the input speech is set regardless of the type of phoneme. It is possible to make it uniform. For example, the threshold value setting means variably controls the threshold value according to the phoneme index value so that the phoneme whose peak value tends to be lower is lowered. However, if the phoneme index value indicates a specific phoneme, if the threshold value is lowered, the frequency (probability) of the sound corresponding to the phoneme is increased compared with other phonemes (the pronunciation frequency is increased). A configuration (which is non-uniform depending on the type) is also realized. The format of the sound data is arbitrary. For example, data including sound designation (note number) (MIDI data) and data indicating a time waveform of sound (waveform data) are suitable as sound data. Further, in a specific band used for calculating the phoneme index value in the input speech, the difference in the phoneme index value according to the phoneme of the input speech becomes remarkable (that is, the phoneme in the energy distribution on the frequency axis). (Including a band in which the corresponding feature appears prominently).

  In a preferred aspect of the present invention, the index calculating means includes a filter processing means for extracting a component in a specific band from the input speech, and a first intensity detecting means for detecting the intensity of the component processed by the filter processing means (for example, FIG. 1 intensity detecting unit 144), second intensity detecting means for detecting the intensity of the input voice (for example, intensity detecting unit 146 in FIG. 1), the intensity detected by the first intensity detecting means and the second intensity detecting means Calculating means for calculating a phonological index value based on a relative ratio to the measured intensity. According to the above aspect, since the phonological index value is calculated based on the relative ratio between the intensity of the component selectively extracted from the input voice and the intensity of the input voice, regardless of the difference in the intensity of the input voice. It is possible to calculate a phoneme index value that changes appropriately according to phonemes. The calculation of the phoneme index value based on the intensity relative ratio includes a process of calculating the phoneme index value from a function including the intensity relative ratio as a variable in addition to the process of calculating the intensity relative ratio as the phoneme index value.

  In a preferred aspect of the present invention, the index calculation means calculates a phoneme index value for each of a plurality of components belonging to separate bands of the input speech, and the sound selection means selects a sound based on the plurality of phoneme index values. To do. According to the above aspect, the selection candidates (sound types) according to the phoneme index values are diversified as compared with the configuration in which one phoneme index value is calculated from the components belonging to one band of the input speech. It is possible.

  The sound control device according to a preferred aspect of the present invention includes a corresponding sound setting unit that variably sets a relationship between a phoneme index value and a sound, and the sound selection unit is an index according to the relationship set by the corresponding sound setting unit. The sound corresponding to the phoneme index value calculated by the calculation means is selected. According to the above aspect, since the relationship between the phoneme index value and the sound is variably set, for example, a user's desired sound can be reproduced according to the phoneme.

In a preferred aspect of the present invention, the threshold value setting means variably sets each of the first threshold value for determination of pronunciation and the second threshold value for determination of mute according to the phoneme index value, It is determined whether or not the peak value detected by the peak detection means exceeds the first threshold value, and whether or not the peak value detected by the peak detection means is lower than the second threshold value. When the sound generation determination unit determines that the detected peak value exceeds the first threshold value, sound data indicating the generation of the sound selected by the sound selection unit is generated, and the peak value detected by the peak detection unit sets the second threshold value. If the sound generation determining means determines that the sound is below, sound data indicating the mute of the sound is generated .

The sound control device according to a preferred aspect of the present invention includes a corresponding volume setting unit that variably sets a relationship between the intensity of the input sound (for example, the volume or peak value of the input sound) and the sound volume indicated by the sound data, Volume determination means for determining the volume corresponding to the peak value detected by the peak detection means in the relationship set by the volume setting means, and the data generation means is sound data indicating the sound of the volume set by the volume determination means Is generated. According to the above aspect, since the relationship between the intensity of the input sound and the volume of the sound indicated by the sound data is variably set, for example, the aspect of sufficiently ensuring the volume of the reproduced sound even when the volume of the input sound is low Alternatively, a mode in which the volume of the reproduced sound is suppressed even when the volume of the input sound is high is appropriately employed.

The sound control device according to the present invention is realized by hardware (electronic circuit) such as a DSP (Digital Signal Processor) dedicated to each process, and a general-purpose arithmetic processing device such as a CPU (Central Processing Unit) and a program It is also realized through collaboration with. The program according to the present invention includes an index calculation process (for example, step S3 in FIG. 5) for calculating a phoneme index value that changes according to the phoneme of the input speech based on the intensity of a component of a specific band of the input speech, A sound selection process (for example, step S5 in FIG. 5) for selecting one of the sounds based on the phoneme index value, a peak detection process for detecting the peak value of the input voice, and a threshold value variably set according to the phoneme index value The generation of the sound selected in the sound selection process when the sound generation determination process determines that the peak value exceeds the threshold value, and the sound generation determination process determines that the peak value exceeds the threshold value. The computer is caused to execute data generation processing (for example, step S11 in FIG. 5) for generating sound data. Even with the above program, the same operations and effects as the sound control apparatus according to the present invention are exhibited. The program of the present invention is provided to the user in a form stored in a computer-readable recording medium and installed in the computer, or is provided in a form distributed via a communication network and installed in the computer. The

<A: First Embodiment>
FIG. 1 is a block diagram showing the configuration of the sound control apparatus according to the first embodiment of the present invention. The sound control device 100 is a device that generates a percussion instrument performance sound corresponding to the phoneme of an onomatopoeia uttered by a user (for example, a sound such as “don” or “pan” that simulates a percussion instrument performance sound). For example, if the user utters the onomatopoeia “Don”, the performance sound of the bass drum is played, and if the user utters the onomatopoeia “Pan”, the performance sound of the hi-hat cymbal is played. It is.

  As shown in FIG. 1, the sound control device 100 is realized by a computer system including a control device 10 and a storage device 40. The control device 10 is an arithmetic processing device that executes various processes by executing a program. The storage device 40 stores a program executed by the control device 10 and various data used by the control device 10.

  An input device 50, an A / D converter 62, and a sound source circuit 72 are connected to the control device 10. The input device 50 includes a plurality of operators that are operated by a user. The user inputs various instructions to the sound control device 100 by appropriately operating the input device 50. A sound collection device 64 is connected to the A / D converter 62. The sound collection device 64 collects a voice (hereinafter referred to as “input voice”) V uttered by the user. The A / D converter 62 generates a digital audio signal SV representing a time waveform of the input audio V picked up by the sound pickup device 64.

  The control device 10 functions and functions as each element shown in FIG. 1 to generate and output sound data DS indicating a percussion instrument performance sound corresponding to the input sound V (audio signal SV). The sound data DS is digital data in a format compliant with the MIDI (Musical Instrument Digital Interface) standard. The tone generator 72 (MIDI tone generator) generates a data string indicating the waveform of the percussion instrument performance sound based on the sound data DS. The data string output from the sound source circuit 72 is converted into an analog sound signal by the D / A converter 74. The sound emitting device 76 amplifies the sound signal output from the D / A converter 74 and radiates a sound wave corresponding to the amplified sound signal.

  Next, a functional configuration of the control device 10 will be described. 1 divides the audio signal SV (input audio V) into a plurality of frames (for example, sections of about 1 millisecond) on the time axis. The audio signal SV of each frame is supplied to the index calculation unit 14 and the peak detection unit 16.

  The index calculation unit 14 generates a phoneme index value A for the audio signal SV of each frame. The phoneme index value A is a numerical value that changes according to the phoneme (phoneme) of the input speech V. That is, the input speech V having a sufficiently different phoneme index value A is discriminated as a separate phoneme.

  FIG. 2 is a graph showing an outline of the frequency spectrum Q of the uttered sound for each phoneme type. Part (A) of FIG. 2 is the frequency spectrum Q of the bilateral sound (/ b /, / p /), and part (B) of FIG. 2 is the frequency spectrum Q of the gum sound (/ t /, / d /). In FIG. 2, part (C) is the frequency spectrum Q of the soft palate sound (/ k /, / g /). As shown in each part of FIG. 2, the frequency spectrum Q of the uttered sound is different for each phoneme depending on the principle of the utterance and the combination with the subsequent vowel. For example, the frequency spectrum Q (part (A)) of both lip sounds is distributed such that the intensity decreases as the frequency increases, whereas the frequency spectrum Q (part (B)) of the gum sound decreases as the frequency decreases. To be distributed. In addition, the frequency spectrum Q (part (C)) of the soft palate sound has a maximum intensity in the middle range and decreases in the low and high ranges.

  As described above, using the phenomenon that the frequency spectrum Q of the input voice V is different depending on the phoneme, the index calculation unit 14 in FIG. 1 performs a specific frequency band (hereinafter referred to as “discrimination band”) in the input voice V. The phoneme index value A is calculated on the basis of the intensity of the component. As shown in FIG. 1, the index calculation unit 14 of this embodiment includes a filter processing unit 142, an intensity detection unit 144, an intensity detection unit 146, and a calculation unit 148.

  The filter processing unit 142 selectively extracts the component VC in the discrimination band from the audio signal SV. For example, a low-pass filter whose cutoff frequency is the upper limit frequency of the discrimination band and a band-pass filter whose pass band is the discrimination band are suitably employed as the filter processing unit 142. The discrimination band is selected statistically or experimentally so that the difference in the distribution of the frequency spectrum Q is remarkable in the band among the plurality of phonemes to be distinguished by the phoneme index value A. In this embodiment, the bilateral sound of part (A) in FIG. 2 (for example, onomatopoeia such as “bang” and “bread”) is distinguished from the gum sound (for example, onomatopoeia such as “tan” and “don”) in part (B) of FIG. It is assumed for the sake of convenience. In the band BL (low band) lower than the frequency fc1 shown in the part (A) and the part (B) of FIG. 2, the difference in the frequency spectrum Q between the bilateral sound and the gum sound becomes remarkable. Accordingly, the band BL is set in the filter processing unit 142 as a discrimination band.

  The intensity detector 144 in FIG. 1 detects the intensity (power) PC of the component VC extracted by the filter processor 142 for each frame. For example, the intensity PC is obtained by dividing the square root of the sum of the squares of the amplitude values of the samples in the frame indicating the waveform of the component VC (each sample of the filtered audio signal SV) by the total number of samples in the frame. It is a numerical value. On the other hand, the intensity detector 146 detects the intensity (power) P0 of the input voice V that has not been processed by the filter processor 142 for each frame. The intensity P0 is calculated by the same method as the intensity PC.

  The calculation unit 148 calculates the relative ratio of the intensity PC to the intensity P0 as the phoneme index value A (A = PC / P0). As understood from part (A) of FIG. 2, since the intensity PC in the discrimination band (band BL) is high for the bilateral sound, the phoneme index value A is large when the phoneme of the input voice V is the bilateral sound. It becomes a numerical value. On the other hand, as shown in part (B) of FIG. 2, the intensity PC in the discrimination band (band BL) is low for gum sounds, so that the phoneme index value A is a small numerical value when the phoneme of the input voice V is a gum sound. It becomes. Therefore, it is possible to roughly discriminate the phoneme of the input speech V according to the magnitude of the phoneme index value A calculated by the calculation unit 148.

The sound selection unit 22 in FIG. 1 selects any one of a plurality of percussion instrument performance sounds based on the phoneme index value A. A code (hereinafter referred to as “note number”) Nn designating the performance sound selected by the sound selector 22 is output from the sound selector 22 to the data generator 30. To specify the note number Nn corresponding to the phoneme index value A, a table (referred to as “sound selection table”) TBL stored in the storage device 40 is used.

  FIG. 3 is a schematic diagram showing the contents of the sound selection table TBL. As shown in the figure, the sound selection table TBL is a table in which a note number (percussion instrument type) Nn is associated with each of a plurality of ranges of numerical values of the phoneme index value A. For example, note number Nn1 for designating hi-hat cymbals is associated with range a1 of phoneme index value A corresponding to both lip sounds, and note number for designating bass drums in range a2 of phoneme index value A corresponding to gum sounds. Nn2 is associated. The sound selection unit 22 searches the sound selection table TBL for the range to which the phoneme index value A calculated by the calculation unit 148 belongs, and acquires the note number Nn corresponding to the range from the storage device 40.

  The corresponding sound setting unit 23 in FIG. 1 variably controls the relationship between the phoneme index value A and the note number Nn in the sound selection table TBL. For example, the corresponding sound setting unit 23 stores each range of the phoneme index value A in the sound selection table TBL in association with the note number Nn corresponding to the type of percussion instrument designated by the user by operating the input device 50. Store in device 40. Therefore, the user can appropriately change the performance sound of the percussion instrument that is output when each phoneme is uttered.

  The peak detector 16 detects the peak intensity (hereinafter referred to as “peak value”) PK of the input voice V on the time axis for each frame. A known technique is arbitrarily adopted for detection of the peak value PK. For example, a configuration in which an envelope of the time waveform of the input voice V is specified and the amplitude of the peak in the frame in the envelope is detected as the peak value PK is suitable.

  The sound generation determination unit 24 determines the time of sound generation and mute according to the magnitude of the peak value PK detected by the peak detection unit 16. More specifically, the sound generation determination unit 24 instructs the data generation unit 30 to generate sound in a frame in which the peak value PK exceeds the threshold value TON, and the data generation unit 30 in a frame in which the peak value PK is lower than the threshold value TOFF. To mute.

  By the way, the magnitude of the peak value PK tends to depend on the phoneme of the input voice V. That is, there are phonemes in which the peak value PK is likely to increase and phonemes in which the peak value PK is difficult to increase. Therefore, in the configuration in which the threshold value TON (threshold value TOFF) is a fixed value regardless of the phoneme of the input speech V, for example, the probability that the pronunciation determination unit 24 determines the time of pronunciation as the phoneme whose peak value PK is difficult to increase decreases. Therefore, inconsistency that the frequency of pronunciation of the performance sound differs depending on the phoneme occurs.

  Therefore, the threshold value setting unit 25 in FIG. 1 variably sets the threshold value TON and the threshold value TOFF according to the phoneme of the input voice V. For the recognition of phonemes by the threshold setting unit 25, the phoneme index value A calculated by the index calculation unit 14 is used. That is, the threshold value setting unit 25 decreases the threshold value TON and the threshold value TOFF when the phoneme index value A indicates a phoneme in which the peak value PK is difficult to increase, compared to the case of a phoneme in which the peak value PK is likely to increase. . According to the above configuration, there is an advantage that the frequency of pronunciation of the performance sound corresponding to each phoneme is made uniform for a plurality of phonemes.

  The volume determination unit 26 in FIG. 1 determines the volume of the performance sound according to the peak value PK detected by the peak detection unit 16. A numerical value (hereinafter referred to as “velocity”) VEL designating the volume determined by the volume determination unit 26 is output to the data generation unit 30. The corresponding volume setting unit 27 variably sets the relationship between the peak value PK and the velocity VEL as described below.

  The storage device 40 stores a plurality of functions F (hereinafter referred to as “volume function”) that define the relationship between the peak value PK and the velocity VEL. FIG. 4 is a conceptual diagram showing the contents of each volume function F (F1 to F3). As shown in FIG. 4, the change in velocity VEL with respect to the peak value PK is different for each volume function F. For example, the volume function F1 defines the relationship between the peak value PK and the velocity VEL so that the slope decreases when the peak value PK exceeds the numerical value p1, whereas the volume function F2 has the peak value PK having the numerical value p2. The relationship between the peak value PK and the velocity VEL is defined so that the slope increases when the value is exceeded. The volume function F3 defines the velocity VEL so as to increase linearly with respect to the peak value PK. The corresponding volume setting unit 27 selects the volume function F designated by the user by operating the input device 50 from the storage device 40. The volume determination unit 26 calculates the velocity VEL by substituting the peak value PK into the volume function F selected by the corresponding volume setting unit 27. Therefore, the user can appropriately change the change mode (volume function F) of the velocity VEL with respect to the volume of the input voice V. For example, when the user selects the volume function F1 in FIG. 4, a performance sound with a sufficient volume (velocity VEL) is generated even when the volume of the utterance is low, and the user selects the volume function F2. In this case, the volume of the performance sound is suppressed even when the volume of the utterance is high.

  The data generation unit 30 generates sound data DS corresponding to the results of operations performed by the sound selection unit 22, the sound generation determination unit 24, and the volume determination unit 26. Specifically, the data generation unit 30 generates sound data DS (note-on event) for instructing sound generation and outputs the sound data to the sound source circuit 72 in response to an instruction for sound generation by the sound generation determination unit 24. The sound data DS instructing pronunciation includes the note number Nn designated by the sound selection unit 22 and the velocity VEL designated by the volume determination unit 26. By outputting the above sound data DS to the sound source circuit 72, the performance sound of the percussion instrument corresponding to the phoneme of the input voice V is output from the sound emitting device 76 at a volume corresponding to the peak value PK of the input voice V. Is done. On the other hand, when the sound generation determination unit 24 is instructed to mute, the data generation unit 30 outputs sound data DS (note-off event in which zero is specified as the velocity VEL) that instructs the mute of the performance sound corresponding to the note number Nn. It is generated and output to the sound source circuit 72.

  Next, the overall flow of processing executed by the control device 10 will be described with reference to FIG. The process of FIG. 5 is started when the user gives the input device 50 an operation for instructing activation of the program. When the processing of FIG. 5 is started, the dividing unit 12 cuts out one frame from the audio signal SV supplied from the A / D converter 62 (step S1). Next, the detection of the intensity PC by the filter processing unit 142 and the intensity detection unit 144, the detection of the intensity P0 by the intensity detection unit 146, and the detection of the peak value PK by the peak detection unit 16 are sequentially performed (step S2). Further, the calculation unit 148 calculates a phoneme index value A from the intensity P0 and the intensity PC (step S3).

  Next, the control device 10 updates various variables according to the operation on the input device 50 (step S4). More specifically, the corresponding sound setting unit 23 updates the contents of the sound selection table TBL (correspondence between each range of the phoneme index value A and the note number Nn) according to an operation on the input device 50, and a corresponding volume setting unit. 27 selects one of a plurality of volume functions F stored in the storage device 40 in accordance with an operation on the input device 50. Further, the control device 10 sets the numerical value TH1 and numerical value TH2 that are candidates for the threshold value TON and the numerical value TL1 and numerical value TL2 that are candidates for the threshold value TOFF in accordance with an operation on the input device 50 (step S4).

  Next, the sound selection unit 22 specifies the note number Nn corresponding to the phoneme index value A calculated in step S3 from the sound selection table TBL (step S5). The threshold setting unit 25 sets the threshold TON and the threshold TOFF according to the phoneme index value A calculated in step S3 (step S6). That is, for example, when the phonological index value A is in the range a1 corresponding to both lip sounds, the numerical value TH1 is set to the threshold value TON and the numerical value TL1 is set to the threshold value TOFF, and the phonological index value A is in the gum sound range a2. In such a case, the numerical value TH2 is set to the threshold value TON and the numerical value TL2 is set to the threshold value TOFF.

  Next, the control device 10 determines whether or not the status flag SF indicates mute and the peak value PK detected in step S2 exceeds the threshold value TON set in step S6 (step S7). The status flag SF is a code for identifying whether the current state is a sounding state or a mute state.

  When the result of step S7 is affirmative (that is, when the current frame corresponds to the starting point of sound generation), the volume determination unit 26 sets the peak value PK detected at step S2 to the volume function F selected at step S4. By substituting, the velocity VEL is calculated (step S8). On the other hand, when the result of step S7 is negative (that is, when the past pronunciation continues or when the peak value PK does not reach the threshold value TON), the control device 10 indicates that the status flag SF indicates the pronunciation, and It is determined whether or not the peak value PK detected in step S2 is lower than the threshold value TOFF set in step S6 (step S9).

  When the result of step S9 is affirmative (that is, when the current frame corresponds to the end point of sound generation), the volume determination unit 26 sets the velocity VEL to zero (step S10). On the other hand, when the result of step S9 is negative (that is, when there is no change from one of the sound generation and mute to the other in the current frame), the control device 10 moves the process to step S1 and outputs the audio signal SV. Similar processing is performed for the next frame.

  When step S8 or step S10 is completed, the data generation unit 30 generates sound data DS according to the result of the process relating to the current frame (step S11). That is, when the status flag SF indicates mute (when sounding is changed in the current frame), the data generation unit 30 includes a note-on event including the note number Nn set in step S5 and the velocity VEL set in step S8. Is generated as sound data DS and output to the sound source circuit 72. Therefore, a percussion instrument performance sound corresponding to the phoneme uttered by the user is output from the sound emitting device 76. On the other hand, when the status flag SF indicates sounding (when the state flag SF is changed to mute), the data generating unit 30 includes a note-off event including the note number Nn in step S5 and the velocity VEL set to zero in step S10. Is generated as sound data DS and output to the sound source circuit 72.

  Next, the control device 10 inverts the state flag SF from one of sound generation and mute to the other (step S12), and determines whether or not it is time to end the reproduction of the performance sound (step S13). The user can instruct the control device 10 to end the reproduction by appropriately operating the input device 50. When the result of step S13 is negative (for example, when the end of reproduction has not been instructed yet), the control device 10 shifts the process to step S1 and executes the same process for the next frame of the audio signal SV. . On the other hand, when the result of step S13 is affirmative, the control device 10 ends the process of FIG.

  As described above, in this embodiment, the phoneme index value A, which is an index for distinguishing the phoneme of the input speech V, is calculated based on the intensity PC of the component VC in the discrimination band (band BL) of the input speech V. Therefore, speech recognition of the input speech V is not necessary in principle. Therefore, it is possible to reduce the capacity required for the storage device 40 and the processing load by the control device 10 as compared with the technique of Patent Document 1.

  Even though the intensity PC of the component VC differs depending on the phoneme, for example, in the configuration in which the intensity PC itself of the component VC is adopted as the phoneme index value A, it depends on the volume (intensity P0) of the input voice V. Therefore, there is a possibility that the phoneme cannot be properly distinguished from the phoneme index value A alone. In this embodiment, since the phoneme index value A is calculated based on the relative ratio between the intensity PC of the component VC and the overall intensity P0 of the input voice V, each phoneme is obtained regardless of the magnitude P0 of the input voice V. There is an advantage that a phoneme index value A that can be appropriately classified is calculated. As can be understood from the above description, for example, after the audio signal SV is normalized (standardized) by the intensity for each frame so that the maximum value of the amplitude becomes a predetermined value (for example, 1), the index calculation unit 14 In the supply configuration, the intensity PC itself detected by the intensity detection unit 144 may be used as the phoneme index value A.

  Note that when a percussion instrument performance sound (for example, a cymbal performance sound) whose performance sound continues in time is assumed to be played back, a note-off event is necessary, but a percussion instrument performance sound that does not continue in time In the case where only the performance sound (that is, the performance sound such as a bass drum or hi-hat cymbal that occurs only instantaneously) is assumed to be reproduced, the note-off event is unnecessary. Therefore, a configuration in which the data generation unit 30 generates only the note-on event as the sound data DS is also employed. In a configuration using a note-off event, the velocity of the note-off event may be designated as a numerical value other than zero.

<B: Second Embodiment>
Next, a second embodiment of the present invention will be described. In addition, about the element which an effect | action and function are common in 1st Embodiment in this form, the same code | symbol as the above is attached | subjected and each detailed description is abbreviate | omitted suitably.

  FIG. 6 is a block diagram showing a specific configuration of the index calculation unit 14. The filter processing unit 142 of the present embodiment extracts a plurality of components VC (VC1 to VC3) having different frequency bands from the audio signal SV. As shown in FIG. 6, a filter bank composed of three filter units FL (FL1 to FL3) having different passbands is suitably employed as the filter processing unit 142. The filter unit FL1 is a band-pass filter or low-pass filter that extracts the component VC1 of the low frequency side band BL (˜fc1) in FIG. 2 from the audio signal SV, and the filter unit FL3 is the high frequency side band BH (fc2˜). The band-pass filter or high-pass filter that extracts the component VC3 from the audio signal SV, and the filter unit FL2 is a band-pass filter that extracts the component VC2 of the intermediate band BM (fc1 to fc2) from the audio signal SV.

  The intensity detection unit 144 detects the intensity PC (PC1 to PC3) for each of the three types of components VC1 to VC3. The method for detecting the intensity PC from the component VC is the same as in the first embodiment. On the other hand, the intensity detector 146 detects the intensity P0 of the audio signal SV as in the first embodiment.

  The calculation unit 148 calculates the relative ratio between each of the intensities PC1 to PC3 and the intensity P0 as the phoneme index value A (A1 to A3). The phoneme index value A1 is a numerical value (A1 = PC1 / P0) corresponding to the intensity PC1 of the component VC1 of the band BL, and the phoneme index value A2 is a numerical value (A2 = PC2 / P0) corresponding to the intensity PC2 of the component VC2 of the band BM. The phoneme index value A3 is a numerical value (A3 = PC3 / P0) corresponding to the intensity PC3 of the component VC3 of the band BH. Therefore, it is possible to distinguish the phoneme of the input voice V according to the magnitude of the phoneme index values A1 to A3.

  For example, as understood from FIG. 2, when the phonological index value A1 and the phonological index value A2 exceed a predetermined threshold and the phonological index value A3 falls below the threshold (that is, the intensities of the band BL and the band BM in the frequency spectrum Q). Is higher than the band BH), the phoneme of the input voice V is discriminated as a bilateral sound. Also, when the phoneme index value A2 and the phoneme index value A3 exceed the threshold and the phoneme index value A1 falls below the threshold (that is, when the intensity of the band BM and the band BH in the frequency spectrum Q is higher than the band BL), The phoneme of the input voice V is discriminated as a gum sound. Further, when the phoneme index value A2 exceeds the threshold and the phoneme index value A1 and the phoneme index value A3 are below the threshold, the phoneme of the input voice V is discriminated as a soft palate sound.

  The sound selection table TBL associates each range of phoneme index values A1 to A3 corresponding to individual phonemes with the note number Nn. The sound selection unit 22 searches the sound selection table TBL for the note number Nn corresponding to the range of the phoneme index values A1 to A3 calculated by the index calculation unit 14 and instructs the data generation unit 30 to do so. On the other hand, the threshold value setting unit 25 variably controls the threshold value TON and the threshold value TOFF according to the phoneme discriminated from the phoneme index values A1 to A3.

  With the above configuration, the same operations and effects as in the first embodiment are achieved. Also, since the phoneme index values A1 to A3 are calculated for each of the plurality of components VC1 to VC3 corresponding to the separate bands (BL, BM, BH), a large number of phonemes are distinguished from those of the first embodiment. It is possible. Therefore, there is an advantage that various performance sounds can be selectively reproduced according to the phoneme of the input voice V. In the above embodiment, three types of phoneme index values A1 to A3 are calculated, but the number of phoneme index values A (the number of components VC extracted from the input speech V) is arbitrary.

<C: Modification>
Various modifications as exemplified below can be added to the above embodiments. Two or more aspects may be arbitrarily selected from the following examples and combined.

(1) Modification 1
The format of the sound data DS is not limited to the above example (MIDI format). A configuration in which the data generation unit 30 generates a data string (sample string) indicating a waveform on the time axis of a percussion instrument performance sound as sound data DS is also preferably employed. For example, the storage device 40 stores waveform data indicating the waveform of the performance sound for each of a plurality of types of percussion instruments. When sound generation is instructed from the sound generation determination unit 24, the data generation unit 30 selects the percussion instrument waveform data corresponding to the note number Nn specified by the sound selection unit 22 from the plurality of waveform data, and the volume of the waveform data. (Amplitude value) is increased / decreased according to velocity VEL, and then output to D / A converter 74. According to the above configuration, there is an advantage that the tone generator circuit 72 conforming to MIDI is unnecessary.

(2) Modification 2
In each of the above embodiments, the performance sound of a percussion instrument is exemplified, but the reproduction sound is arbitrarily changed. A configuration in which the sound selection unit 22 selects a performance sound of any of a plurality of musical instruments including instruments other than percussion instruments in accordance with the phonological index value A is also suitable. Further, the playback sound is not limited to the performance sound of the musical instrument. For example, the sound of applause may be reproduced.

  A configuration in which the data generation unit 30 generates sound data DS indicating any one of a plurality of performance sounds generated by one musical instrument is also suitable. For example, sound data DS specifying the note number Nn generated by the sound selection unit 22 as the pitch of the performance sound of one musical instrument is output from the data generation unit 30 to the sound source circuit 72. In the configuration in which the sound data DS is waveform data as in the first modification, the pitch of the waveform data of the performance sound of a specific instrument is converted according to the note number Nn and then output to the D / A converter 74. A configuration is adopted.

(3) Modification 3
In each of the above embodiments, the configuration in which the velocity VEL is set according to the peak value PK is exemplified. However, since the peak value PK and the intensity P0 of the audio signal SV are highly likely to be linked, the intensity detection unit 146 has detected. A configuration is also employed in which the sound volume determination unit 26 determines the velocity VEL based on the intensity P0.

(4) Modification 4
In each of the above embodiments, the configuration in which the input voice V is processed in the time domain is exemplified, but a configuration in which the intensity PC and the peak value PK are specified based on the frequency spectrum in which the voice signal SV is expanded in the frequency domain is also employed. . However, according to the configuration in which processing is performed in the time domain as in each of the above embodiments, frequency analysis such as FFT (Fast Fourier Transform) processing is unnecessary, and therefore, there is an advantage that the processing load by the control device 10 is reduced. is there.

(5) Modification 5
A configuration in which each function of the control device 10 in each of the above embodiments is realized by an electronic circuit such as a DSP, and a configuration in which each function of the control device 10 is realized by a plurality of integrated circuits are also suitable. Further, the sound collection device 64 and the sound emission device 76 are not essential requirements for the sound control device 100. For example, in the configuration in which the audio signal SV stored in the storage device 40 and the audio signal SV distributed via the communication network are processed, the sound collection device 64 and the A / D converter 62 are omitted. In the configuration in which the sound data DS generated by the data generation unit 30 is stored in the storage device 40 or in the configuration in which the sound data DS is transmitted to other devices via the communication network, the sound emitting device 76 or the D / A conversion is performed. The device 74 (and the sound source circuit 72) is omitted.

It is a block diagram which shows the structure of the sound control apparatus which concerns on 1st Embodiment of this invention. It is a conceptual diagram for demonstrating the difference in the frequency spectrum according to a phoneme. It is a conceptual diagram which shows the content of the sound selection table. It is a conceptual diagram which illustrates the some volume function which defines the relationship between a peak value and velocity. It is a flowchart of the process by a control apparatus. It is a block diagram which shows the structure of the parameter | index calculation part in 2nd Embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 ... Sound control apparatus, 10 ... Control apparatus, 12 ... Division | segmentation part, 14 ... Index calculation part, 142 ... Filter processing part, 144 ... Strength detection part, 146 ... Strength detection part, 148 ... Arithmetic unit, 16 …… Peak detection unit, 22 …… Sound selection unit, 23 …… Corresponding sound setting unit, 24 …… Sound generation determination unit, 25 …… Threshold setting unit, 26 …… Volume determination unit, 27 …… Correspondence Volume setting unit, 30 ... Data generation unit, 40 ... Storage device, 50 ... Input device, 62 ... A / D converter, 64 ... Sound collection device, 72 ... Sound source circuit, 74 ... D / A converter, 76 …… Sound emitting device.

Claims (4)

Index calculation means for calculating a phoneme index value that changes according to the phoneme of the input speech based on the intensity of a component of a specific band of the input speech;
Sound selecting means for selecting any one of a plurality of sounds based on the phonological index value;
Peak detecting means for detecting a peak value of the input voice;
Threshold setting means for variably setting a threshold according to the phoneme index value;
Pronunciation determination means for determining whether or not the peak value exceeds the threshold;
A sound control apparatus comprising: data generation means for generating sound data indicating generation of a sound selected by the sound selection means when the sound generation determination means determines that the peak value exceeds the threshold value . Corresponding volume setting means for variably setting the relationship between the intensity of the input voice and the volume of the sound indicated by the sound data;
A volume determining means for determining a volume corresponding to the peak value detected by the peak detecting means in the relationship set by the corresponding volume setting means;
The data generation unit generates sound data indicating a sound having a volume set by the volume determination unit.
The sound control device according to claim 1 . The threshold value setting means variably sets each of a first threshold value for determination of pronunciation and a second threshold value for determination of mute according to the phoneme index value,
The pronunciation determination unit determines whether the peak value detected by the peak detection unit exceeds the first threshold value, and whether the peak value detected by the peak detection unit is lower than the second threshold value. ,
The data generation means generates sound data indicating the occurrence of the sound selected by the sound selection means when the sound generation determination means determines that the peak value detected by the peak detection means exceeds the first threshold value. The sound control device according to claim 1 or 2 , wherein when the sound generation determination means determines that the peak value detected by the peak detection means is below the second threshold value, sound data indicating mute of the sound is generated. . An index calculation process for calculating a phoneme index value that changes according to the phoneme of the input speech based on the intensity of a component of a specific band of the input speech;
A sound selection process for selecting one of a plurality of sounds based on the phonological index value;
A peak detection process for detecting a peak value of the input voice;
A threshold setting process for variably setting a threshold according to the phoneme index value;
Pronunciation determination processing for determining whether or not the peak value exceeds the threshold;
A program for causing a computer to execute a data generation process for generating sound data indicating the generation of a sound selected in the sound selection process when it is determined in the sound generation determination process that the peak value exceeds the threshold value .

Images