JP2009122610A - Harmony sound generation device and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a system for generating harmony sound in a register intended by a user. <P>SOLUTION: The register corresponding to the harmony sound is set (D). In a first mode, when a pitch of an input speech signal Sm is in the set register, the pitch is made effective, and when it is outside, the pitch is corrected so as to enter the set register, or it is overridden (E). In a second mode, when the note of supplied performance information is in the set register, the note is made effective, and when the note is outside, the note is corrected so as to enter the set register (F). Then, based on the pitch Pm of a speech signal Sm which is made effective or corrected, or the note Pi of performance information, a note Nh of the harmony sound is determined (G). A musical sound signal Si having the harmony note Nh (H) is processed (vocoder) by using the speech signal Sm, and a harmony sound signal Sb is generated (J). Or, the harmony sound signal is generated by processing the speech signal Sm (pitch conversion) so as to have the harmony note Nh. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a harmony sound generation system that generates a desired harmony sound using an input audio signal.

Conventionally, a music signal processing apparatus that detects a pitch from an input audio signal and generates a harmony sound based on the detected pitch is known from Patent Document 1, for example.
Japanese Patent No. 3329245

  However, the conventional apparatus may not be able to correctly detect the pitch of the audio signal due to various factors, and cannot generate a harmony sound in the range intended by the user. For example, depending on the sound quality of the input audio signal, the pitch may be detected to be higher or lower than the octave, and in this case, a harmony sound having a pitch not intended by the user is generated. Even if the pitch detection itself is correct, if the range of the audio signal that can be input by the user and the range of the harmony sound to be generated are different, the harmony sound of the intended range is used. Cannot be generated.

  In view of such circumstances, in particular, the present invention provides a harmony sound generation system capable of generating a harmony sound in a sound range intended by the user by preventing generation of a harmony sound in a sound range not intended by the user. With the goal.

  According to the first feature of the present invention, the sound input means (14, 7) for inputting the sound signal (Sm), the sound range setting means (D) for setting the sound range, and the sound signal ( Sm), the pitch detection means (E1) for detecting the pitch, and whether the pitch detected by the pitch detection means falls within the range set by the range setting means, and when the pitch enters the range The pitch adjustment means (E2) that makes the pitch effective and corrects or invalidates the pitch so that it enters the sound range when the pitch does not enter the sound range, and the pitch (Pm) made effective by the pitch adjustment means ) Or harmony pitch determination means (G) for determining the pitch (Nh) of the harmony sound based on the corrected pitch (Pm), and the voice signal (Sm) input by the voice input means. Harmony sound generation means (H, J, L; N, P) for generating a harmony sound (Sb; Se) having a pitch (Nh) determined by the harmony pitch determination means by processing or using A harmonious sound generator (claim 1) and a sound range setting for setting a sound range in a computer having a sound input means (14, 7) for inputting a sound signal (Sm) and functioning as a harmony sound generator. Step (D), a pitch detection step (E1) for detecting the pitch of the audio signal (Sm) input by the audio input means, and the pitch detected in the pitch detection step falls within the range set in the range setting step. If the pitch falls within the range, the pitch is valid. If the pitch does not fall within the range, the pitch is entered into the range. Harmony for determining the pitch (Nh) of the harmony sound based on the pitch adjustment step (E2) to be corrected or invalidated and the pitch (Pm) or the corrected pitch (Pm) validated in the pitch adjustment step The pitch determination step (G) and the harmony sound (Sb; Se) having the pitch (Nh) determined in the harmony pitch determination step by processing or using the voice signal (Sm) input by the voice input means. ) Generating a harmony sound generating step (H, J, L; N, P). Note that the parentheses indicate reference symbols or the like of the embodiments added for convenience of understanding, and the same applies to the following.

  According to the second feature of the present invention, the sound input means (14, 7) for inputting the sound signal (Sm), the sound range setting means (D) for setting the sound range, and the pitch for supplying the pitch information. When the pitch of the pitch information supplied by the information supply means (A) and the pitch information supply means falls within the range set by the range setting means, and the pitch enters the range Indicates that the pitch is valid, and when the pitch does not fall within the range, the pitch adjustment means (F) for correcting the pitch so that it falls within the range, and the pitch that has been validated by the pitch adjustment means (Pi, Ph) or harmony pitch determination means (G) for determining the pitch (Nh) of the harmony sound based on the corrected pitch (Pi, Ph), and the voice signal input by the voice input means ( By processing or using Sm), the harmony pitch determination means A harmony sound generating device (H, J, L; N, P) for generating a harmony sound (Sb; Se) having a predetermined pitch (Nh); and A sound range setting step (D) for setting a sound range and a sound for supplying pitch information to a computer having sound input means (14, 7) for inputting a sound signal (Sm) and functioning as a harmony sound generating device; It is determined whether the pitch of the pitch information supplied in the pitch information supply step (A) and the pitch information supply step falls within the range set in the range setting step, and the pitch enters the range. When the pitch does not fall within the range, the pitch adjustment step (F) for correcting the pitch to fall within the range, and the sound made effective at the pitch adjustment step High (Pi, Ph) or modified The harmony pitch determination step (G) for determining the pitch (Nh) of the harmony sound based on the pitch (Pi, Ph) and the voice signal (Sm) input by the voice input means are processed or used. , A harmony sound generation step (H, J, L; N, P) for generating a harmony sound (Sb; Se) having the pitch (Nh) determined in the harmony pitch determination step is executed. A sound generation program (Claim 4) is provided.

  According to the first feature of the present invention (claims 1 and 3), the harmony sound (Sb) having the pitch (Nh) determined based on the pitch (Pm) of the input sound signal (Sm) is generated. (H, J, L; N, P) In a harmony sound generation system, a sound range setting function (D) is provided, and is set when the pitch of the detected audio signal deviates from the sound range set by the sound range setting function. Therefore, the harmony sound in response to the pitch of the voice is prevented from being generated in the range not intended by the user, and the range intended by the user is determined. It is possible to reliably generate a harmony sound.

  According to the second aspect of the present invention (claims 2 and 4), the harmony sound (Sb) having the pitch (Nh) determined based on the pitch (Pi, Ph) of the supplied performance information is generated. (H, J, L; N, P) In a harmony sound generation system, a sound range setting function (D) is provided, and when the pitch of the supplied performance information deviates from the sound range set by the sound range setting function, it is set. (F) is modified so that it falls within the specified sound range, so that the harmony sound responding to the pitch of the performance sound is not generated in the range not intended by the user, and the harmony in the range intended by the user is determined. Sound can be generated reliably.

[System Overview]
An electronic music apparatus incorporating a function of a harmony sound generation system according to an embodiment of the present invention is a kind of computer having a music information processing function including a harmony sound generation process, and is a music dedicated device such as an electronic musical instrument or a predetermined device. A general-purpose information processing apparatus such as a personal computer having a music information processing function is used. FIG. 1 is a block diagram showing the hardware configuration of an electronic music apparatus in which a harmony sound generation system according to an embodiment of the present invention is constructed.

  This electronic music apparatus includes a central processing unit (CPU) 1, a random access memory (RAM) 2, a read only memory (ROM) 3, a storage device 4, a performance operation detection circuit 5, a setting operation detection circuit 6, and an A / D conversion. A circuit 7, a display circuit 8, a sound source / effect circuit 9, a communication interface (I / F) 10, etc. are provided, and these elements 1 to 10 are connected to each other via a bus 11.

  The CPU 1 constitutes a data processing unit together with the RAM 2 and the ROM 3, and executes various music information processing including a harmony sound generation process using a clock by a timer according to a predetermined control program including a harmony sound generation program. The RAM 2 is used as a work area for temporarily storing various data necessary for these processes. The ROM 3 stores various control programs including a harmony sound generation program, preset effect tables, performance data, and the like in order to execute these processes.

  The storage device 4 includes a storage medium such as an HD (hard disk), an FD (flexible disk), a CD (compact disk), a DVD (digital multipurpose disk), a flash memory, and the like, and a driving device for the storage device. Programs, tables, performance data, and the like can be stored in any storage medium. The storage medium may be detachable or may be built in the electronic music device.

  The performance operation detection circuit 5 constitutes a performance operation section together with a performance operation element 12 such as a keyboard, detects performance operation of the performance operation element 12, and transmits performance operation information corresponding to the detected contents to the data processing sections (1 to 3). To introduce. The data processing unit can send actual performance data obtained by processing the performance operation information to the sound source / effect circuit 9 or store it in the storage device 4 as automatic performance data. The setting operation detection circuit 6 constitutes a setting operation unit together with a setting operation element (panel operation element) 13 such as a switch or a mouse, detects the operation content of the setting operation element 13 and performs data processing on setting information corresponding to the detection content Introduce to the department. For example, the setting operator 13 includes a mode designation switch for designating the first and second modes, a tone range designation operator, a harmony type designation operator, a tone color designation operator, an effect selection operator, and the like. It is.

  The A / D conversion circuit 7 constitutes an audio input unit together with the microphone 14, and A / D converts the audio signal (microphone sound) input from the microphone 14 and introduces it to the data processing unit. The display circuit 8 includes a display 15 such as an LCD for displaying various screens necessary for performance and settings, and indicators / lamps. The display circuit 8 controls the display and lighting contents according to commands from the data processing unit, and performs performance operations and settings. Provide display assistance for operation, microphone sound input, etc.

  The sound source / effect circuit 9 includes a sound source and a DSP, and generates a musical sound signal based on actual performance data obtained from performance operation information of the performance operation unit (12-5) and automatic performance data obtained from the ROM 3 or the storage device 4. And generating an effect on the musical sound signal (performance sound signal), adding an effect to the audio signal from the A / D conversion circuit 7, and generating a musical sound signal and an audio signal based on various parameters for generating a harmony sound. Is controlled to generate a harmony sound signal, and an effect can be imparted to the harmony sound signal. The sound system 16 includes a D / A conversion unit, an amplifier, a speaker, and the like, and generates a musical sound signal or sound signal output from the sound source / effect circuit 8, a musical sound (performance sound) based on the harmony sound signal, a sound, or a harmony sound. To do.

  The communication I / F 10 includes a music-only wired I / F such as MIDI, a general-purpose short-distance wired I / F such as USB and IEEE 1394, a general-purpose network I / F such as Ethernet (registered trademark), a wireless Lan, and Bluetooth (registered trademark). The storage device 4 includes one or a plurality of general-purpose short-range wireless I / Fs, etc., and exchanges MIDI performance data with other music devices or receives control programs and data from an external server computer or the like. Can be saved.

  When the electronic music apparatus is composed of a PC, the performance operator 12 is externally attached. The server computer has the same hardware configuration as that shown in FIG. 1, but may not include a performance operator, a performance operation detection circuit, a sound source / effect circuit, and the like.

[Harmony sound generation system using vocoder]
In the harmony sound generation system according to one embodiment of the present invention, the tone signal determined based on the pitch of the input sound or the pitch of the performance information is obtained by processing the musical sound signal using the input sound using a vocoder. When generating a harmony sound, adjust the pitch of the pitch of the input sound or the pitch of the performance information to prevent the generation of a harmony sound in a range not intended by the user, or automatically according to the harmony type setting Appropriate effects can be preset for voice and harmony sounds.

  FIG. 2 is a functional block diagram of a harmony sound generation system using a vocoder according to an embodiment of the present invention. In this harmony sound generation system, a voice input unit (microphone sound input unit) 14-7, pitch information supply unit A, chord information supply unit B, harmony type setting unit C, range setting unit D, pitch detection unit E1, and pitch Pitch adjustment unit E composed of correction unit E2, pitch adjustment unit or pitch correction unit F, harmony sound determination unit G, sound source H, vocoder unit J composed of feature extraction unit J1 and formant filter unit J2, performance sound effect imparting unit K, the harmony sound effect imparting unit L, the sound effect imparting unit M, etc. constitute a harmony sound generating system, the pitch adjusting unit E operates only in the first mode (pitch detection type mode), and the pitch adjusting unit F is the first one. Operates only in 2 modes (pitch specification type mode). The function units A to G correspond to the functions of the data processing units 1 to 3, and the function units H to M correspond to the functions of the sound source / effect circuit 9.

  FIG. 3 is a diagram for explaining the function of a vocoder constituting a vocoder unit in a harmony sound generation system according to an example of the present invention. The vocoder principle is related to the way people speak. A person utters a voice by resonating the sound generated in the vocal cords with the mouth and nose, but the vocoder's resonating part works as a kind of filter with frequency characteristics and uses the characteristics extracted from the microphone input of the electronic musical instrument. The instrument sound input from the sound source is processed to generate a large number of formants. In other words, as shown in FIG. 3 (1), the vocoder J extracts the sound characteristics for the filter from the sound (microphone sound) signal Sm input from the microphone 14 via the A / D converter (J 1). A performance sound (instrument sound) signal Si from the sound source H is passed through a large number of bandpass filters (BPF) to generate a vocoder sound signal (so-called “robot voice”) Sv that reproduces the formant of the sound. For example, if you play a keyboard while talking using a microphone, you can make a musical instrument sound like a robot voice.

  Explaining in more detail according to the configuration example shown in FIG. 3 (2), a plurality of (n) resonance units (Band1 to n) provided for each predetermined frequency band with respect to the audio signal Sm are respectively for feature extraction. A filter (BPFm1 to BPFmn) and a level detector, a formant reproduction filter (BPFi1 to in), a volume controller, an output amplifier (G1 to Gn), and the like. The noise signal is removed from the microphone-input audio signal Sm by a noise gate, and the frequency components in each band are detected by the feature extraction filters (BPFm1 to mn) in the respective resonance units (Band1 to n). The detector detects the level of each frequency component.

  The performance sound signal Si from the sound source H is added with noise generated in the vocoder, and then the frequency components of each band are extracted through the formant reproduction filters (BPFi1 to BPFin) of the respective resonance units (Band1 to n). It is introduced into the volume controller (note that the cutoff frequencies of BPFm1 to mn and BPFi1 to in can be made different by setting the shift amount or the offset amount, and do not necessarily match). In each volume controller, the volume of each frequency component of the performance sound signal Si is controlled in accordance with each frequency component level of the audio signal Sm, and is output from each resonance unit (Band1 to n) through each output amplifier (G1 to Gn). Is done.

  Then, the high-pass filter (HPF) passing signal of the audio signal Sm is added to the signal obtained by adding the output signals from the respective resonance units (Band 1 to Band n) (Wet), and further, the performance sound signal Si (Dry) is added. Thus, the vocoder sound signal Sv is obtained.

  Now, returning to the description of the functional block diagram of FIG. Corresponding to the automatic performance processing function portion (automatic performance device) of the data processing units 1 to 3 for processing performance data from the ROM 3 or the storage device 4, and obtained from the actual performance information of the performance operation unit 12-5. In accordance with the actual performance data and the automatic performance data read from the storage device 4, one or more pieces of performance information Np are sequentially supplied to the sound source H and necessary performance information is supplied to the chord information supply unit B. In the second mode, necessary pitch information (for example, pitch information Pi, Ph by melody performance or harmony performance) is also supplied to the pitch adjustment unit F. If the chord progression data is included in the automatic performance data used for the automatic performance, the chord progression data itself may be supplied to the chord information supply unit B.

  The chord information supply unit B specifies a chord by detecting a chord (code) based on the pitch information supplied from the pitch information supply unit A, and obtains chord information (code information) Cd representing the specified chord. It is supplied to the harmony sound determination unit G. If the chord progression data is included in the automatic performance data, the chord information Cd constituting the chord progression data supplied from the pitch information supply unit A is sequentially supplied to the harmony sound determination unit G.

  The harmony type setting unit C gives to the harmony sound determination unit G the type designation information Th for designating the type of harmony sound to be generated (harmonic type) based on the operation of the harmony type designation operator in the setting operator 13. . This type Th is called a harmony type, and is roughly divided into a vocoder type and a chordal type (chord type) as will be described later. The harmony type setting unit C can automatically preset the effect types Eb and Ec to be given to the vocoder sound signal Sv and the sound signal Sm, respectively, in accordance with the setting state of the harmony type Th. As for effect setting, when there is no effect selection operation by the user or when “preset” is specified, the standard setting mode is set, and the harmony type setting unit C is automatically set according to the set harmony type Th. The effect designation information for designating the effect types Eb and Ec is output to the harmony sound and sound effect applying units L and M, respectively.

  In the voice input unit (microphone sound input unit) 14-7, an analog voice signal (waveform signal) capable of detecting at least the pitch of singing, instrumental sound, etc. is input from the microphone 14, and the input voice signal is A / D conversion is performed by the A / D conversion circuit 7 to convert it into a digital audio signal Sm that can be used in this harmony sound generation system.

  The sound range setting unit D sets the sound range (pitch range) assumed for the audio signal input from the sound input unit 14-7 based on the operation of the sound range specifying operation unit 13 by the setting operation unit 13, for example, the user Can set the range that can be sung by the range setting operation. The range setting unit D also outputs range specification information representing the set range (set range) to the pitch adjustment unit E in the first mode and to the pitch adjustment unit F in the second mode.

  In the first mode in which the pitch information of the voice is used for determining the harmony sound, the pitch adjusting unit E is activated and the pitch information used for determining the harmony sound is adjusted. Therefore, in the pitch adjustment unit E, first, the pitch detection unit E1 detects the pitch of the audio signal Sm input from the audio input unit 14-7, and the subsequent pitch correction unit E2 detects the detected audio signal Sm. Is in the set sound range specified by the sound range specifying information from the sound range setting unit D, and the pitch information is processed according to the determination result. That is, when the pitch of the audio signal Sm detected by the pitch detection unit E1 is within the set sound range, the pitch information Pm is output to the harmony sound determination unit G as it is, and when it is not within the set sound range, The detected pitch is shifted by an octave and corrected so as to fall within the set sound range and the corrected pitch information Pm is output to the harmony sound determination unit G, or the pitch detection by the pitch detection unit E1 has failed. Ignore speech input. In the harmony sound determination unit G, the pitch information Pm from the pitch adjustment unit E is used as input sound information indicating a reference sound for determining the pitch (pitch) Nh of the harmony sound.

  On the other hand, in the second mode in which the pitch information of the performance sound is used to determine the harmony sound, the pitch adjustment unit F is activated in place of the pitch adjustment unit E, and the pitch information of the performance sound used to determine the harmony sound (For example, pitch information Pi of melody performance) is adjusted. The pitch adjustment unit F determines whether or not the pitch of the performance sound indicated by the pitch information supplied from the pitch information supply unit A is within the set range specified by the range specification information from the range setting unit D. If the pitch of the performance sound is within the set range, the pitch information Pi is output to the harmony sound determination unit G as it is, and if it is not within the set range, the pitch of the performance sound is determined. The pitch information Pi after the correction is made so as to be within the set sound range by the octave shift, and the corrected pitch information Pi is output to the harmony sound determination unit G. In the harmony sound determination unit G, the pitch information Pi from the pitch adjustment unit F is used as input sound information indicating a reference sound for determining the pitch Nh of the harmony sound.

  In the second mode, instead of supplying the performance pitch information (Pi) from the pitch information supply unit A and the chord information Cd from the chord information supply unit B to the harmony sound determination unit G, as shown by broken lines. The pitch information Ph based on the harmony performance can be supplied to the harmony sound determination unit G to determine the harmony sound based on the harmony performance. Also in this case, similarly to the pitch information Pi, the pitch of the harmony performance is adjusted by the pitch adjustment unit F, and the adjusted harmony performance pitch information Ph is supplied to the harmony sound determination unit G.

  The harmony sound determination unit G is based on the input sound information (Pm, Pi) and the chord information Cd or the harmony performance pitch information Ph according to the harmony type specified by the type specification information Th from the harmony type setting unit C. The pitch (pitch) Nh of one or more harmony sounds is determined, and harmony sound information of the determined pitch Nh is supplied to the sound source H.

  The sound source H generates musical tone signals Si and Sj based on the harmony sound information Nh supplied from the harmony sound determination unit G and the performance sound information Np from the pitch information supply unit A. Of the generated tone signals, the tone signal Si of the performance part to be supplied to the vocoder part J is supplied to the vocoder part J. When the harmony type Th is “vocoder”, the musical tone signal Si corresponding to the performance sound information Np of the predetermined performance part in the actual performance data or the automatic performance data based on the performance operation of the performance operator 12 is supplied to the vocoder section J. The For example, if the part to which the vocoder effect is to be applied is the upper keyboard (melody part), the tone signal Si of the timbre set on the upper keyboard is supplied to the vocoder section J. Further, the musical sound signal Sj of the other performance part is supplied to the performance sound effect imparting unit K, and the performance sound effect imparting unit K imparts the effect to the musical sound signal of the other performance part according to the effect information set for the part. Then, the musical sound signal Sa of the other part to which the effect is given is output to the sound system 16 as a performance sound signal.

  In the vocoder unit J, the feature extraction unit J1 extracts the feature of the input speech signal from the speech signal converted into analog by the A / D conversion circuit 7, and controls the characteristics of the formant filter J2 based on the feature. The musical sound signal Si supplied from the sound source H to the vocoder part J passes through the formant filter J2, and is converted into a vocoder sound signal Sv having a unique sound quality like a robot voice.

  The musical sound signal Sv that has passed through the vocoder part J is supplied to the harmony sound effect imparting part L, and a predetermined effect is imparted to the sound system 16 as a harmony sound output signal Sb. In the standard setting mode, the effect imparting unit L imparts an effect according to the designation by the effect designation information Eb input from the harmony type setting unit C. That is, the type of effect to be given can be arbitrarily selected by the user, but by selecting “preset”, an appropriate effect type Eb can be set according to the harmony type Th. .

  The sound signal Sm from the microphone is supplied to the sound effect applying unit M, and a predetermined effect is applied to the sound system 16 and is output to the sound system 16 as the sound output signal Sc. In the standard setting mode, the sound effect imparting unit M gives an effect to the sound signal Sm according to the designation by the effect designation information Ec from the harmony type setting unit C, but the effect imparted by the sound effect imparting unit M is arbitrary. The points that can be selected are the same as those of the harmony sound effect imparting unit L. Note that the effect type may be automatically set according to the harmony type Th in both the harmony sound effect imparting unit L and the sound effect imparting unit M, or any one of them.

  As described above, according to the feature relating to the adjustment of the voice pitch or performance pitch in this harmony sound generation system, a range corresponding to the harmony sound is set (D), and in the first mode, the input voice signal If the pitch of Sm falls within the set range, the pitch is valid, and if it falls outside, the audio pitch is modified or ignored so that it falls within the set range (E). In the second mode, the sound of the supplied performance information If the pitch falls within the set range, the pitch is validated, and if it falls outside, the performance pitch is corrected so that it falls within the set range (F). Next, the pitch Nh of the harmony sound is determined based on the pitch Pm of the voice signal Sm that has been validated or modified or the pitches Pi and Ph of the performance information (G). Then, using the audio signal Sm, the musical tone signal Si (H) having the harmony pitch Nh is processed (vocoder) to generate the harmony tone signal Sb (J), or the harmony pitch Nh is set. The voice signal Sm is processed (pitch conversion) to generate a harmony sound signal. This feature makes it possible to generate a harmony sound in the range intended by the user.

  Further, according to the characteristics related to the harmony type setting of the harmony sound generation system, when the harmony sound type Th is set, the types of effects Eb and Ec to be imparted to the harmony sound or the sound are automatically set according to the set type Th. Set (C). Next, based on the set type Th and the pitch Pm of the input audio signal Sm (first mode), or based on the set type Th and the pitch of the supplied pitch information Pi and Ph ( In the second mode, the harmony pitch Nh is determined (G), and the harmony tone signal Sv having the harmony pitch Nh is generated using the voice signal Sm (J). Then, the set types of effects Eb and Ec are given to the generated harmony sound signal Sv or sound signal Sm (L, M). As for the generation of the harmony sound signal, the sound signal Sm may be processed (pitch conversion) so as to have the harmony pitch Nh to generate the harmony sound signal. With this feature, it is possible to set an appropriate effect type according to the type of harmony sound without having knowledge about musical knowledge and effect types.

[Range setting and range determination processing]
For the sound range setting in the sound range setting unit D, for example, the following methods (1) to (4) can be adopted:
(1) A specific pitch range (for example, “C3 to C5”) is designated.
(2) Specify terms related to gender and age such as “male”, “female”, “adult”, “child”. This method is due to the fact that the sound range that can be generated for each gender and age is roughly determined.
(3) Designate a vocal range such as “soprano”, “alto”, “tenor”, “bass”.
(4) The user actually utters a voice and detects the pitch, and the range between the detected highest pitch and the lowest pitch is set as the set sound range. In this case, it is possible to provide a range setting mode, operate the pitch detection unit E1 in the range setting mode, and set the range using the maximum value and the minimum value of the pitch detected by the pitch detection unit E1. .

  In the first mode, the pitch of the input sound is adjusted in the pitch adjustment unit E according to the relationship with the range setting range, and in the second mode, the pitch of the performance sound supplied is adjusted in the pitch adjustment unit F. The pitch is adjusted according to the relationship with the range setting, and the adjusted pitch or pitch becomes the input sound to the harmony sound determination unit G. FIG. 4 is a diagram for explaining the adjustment operation of the voice pitch and the performance pitch in the pitch adjusting unit and the pitch adjusting unit, and FIG. 5 is a flowchart showing the range determination processing in the pitch adjusting unit or the pitch adjusting unit. It is.

  The pitch adjusting unit E performs a pitch adjusting action as shown in FIG. 4A in the first mode. Here, it is assumed that the pitch of the audio signal Sm input from the A / D conversion circuit 7 is a pitch within the set sound range. For example, the user can set the range in which he can sing as the set range, so that the pitch of the input audio signal Sm falls within the set range as shown in a.

  When the pitch detection unit E1 of the pitch adjustment unit E erroneously detects the pitch of the input audio signal Sm for some reason, it is determined by the pitch correction unit E2 that the pitch is out of the set sound range, as in b. There is. In other words, the sound is not originally a sound that can be used for pitch detection, and the pitch detection unit E1 has high accuracy in terms of system performance, but easily picks up disturbance (ambient sound noise). Many. In this case, the pitch correction unit E2 corrects the pitch detected by the pitch detection unit E1 by an octave shift so as to fall within the set sound range as shown in c, or the pitch detection result is changed as shown in d. Invalid. The harmony sound determination unit G determines the pitch of the harmony based on the pitch thus shifted. When it is disabled, there is no harmony sound.

  On the other hand, in the second mode, the pitch adjusting unit F performs a pitch adjusting operation as shown in FIG. During actual performance of performing based on actual performance data, the pitch to be performed by operating the performance operator 12 is limited by the physical range of the performance operator 12, the relationship between the left and right hands during performance, and the like. It may not be possible to obtain the pitch of the desired range indicated by the set range. In this way, when the pitch that is played deviates from the set range, the pitch adjuster F corrects the pitch so that it is shifted within the set range by an octave shift. Then, the harmony sound determination unit G determines the pitch of the harmony based on the pitch thus shifted.

  The pitch adjusting operation in the pitch correcting unit E1 of the pitch adjusting unit E or the pitch adjusting operation in the pitch adjusting unit (pitch correcting unit) F is executed by the range determination processing flow of FIG. That is, in the first mode, when the pitch detection unit E1 receives the pitch detection result from the pitch detection unit E1, first, in step S1, the pitch of the sound detected by the pitch detection unit E1 is outside the set sound range. If it is outside the set range (S1 = YES), the process proceeds to step S2, and the pitch is corrected to the set range or ignored. When it is determined that the detected sound pitch is within the set sound range (S1 = NO) and after the process of step S2, the process returns to a state of waiting for the next pitch detection result.

  In the second mode, when the pitch information of the performance data is given from the pitch information supply unit A, the pitch correction unit F firstly, in step S1, the pitch indicated by the supplied pitch information, that is, the performance. It is determined whether or not the pitch is outside the set range, and if it is outside the set range (S1 = YES), the process proceeds to step S2, and the pitch is corrected to be within the set range. Then, when it is determined that the played pitch is within the set range (S1 = NO) and after the process of step S2, the process returns to a state of waiting for the next pitch information supply.

[Examples of pronunciation series and part structure]
As shown in FIG. 2, the sound generation sequence of this system includes a performance sound generation sequence for outputting a performance sound signal Sa from the input performance sound information Np via the musical sound signal Sj, and a harmony sound from the harmony sound information Nh via the musical sound signal Si. The harmony sound generation sequence that outputs the signal Sb and the sound generation sequence that outputs the audio signal Sc from the input audio signal Sm. Let's see the sound generation sequence from the sound source part (musical signal generation sequence) of the sound source section H. FIG. 6 shows an example of the structure of a pronunciation sequence and parts according to an embodiment of the present invention. In a system in which the performance operator 12 includes, for example, an upper keyboard (UK), a lower keyboard (LK), and a pedal keyboard (PK). These are represented by a pronunciation sequence as shown.

  Here, the function of the harmony type Th set in the harmony type setting unit C will be described. The harmony type Th is a normal performance mode that is not the harmony sound generation mode as shown in FIGS. 7 (1) and 8 (2). In addition to Th = “Through” to indicate, there are “vocoder type” and “codel type”, and the vocoder type consists of one type Th = “vocoder”, and the part corresponding to the input sound (Pm, Pi) Indicates that the vocoder effect should be added directly to the performance sound. The chordal type is composed of a plurality of types Th = “Duet 1”, “Duet 2”,..., And the vocoder effect should be applied to various chordal sounds with respect to the input sound (Pm, Pi). Instruct. The part to which the vocoder effect is to be applied or the part to be used as a reference when determining the sound to which the vocoder effect is applied is set in advance in the UK part (melody part) corresponding to the upper keyboard right key range (melody performance sound Np1), for example. Is done.

  Although not shown in FIG. 6, in the normal performance mode with Th = “Through”, the pitch information supply unit A generates performance sound information Np = Np1 to Np3 based on the performance of each keyboard (UK to PK) as a sound source. The tone generator H generates a musical tone signal based on each piece of musical performance information with a musical tone signal generation sequence of a predetermined tone color set for the UK part, the LK part, and the PK part corresponding to each keyboard.

  As described above, when the UK part corresponding to the melody performance sound Np1 of the upper keyboard is set as the vocoder effect imparting part, when the harmony type Th = “vocoder”, that is, the vocoder type is designated, FIG. The part structure is as shown in That is, in the first mode, the harmony sound determination unit G determines the pitch indicated by the pitch information Pm adjusted (detected and corrected) by the pitch adjustment unit E as the pitch Nh of the vocoder sound, and the sound source unit H In the UK part (melody part), a predetermined tone color (for example, organ) corresponding to this part is determined according to the tone generation parameters (pitch length, strength, etc. other than the pitch) of the melody performance sound Np1 and the determined pitch Nh = Pm. A tone signal Si having a tone pitch Nh = Pm is generated. In the second mode, the pitch indicated by the pitch information Pi of the melody performance sound Np1 adjusted (corrected) by the pitch adjusting unit E is determined as the pitch Nh of the vocoder sound, and the sound source unit H is used in the UK part. Thus, a musical tone signal Si having a pitch Nh = Pi is generated with a predetermined tone color (for example, organ tone color) corresponding to the melody performance tone Np1.

  Then, the vocoder part J processes the predetermined tone color tone signal Si of the UK part by using the voice signal Sm (provides a vocoder effect to the tone signal Si), and uses the vocoder sound signal Sv obtained thereby as a harmony effect applying part L. Send to. In the other LK and PK parts, as in the normal performance mode, musical tone signals Sj generated corresponding to the respective performance sound information Np2, Np3 are sent to the performance sound effect applying unit K. Note that the vocoder sound Sb when Th = “vocoder” is designated may not be called a “harmony sound” in the strict sense, but sounds (audio output Sc) and other musical sounds (performance sound output Sa) In this specification, it is called a “harmonic sound” because it is a sound (Sb) that is pronounced together with the sound.

  On the other hand, when the harmony type Th is designated as the chordal type, the part configuration is as shown in FIG. That is, in the first mode, the harmony sound determination unit G determines the pitch Nh of the harmony sound based on the pitch indicated by the pitch information Pm from the pitch adjustment unit E and the chord information Cd, and also in the second mode, As in the first mode, the harmony pitch Nh is determined based on the pitch indicated by the pitch information Pi of the melody performance sound Np1 input from the pitch adjustment unit F and the chord information Cd. In this case, the harmony pitch Nh is attached differently depending on each type of chordal type Th = “Duet 1”, “Duet 2”,. Only in the second mode, there are types Th = “Harmony 1” and “Harmony 2” (see FIG. 8 (2)), and when these types are set (harmony performance mode), the harmony performance sound. The harmony pitch Nh is determined by the information Ph.

  When the harmony type is a chordal type, the tone generator H is a dedicated part called a harmony sound part, which is a musical tone generation parameter of the performance sound Np1 of the melody part set as a reference part (a pitch other than the pitch, In accordance with the harmony pitch Nh (= Pm, Pi) determined to be strong, etc., a musical tone signal Si having a pitch Nh having a predetermined tone color (for example, violin tone) set in advance corresponding to this dedicated part is generated. To do. In addition, one set of timbres for the harmony sound part may be fixedly set in this system, may be set in association with the harmony type, or the user operates the timbre designating operator. It may be arbitrarily selected in advance by (tone color setting operation). In this case, it is desirable that the tone signal Sv that has passed through the vocoder part J represents a tone having characteristics similar to those of a human voice. For example, the tone color is a continuous tone such as a violin or saxophone. In addition, a musical sound having a frequency characteristic approximate to that of a human voice is preferable.

  In the harmony sound part, the vocoder part J processes the musical tone signal Si of the set tone color by the voice signal Sm (gives a vocoder effect to the musical tone signal Si), and uses the resulting vocoder sound signal Sv as a harmony sound effect giving part. Send to L. In the other UK to PK parts, as in the normal performance mode, the musical sound signals Sj generated corresponding to the respective performance sound information Np1 to Np3 are sent to the performance sound effect applying unit K.

[How to add harmony sound]
The harmony sound determination unit G adds a harmony sound to the performance sound according to the harmony type Th set by the harmony type setting unit C, and generates harmony sound information indicating the pitch Th of the harmony sound. FIG. 7 shows an example of how to add a harmony sound according to one embodiment of the present invention. In FIG. 7A, as described above, the harmony type Th = “through” indicates a normal performance mode that does not generate a harmony sound, and Th = “vocoder” indicates the pitch of the input sounds Pm and Pi. It is instructed that the pitch of the sound Th1 should be set.

  Harmony type Th below “Duet 1” is collectively referred to as “Cordal type” as described above, and “Harmony 1” and “Harmony 2” marked with an asterisk (see FIG. 8 (2): only in the second mode) Indicates that the pitch of the harmony sound Th: Th1, Th2,... Should be set based on the reference input sounds Pm, Pi and the chord information Cd (the symbol “Nh” The pitch is representatively represented). In FIG. 7 (1), “upper” of the input sound is a chord located immediately above the input sounds Pm and Pi when the input sounds Pm and Pi are the root sounds of the chord constituent sounds indicated by the chord information Cd. In the same way, “upper” of the input sound represents the pitch of the chord constituent sound located second on the input sounds Pm, Pi, and “lower” of the input sound represents Similarly, it represents the pitch of the chord constituent sound located immediately below the input sounds Pm and Pi. “Oct” represents an octave. For example, “down 1 oct” represents the pitch of a chord constituent sound located immediately below the input sounds Pm and Pi one octave.

  That is, when a chordal type excluding the * mark is designated, the harmony sound determination unit G has a pitch Nh1, one or more upper and / or lower chord constituent sounds with respect to the pitches Pm, Pi of the input sound. Nh2,... Are determined as the pitches Nh of the harmony sounds, and the manner in which the constituent sounds are attached varies depending on the type of chordal type. For example, as shown in FIG. 7 (2), the chord information Cd indicates a chord Cmaj composed of pitch names C, E, and G, and the root tone is C. 1 mode) or when the melody performance pitch Pi (second mode) indicates the pitch C3 and the harmony type Th = “Duet 1” is designated, “above” the pitch C3 as the root tone. Is determined as the first harmony sound Nh1. Further, in this case, when Th = “trio 2” is designated, the “up” pitch E3 of the root tone C3 is set as the first harmony tone Nh1, and the “down” pitch G3 is set as the second harmony tone Nh2. decide.

[Selection of effect type and harmony type]
In this system, a predetermined effect type is automatically preset in a predetermined effect applying unit according to the setting of the harmony type, or a desired type of effect is given to the effect applying unit designated by the user by an effect selection operation by the user. Can be set. FIG. 8 is a diagram for explaining a selection mode of effect types in one embodiment of the present invention. When any of the harmony types is set in the system, the user operates the effect selection operator to instruct a desired option from the effect type options shown in FIG. Arbitrary effect types can be set in the arbitrary effect imparting portions L and M.

  “Preset” is an option for referring to the preset effect table shown in FIG. 8 (2) and instructing that the effect types Eb and Ec should be set in the harmony sound and sound effect imparting units L and M according to the contents. Although it is set by default in the system, it can also be selected by a user's effect selection operation. That is, when the harmony type Th is designated by the operation of the harmony type designation operator when “preset” is set, the preset effect table of FIG. The type of effect imparted to the vocoder harmony sound signal Sv to the sound signal Sm is automatically preset by the sound effect imparting unit M. Further, when “preset” is selected by a user operation when any of the harmony types is set, this table is referred to and a predetermined effect is set in the predetermined effect applying units L and M, which is the same as the preset content. You can return to the setting.

  In the preset effect table, as a basic policy, in the case of pop-type harmony types such as duets and trios (“Duet 1” to “Quartet 2”), it is set to “Thru”, and the harmony type of large chorus ( The chorus effect (“chorus 1”) is given in the case of “Choire 1” and “Choire 2”), and “detune” is given to the type that sings the same voice part in plural (“Unison”). In the table, “Thru” of the harmony content indicates that no harmony sound is set, and “through” in the effect type column indicates that no effect is given.

  In addition, when the effect imparting portions L and M to be effect imparted are designated and “Chorus 1” is selected, the effect imparting portions L and M are set so as to impart the first chorus effect. Similarly, if you specify the effect imparting section and select "Chorus 2", "Detune", "Distortion", ..., set another second chorus effect, Detune, Distortion (distortion effect), ... To do.

  FIG. 9 is a flowchart showing harmony selection processing in the harmony sound generation system according to one embodiment of the present invention. In the first step T1 of the harmony selection process, the harmony type Th is selected by the user's harmony type designation operation and set in the harmony sound determination unit C. In the next step T2, it is determined whether or not the currently selected effect type is “preset”, that is, one selected according to the preset effect table. If it is “preset” (T2 = YES), step T3 is determined. Then, the effect type associated with the selected harmony type is selected and set in the effect imparting units L and M. After this setting process, the process returns to a state of waiting for the next harmony type selection. If it is determined in step T2 that it is not “preset” (T2 = NO), the process immediately returns to a state of waiting for the next harmony type selection.

  FIG. 10 is a flowchart showing effect type selection processing in the harmony sound generation system according to one embodiment of the present invention. In the first step U1 of the effect type selection process, the effect type set in the effect applying units L and M is selected by the effect selection operation by the user. In the next step U2, it is determined whether or not the selected effect type is “preset”, that is, according to a preset effect table. If the selected effect type is “preset” (U2 = YES), in step U3, the effect type associated with the current harmony type is set in the effect imparting units L and M, and the selected effect is selected. If the type is not “preset” (U2 = NO), in step U4, the selected effect type is set in the effect applying units L and M, and the process returns to the state of waiting for the next effect type selection.

[Function of pitch response harmony sound generation system using vocoder]
In FIG. 2, the “pitch detection type” first mode for determining the pitch of the harmony sound based on the detection of the voice pitch Pm and the “pitch” for determining the pitch of the harmony sound based on the designation of the performance pitch Pi. The system having the “designated type” second mode has been described. However, a “pitch detection type” system dedicated to the first mode or a “pitch designated type” system dedicated to the second mode may be individually constructed. . In the case of a “pitch specification type” system, that is, a “harmonic sound generation system that generates a harmony sound in response to a performance pitch using a vocoder”, input is performed from the microphone 14 as in the first mode, that is, the pitch detection type. The pitch detection result of the generated audio signal Sm is not required, and instead, the performance pitch Pi supplied from the pitch information supply unit A is used as a reference for determining the harmony sound. The correction unit E2 can be omitted, and as described above, not only the tone Np is determined for the harmony by the chord information Cd, but also the harmony tone Np is determined by the pitch Ph by the harmony performance regardless of the chord information Cd. Harmony sound with unique sound quality by vocoder can be generated.

  FIG. 11 is a block diagram for explaining the function of “a harmony sound generating system for generating a harmony sound in response to a pitch using a vocoder” according to an embodiment of the present invention. As shown in FIG. 11, the pitch-designated harmony sound generation system using a vocoder has the following characteristics. For example, melody performance information Np (= pitch Pi) is input as first performance information, and chord information Cd is input separately from this, or harmony performance information Ph is input as second performance information. (A, B). On the other hand, in the chord supply mode [the operation mode when a chordal type other than the mark * is set in FIG. 8 (2)], the harmony sound determination unit G performs the pitch Pi and the first performance information Np. Based on the pitch of the chord information Cd, the pitch Nh of the harmony sound to be added to the first performance information Np is determined, or the harmony performance mode [Fig. 8 (2) "Harmony" type is selected. In the operation mode when set, the same pitch Nh is determined based on the pitch of the second performance information Ph. The tone generator H generates a first musical sound signal Sj based on the first performance information Np and a second musical sound signal Si based on the determined harmony pitch Nh. The vocoder unit J detects the frequency characteristic of the input audio signal Sm (J1), controls the frequency characteristic of the second musical sound signal Si in accordance with the detected frequency characteristic of the audio signal Sm, and the first musical tone signal Sj. A vocoder sound signal Sv with harmony is generated (J2). The first musical sound signal Sj and the vocoder sound signal Sv are output to the sound system 16. With this feature, it is possible to generate a vocoder sound Sv with an appropriate harmony with the melody sound Sj in a performance form in which a melody is played with the right hand and a chord or harmony is played with the left hand.

  This will be described as follows according to the part configuration (second mode) of the musical tone generation sequence of FIG. 6 (2). For example, when the harmony type of chord supply mode (code type other than *) is set and chord instruction operation is performed in the left key range for the melody performance in the right key range, the harmony sound determination unit G The pitch information Pi indicated by the melody performance information Np = Np1 is supplied from the high information supply unit A, and the chord information Cd based on the chord instruction operation is supplied via the chord information supply unit B. The part G determines the pitch Nh of the harmony sound to be added to the melody performance information Np = Np1, based on the pitch of the melody pitch Pi and the chord information Cd. Alternatively, if the harmony performance mode is selected and the harmony performance is set in the left key range against the melody performance in the right key range, the harmony sound determination section G will receive the harmony performance information. Since the pitch information Ph shown is supplied from the pitch information supply unit A, the harmony sound determination unit G, based on the pitch of the harmony performance pitch Ph, the pitch of the harmony sound to be added to the melody performance information Np. Nh is determined.

  The tone generator H outputs a musical sound signal Sj based on the melody performance information Np from the harmony sound determination unit G from the UK part (melody part) in both the chord supply mode and the harmony performance mode, and also the harmony sound part. That is, a tone signal Si having a predetermined tone color having a pitch Nh is generated and output to the vocoder part J by a dedicated part for applying the vocoder.

  Therefore, when generating a harmony sound using a vocoder, the user can play a vocoder with an appropriate harmony in a performance form in which the user plays a melody with the right hand and plays chords or harmonies with the left hand. Even novice users who cannot play overtones but can play chords with their left hand while playing a single melodic melody with their right hand can apply a vocoder to a performance with an appropriate harmony.

Here, when the harmonization mode in the system of FIG. 11 is arranged, as explained in FIG. 7 (1), by selecting the harmony type Th,
(1) When Th = “vocoder”, a performance with a specified keyboard part (for example, UK part Np1) is used as it is to generate a vocoder sound.
(2) Th = Cordal type [“Duet 1”, “Duet 2”,... (Excluding “Harmony 1” and “Harmony 2”])
(3) When Th = “Harmony 1” / “Harmony 2”, harmonies are added with the pitch played in the harmony part (Ph) (see the row marked with * in FIG. 8 (2)),
Whether to use one of these is set.

  In the mode (2) of the chord supply mode in which the harmony is given by the chord Cd, as described in FIG. 7 (2), the chord is applied to the top note of the part that plays the melody (for example, Np1: Pi of the UK part). Harmony sounds Nh: Nh1, Nh2,... In which the constituent sounds are changed to appropriate pitches are created, and a vocoder is applied to the musical tone signal Si of the created harmony sounds Nh. In the example of FIG. 7 (2), with respect to the same number of harmony sounds Nh, the chord constituent sounds are developed [open voicing (pitch is separated), closed voicing (pitch is close)]. , Etc.] (for example, three types Th = “trio 1” to “trio 3” for the harmony of the trio), and the code is obtained by designating one of the types Th. The development of the constituent sounds can be set in a predetermined manner. It should be noted that a harmony expansion change / edit screen may be displayed on the display 15 by a user operation, and the manner in which the chord component sound is expanded may be changed using this screen. May be registered as a new harmony type Th in the code supply mode and used again.

In addition, for the harmony performance mode (3) in which the harmony part performance (Ph) is applied, the harmony type Th marked with an asterisk (*) in the harmony type table consisting of the two right columns in FIG. As shown in the “Harmony content” column of the harmony performance performance mode of the cordal type Th = “Harmony 1” / “Harmony 2”,
(H1) Use the harmony part performance (Ph) as it is for the harmony performance sound Nh (Th = “Harmony 1”), or
(H2) The pitch of the harmony part performance (Ph) is shifted and used for the harmony performance sound Nh (for example, Th = “Harmony 2”).
A vocoder is applied to a tone signal Si having a predetermined tone color based on any of the harmony performance sound information Nh. In the example of FIG. 8 (2), for the pitch shift of (H2), a harmony sound that is shifted within an octave with respect to the top note of the part that plays the harmony melody (Np1 = Pi of the UK part) is used. Although only what to do is shown, various pitch shift modes can be prepared.

  FIG. 12 shows a further development example of the harmony performance sound in one embodiment of the present invention. That is, in the harmony type of the harmony performance mode shown in FIG. 12, Th = “Harmony 1” uses the harmony note Ph as it is in FIG. 8 (2), and Th = “Harmony 2” also in FIG. In the same way as in 2), it is instructed to shift the harmony note Ph so that it falls within one octave of the input sound (pitch of the UK part melody performance sound Np1) Pi. “Harmony 3” that shifts notes by −1 octave and “Harmony 4” that shifts notes by +1 octave. In addition, the harmony performance sound expansion type that adds harmony in the performance of the harmony part can be changed in the harmony expansion change / edit screen in the same way as the case of adding harmony with the type code Cd. Alternatively, the development content of the harmony performance sound changed by the user may be registered as a new harmony performance mode harmony type Th and used again.

In the system of FIG. 11, the performance information to be applied to the vocoder is as follows:
(A) Apply the vocoder only to the harmony sound of the chord sound or the harmony performance sound as described above,
(B) You may comprise so that a user can select by mode setting whether to apply a vocoder also to a melody sound itself.

In addition, the time for applying the vocoder in the chord supply mode using the chord information Cd is
(A) adding harmony only while the code instruction operation is actually performed (the code designation key is pressed),
(B) Even if this operation is stopped (even if the code designation key is released), the user can select whether to continue adding until there is a release operation or a next code instruction operation in the mode setting. May be.

[Harmony sound generation system using pitch conversion of speech]
In the harmony sound generation system according to one embodiment of the present invention, the input sound is processed using the pitch conversion of the input sound, so that the pitch harmony determined based on the pitch of the input sound or the pitch of the performance information. When generating sound, adjust the pitch of the pitch of the input sound or the pitch of the performance information to prevent the generation of harmony sound in a range not intended by the user, or automatically according to the harmony type setting You can preset an appropriate effect to the harmony sound.

  FIG. 13 is a functional block diagram of a harmony sound generation system using sound pitch conversion according to an embodiment of the present invention. In this system, instead of the vocoder unit J shown in FIG. 2, a pitch conversion unit N that converts the pitch of the audio signal Sm input from the audio input unit 14-7 is used, and the pitch conversion unit N determines the harmony sound. The vocoder-use harmony described with reference to FIG. 2 is that the pitch of the audio signal Sm is converted based on the pitch information Nh of the harmony sound determined by the part G, and the pitch-converted audio signal Sn is used as the harmony sound. Although it is different from the sound generation system (hereinafter referred to as “first system”), the rest is the same as the first system.

  That is, the sound source unit H generates a musical sound signal Sk corresponding to the performance sound information (performance information) Np supplied from the pitch information supply unit A, and the performance sound effect applying unit K applies a predetermined sound signal Sk to the musical sound signal Sk. Gives an effect. Further, the sound effect applying unit M provides a predetermined effect to the sound signal Sm input from the microphone 14 via the A / D conversion circuit 7. On the other hand, the pitch conversion unit N performs pitch conversion on the input audio signal Sm according to the harmony pitch Nh determined by the harmony sound determination unit G, and processes it into a harmony sound Sn having the pitch Nh. The sound effect imparting unit P imparts a predetermined effect to the harmony sound Sn. The performance sound, voice, and harmony sound signals Sa, Sc, and Se to which the respective effects are imparted by the respective effect imparting units K, M, and P are output to the sound system 16.

  Here, the setting mode of the effect imparted by the harmony sound effect imparting unit P and the sound effect imparting unit M is the same as that of the first system. For example, in the standard setting mode, the effect designation information from the harmony type setting unit C The effect according to the harmony type Th is given to each signal Se and Sc according to Ee and Ec. Further, the manner of determining the harmony pitch Nh in the harmony sound determination unit G is the same as that of the first system, but it will be briefly described as follows.

  In the first mode, the pitch adjustment unit E operates to detect the pitch of the input audio signal Sm (E1) as described in FIGS. 4 (1) and 5 and the detected pitch is the sound range setting unit D. If the detected pitch is not within the set range, the detected pitch is corrected so as to fall within the set range (E2), and the correct pitch Pm of the input audio signal Sm is sent to the harmony sound determination unit G as input sound information. The harmony sound determination unit G is input from the pitch information supply unit A via the chord information supply unit B according to the harmony type specified by the type specification information Th from the harmony type setting unit C as described in FIG. Is determined based on the chord information Cd to be performed and the pitch information Pi from the pitch adjustment unit E [however, in FIG. 7 (1) and FIG. 8 (2), the pitches Nh: Nh1, Nh2,. Harmony type Th = "There is no type and harmony sound corresponding to" Vocoder "," Harmony 1 "," Harmony 2 ").

  On the other hand, in the second mode, the pitch adjustment unit F operates, and as described in FIGS. 4B and 5, the performance sound information (the performance sound information (performance information) Np) that serves as a reference for the harmony sound ( For example, when the pitch of the melody performance information) Np1 is not within the set range of the range setting unit D, the pitch is corrected so as to fall within the set range, and the correct pitch Pi of the performance sound information is set to the harmony sound determination unit G. Send to. Alternatively, when the pitch of the performance information by the harmony performance is not within the set range of the range setting unit D, the pitch is corrected so as to fall within the set range, and the correct pitch Ph of the performance sound information is set to the harmony sound determination unit G. Send to. As described with reference to FIG. 7, the harmony sound determination unit G uses the chord information Cd input through the chord information supply unit B and the pitch adjustment unit F according to the harmony type Th specified from the harmony type setting unit C. The pitch of the harmony sound Nh: Nh1, based on the pitch information Pi from the pitch adjustment unit F, based on the pitch information Pi of the pitch, or according to the predetermined harmony type Th from the harmony type setting unit C. Nh2, ... are determined.

  In FIG. 13, the pitch of the harmony sound is determined based on the designation of the performance pitch Pi in the “pitch detection type” first mode in which the pitch of the harmony sound is determined based on the detection of the voice pitch Pm. Although the system having the second mode of the “pitch designation type” has been described, the “pitch detection type” system dedicated to the first mode or the “pitch designation type” system dedicated to the second mode has been described. Can be built individually.

[Various Embodiments]
The preferred embodiment of the present invention has been described above with reference to the drawings. However, this is merely an example, and various modifications can be made without departing from the spirit of the invention. For example, the formant filter characteristic control parameter in the vocoder unit (J) and the pitch conversion characteristic control parameter in the pitch conversion unit (N) may be changed according to the range set by the range setting unit (D). . For example, the parameters of the vocoder (J) and pitch converter (N) are initially set by the harmony type (Th), and the initial set values are corrected according to the sound range set by the sound range setting unit (D). You may do it. Thereby, it is possible to generate a harmony sound (Sv, Sn) having an appropriate sound quality suitable for the set sound range.

1 shows a hardware configuration block diagram of an electronic music apparatus in which a harmony sound generation system according to an embodiment of the present invention is constructed. FIG. 1 is a functional block diagram of a harmony sound generation system using a vocoder according to one embodiment of the present invention. It is a figure for demonstrating the function of a vocoder. It is a figure for demonstrating the adjustment effect | action of the audio | voice pitch and performance pitch by one Example of this invention. It is a flowchart showing the range determination process in the pitch adjustment part or pitch adjustment part of the harmony sound generation system by one Example of this invention. An example of the structure of a pronunciation sequence and part in one embodiment of the present invention is shown. An example of how to add a harmony sound in one embodiment of the present invention will be described. It is a figure for demonstrating the selection aspect of the effect kind in one Example of this invention. It is a flowchart showing the harmony selection process in the harmony sound generation system by one Example of this invention. It is a flowchart showing the effect kind selection process in the harmony sound production | generation system by one Example of this invention. It is a block diagram for explaining a function of “a harmony sound generation system that generates a harmony sound in response to a pitch using a vocoder” according to an embodiment of the present invention. It is a further development example of the harmony performance sound in one Example of this invention. 1 is a functional block diagram of a harmony sound generation system using pitch conversion according to an embodiment of the present invention.

Explanation of symbols

Th Type designation information indicating the set harmony type or its type,
Eb, Ee Harmony sound signal Sv, the effect designation information automatically set to Sn or its effect,
Ec Effect designation information automatically set to the audio signal Sm or its effect,
Sm; Sc voice input signal and voice output signal,
Pm voice pitch information used for reference input sound of Pm chord development or its pitch,
Performance pitch information used for reference input sound of Pi chord sound development or its pitch,
Cd; Ph Chord information (or its pitch) and harmony performance information (or its pitch),
Np: Np1, Np2, ... performance sound information,
Nh Harmony sound information indicating the harmony sound to be generated or its pitch,
Si; Sj musical tone signal converted into vocoder sound signal Sv and other musical tone signals,
Sa; Sb, Se Performance sound output signal and harmony sound output signal.

Claims (4)

A voice input means for inputting a voice signal;
A range setting means for setting a range;
Pitch detecting means for detecting the pitch of the voice signal input by the voice input means;
It is determined whether or not the pitch detected by the pitch detecting means falls within the range set by the range setting means. When the pitch enters the range, the pitch is valid, and when the pitch does not enter the range. Pitch adjusting means for correcting or invalidating the pitch so as to fall within the range; and
Harmony pitch determination means for determining the pitch of the harmony sound based on the pitch validated by the pitch adjustment means or the corrected pitch;
Harmony sound generation means comprising: harmony sound generation means for generating a harmony sound having a pitch determined by the harmony pitch determination means by processing or using a voice signal input by the voice input means apparatus. A voice input means for inputting a voice signal;
A range setting means for setting a range;
Pitch information supply means for supplying pitch information;
It is determined whether or not the pitch of the pitch information supplied by the pitch information supply means falls within the range set by the range setting means, and when the pitch enters the range, the pitch is validated. A pitch adjusting means for correcting the pitch to fall within the range when the pitch does not fall within the range;
Harmony pitch determination means for determining the pitch of the harmony sound based on the pitch validated or corrected by the pitch adjustment means;
Harmony sound generation means comprising: harmony sound generation means for generating a harmony sound having a pitch determined by the harmony pitch determination means by processing or using a voice signal input by the voice input means apparatus. A computer having a voice input means for inputting a voice signal and functioning as a harmony sound generator,
A range setting step for setting the range;
A pitch detection step for detecting the pitch of the audio signal input by the audio input means;
It is determined whether or not the pitch detected in the pitch detection step falls within the range set in the range setting step. When the pitch enters the range, the pitch is valid, and when the pitch does not enter the range. A pitch adjusting step for correcting or invalidating the pitch to fall within the range;
Harmony pitch determination step for determining the pitch of the harmony sound based on the pitch validated or corrected in the pitch adjustment step;
A harmony sound generation program for executing a procedure including a harmony sound generation step for generating a harmony sound having a pitch determined in the harmony pitch determination step by processing or using a voice signal input by the voice input means. A computer having a voice input means for inputting a voice signal and functioning as a harmony sound generator,
A range setting step for setting the range;
A pitch information supply step for supplying pitch information;
It is determined whether or not the pitch of the pitch information supplied in the pitch information supply step falls within the range set in the range setting step, and when the pitch enters the range, the pitch is valid. A pitch adjustment step for correcting the pitch to fall within the range when the pitch does not fall within the range; and
Harmony pitch determination step for determining the pitch of the harmony sound based on the pitch validated or corrected in the pitch adjustment step;
A harmony sound generation program for executing a procedure including a harmony sound generation step for generating a harmony sound having a pitch determined in the harmony pitch determination step by processing or using a voice signal input by the voice input means.

Images