JP5776205B2 - Sound signal generating apparatus and program - Google Patents

Description

  The present invention relates to a sound signal generation device and a program for generating one or more harmony sound signals by pitch-shifting an input sound signal. In particular, the present invention relates to a technique for reflecting a pitch fluctuation of an input audio signal with an arbitrary magnitude for each of a plurality of harmony sound signals. The sound signal generating apparatus and method of the present invention can be used in a human voice or musical instrument processing system attached to music-related equipment such as karaoke, an electronic musical instrument, an effector, or a personal computer.

  Conventionally, based on an input sound signal such as a musical instrument performance sound or human voice input by a user via a microphone or the like, the input sound signal is higher or lower by a predetermined pitch such as 3 degrees or 5 degrees. Automatically generate one or a plurality of harmony sound signals with different pitches, and reproduce them together with the input sound signal to simultaneously produce a lead sound (input sound) and a harmony sound (additional sound). Electronic music apparatuses and programs that can be used are known. For example, devices described in Patent Literature 1 and Patent Literature 2 shown below are examples of devices related to such devices.

  In the devices known in the art such as those described in Patent Document 1 and Patent Document 2, the fundamental frequency, that is, the fundamental frequency, that is, every predetermined section (or predetermined period) based on frequency information (pitch) obtained by frequency analysis of the input audio signal. Identifies the pitch corresponding to one of the musical pitch names, and the input voice signal (more specifically, the specified pitch) according to a predetermined pitch shift amount determined according to the pitch of the identified input voice signal. 1 of a predetermined target pitch (pitch corresponding to one of the musical pitch names) is obtained by pitch-shifting one cycle of waveform element data cut out and stored by a window function corresponding to the pitch. In addition, a plurality of harmony sound signals are separately generated as independent additional sounds. Furthermore, in the apparatus described in Patent Document 2, when a key-pressing sound (corresponding to an input audio signal) is bent up according to a wheel operation by the user, that is, a pitch bend value (also referred to as a pitch shift amount) is changed. In this case, it is disclosed that the pitch bend amount of the additional sound is corrected so that the additional sound (corresponding to the harmony sound signal) has a pitch that harmonizes with the chord. That is, in the apparatus described in Patent Document 2, a plurality of additional sounds composed of the constituent sounds of the chord are generated according to pre-designated chord information (specifically, chord names such as C major and A minor). It is like that.

Patent No. 2879948 Japanese Patent Laid-Open No. 06-202660

  However, in the conventional devices described in Patent Document 1 and Patent Document 2 described above, the generated harmony sound signal is a semitone unit corresponding to one of the musical pitch names. The height is always constant and does not have a subtle pitch change of less than a semitone (100 cents) (referred to as pitch fluctuation in this specification), so the musical expression of the harmony sound is mechanical and convenient. There was a problem of being bad. In particular, when the input audio signal has pitch fluctuation, there is a large gap between the musically rich expression of the lead sound (input sound) and the musically mechanical expression of the harmony sound (additional sound). Since this can occur, it is easy for the user to feel uncomfortable. Further, in order to generate a harmony sound with a pitch fluctuation (not reflecting the pitch fluctuation of the input audio signal), pitch control such as applying vibrato to the harmony sound signal with a constant pitch may be performed. . By the way, the harmony according to the user's own preference, for example, the harmony that is easy to listen to as a whole, with the pitch fluctuation on the bass side being small and stable compared to the treble side, and the atmosphere (majority, minority, etc.) according to the tune is clear It is necessary to generate multiple harmony sounds with different pitch fluctuations in order to finally generate harmonies that are applied to the sound, and harmonies with a sense of tension by adjusting the pitch fluctuation of the tension note. . However, in order to generate a plurality of harmony sounds having different pitch fluctuations in the past, parameter settings for vibrato control must be performed individually for each harmony sound, which is very troublesome for the user. there were. Therefore, there has been a demand for a sound signal generating device capable of generating one or more harmony sound signals reflecting the pitch fluctuations of the input sound signal with arbitrary magnitudes by a simple operation. Has not been proposed.

  The present invention has been made in view of the above points, and each of one to a plurality of harmony sound signals that are automatically generated based on the input sound signal, with a pitch change (pitch fluctuation) less than a semitone included in the input sound signal itself. It is an object of the present invention to provide a sound signal generation device and a program that can be easily reflected in an arbitrary size.

A sound signal generation device according to the present invention is a sound signal generation device that generates one or more harmony sound signals whose pitches are controlled following the pitch fluctuation of an input sound signal, and inputs the sound signals. An input means; a pitch detection means for sequentially detecting a specific pitch of the input voice signal; and a normalized pitch corresponding to a pitch name from the specific pitch; and the specific pitch is normalized. Difference generating means for obtaining difference pitch information related to the difference between the pitch and a plurality of pitches having pitches different from the normalized pitch are determined as target pitches of the plurality of sound signals to be generated. a target pitch determining means, for each of the plurality of target pitch, in order to adjust the degree of pitch sway to be added to the target pitch, setting means for setting a pitch adjustment information indicating the difference pitch additional percentage respectively , For each of the plurality of target pitch, comprising a sound signal generation means for generating a sound signal having a pitch obtained by modulating the target pitch according to the difference pitch information adding proportion is adjusted according to the pitch adjustment information.

Since the differential pitch information indicates pitch fluctuation (pitch fluctuation component) in the input audio signal, a sound signal (harmony sound) having a pitch obtained by modulating the target pitch according to the differential pitch information is generated. Thus, it is possible to generate a harmony sound signal reflecting the pitch fluctuation of the input sound signal (lead sound). Further, when a plurality of target pitches are modulated according to the difference pitch information in order to generate a plurality of harmony sound signals, the degree of pitch fluctuation added to the target pitch is adjusted for each of the plurality of target pitches. Therefore, pitch adjustment information indicating a difference pitch addition ratio is set for each of the plurality of target pitches, and the target pitch is modulated according to the difference pitch information in which the addition ratio is adjusted according to the pitch adjustment information. It was to generate a harmony sound signal with. Thus, the pitch swing in the input audio signal (fluctuation component of the pitch) is reproduced, while a harmony voice signal that does not inspire discomfort to the user without a wide gap between the lead note, several since the degree of pitch fluctuation of each target pitch can be adjusted independently Ki out that the degree of pitch shaking generates a different harmony voice signal, harmony users to suit their own tastes It can be easily generated and output.

  The present invention can be constructed and implemented not only as a device invention but also as a method invention. Further, the present invention can be implemented in the form of a program of a processor such as a computer or a DSP, or can be implemented in the form of a storage medium storing such a program.

According to the present invention, a difference pitch, which is a difference between the pitch of the input voice signal and the pitch corresponding to the specified pitch name, is obtained, and a plurality of target pitches are subtracted to generate a plurality of harmony sound signals. When adjusting according to the pitch, for each of the plurality of target pitches, in order to adjust the degree of pitch fluctuation added to the target pitch, pitch adjustment information indicating a differential pitch addition ratio is set, and the plurality of target pitches are set. For each target pitch, a harmony sound signal having a pitch obtained by modulating the target pitch in accordance with the differential pitch information in which the addition ratio is adjusted according to the pitch adjustment information is generated. Thus, the pitch swing in the input audio signal (fluctuation component of the pitch) is reproduced, while a harmony voice signal that does not inspire discomfort to the user without a wide gap between the lead note, several since the degree of pitch fluctuation of each target pitch can be adjusted independently, it is possible to the extent of the pitch shaking generates a different harmony voice signal, an effect that.

1 is a block diagram of a hardware configuration showing an embodiment of an overall configuration of a sound signal generation device (electronic music device) according to the present invention. It is a conceptual diagram which shows one Example of the data structure of a harmony table. It is a flowchart which shows the first half process of a harmony sound production | generation process. It is a flowchart which shows the latter half process of a harmony sound production | generation process. It is a conceptual diagram which shows one Example of the data structure of a difference pitch addition ratio table. It is a flowchart which shows a difference pitch addition ratio determination process. It is a flowchart which shows a rule 2 process. It is a flowchart which shows the rule 3 or 4 process. It is a flowchart which shows the rule 5 or 6 process. It is a conceptual diagram which shows the specific example for demonstrating a harmony sound production | generation process. It is a flowchart which shows an example of a pitch detection process.

  Embodiments of the present invention will be described below in detail with reference to the accompanying drawings.

  FIG. 1 is a hardware configuration block diagram showing an embodiment of the overall configuration of a sound signal generation device (electronic music device) according to the present invention. The electronic music apparatus shown in this embodiment is controlled by a microcomputer comprising a microprocessor unit (CPU) 1, a read only memory (ROM) 2, and a random access memory (RAM) 3. The CPU 1 controls the operation of the entire apparatus. A ROM 2, a RAM 3, an input operation unit 4, a display unit 5, a sound source 6, a communication interface (I / F) 7, and a storage device 8 are connected to the CPU 1 via a data and address bus 1D.

  The ROM 2 stores various control programs executed or referred to by the CPU 1, various data such as a harmony table (pitch determination table) shown in FIG. The RAM 3 is a working memory that temporarily stores various data generated when the CPU 1 executes a predetermined control program, or a memory that temporarily stores a control program currently being executed and related data. used. A predetermined address area of the RAM 3 is assigned to each function and used as a register, flag, table, memory, or the like.

  The input operation unit 4 is, for example, an input device such as a microphone for inputting a sound signal such as a human voice uttered by the user or a performance sound of a musical instrument performed by the user, and a performance instructing start / stop of performance (input of a sound signal) In addition to various operators such as a start / stop button and a switch for setting various parameters, a numeric keypad for inputting numeric data, a keyboard for inputting character data, or a mouse may be used. The input device is not limited to a microphone, and chord information necessary for generating a harmony sound signal is generated in response to a user operation, for example, a performance operator such as a keyboard, or chord information previously stored in the ROM 2 or the like. It may be a data input device such as a sequencer that supplies in order of performance. Further, the input operation unit 4 allows the user to edit / set “difference pitch addition ratio” (pitch adjustment information) defined in a difference pitch addition ratio table (see FIG. 5) described later to an arbitrary value. Including various setting operators.

  The display unit 5 is composed of, for example, a liquid crystal display panel (LCD), a CRT, or the like, and is pronounced based on a musical score and / or a generated harmony sound signal related to a lead sound generated based on an audio signal input from a microphone or the like. Various information such as a musical score relating to a harmony sound, a parameter setting state set by various operators, a list of various data stored in advance and a control state of the CPU 1 are displayed.

The tone generator 6 can simultaneously generate musical tone signals on a plurality of channels, and is provided via data and an address bus 1D, for example, a waveform signal temporarily buffered and stored as an input audio signal input via a microphone. Based on this, a sound signal of a lead sound is generated in a certain channel, and a harmony sound signal is generated in another channel based on a waveform signal in which the input sound signal is temporarily buffer-stored. As the sound source waveform of the lead sound, the waveform of the input audio signal temporarily stored in the buffer may be used as it is, or a waveform that is appropriately controlled based on the waveform of the input audio signal, such as pitch or timbre. May be used. As the sound source waveform for the harmony sound signal, a waveform based on the waveform of the input audio signal temporarily stored in the buffer may be used, or other appropriate sound source waveform may be used.
These signals generated from the sound source 6 are generated from a sound system 6A including an amplifier and a speaker. When the sound source 6 generates an input sound signal, a harmony sound signal, etc., for example, gender (type and depth of voice quality such as male voice and female voice), tremolo, volume, pan (localization), detune, reverb (reverberation) ) And the like can be imparted. Note that any conventional configuration may be used for the configuration of the sound source 6 and the sound system 6A. For example, as a sound source waveform generating or reproducing method in the sound source 6, any of various tone synthesis methods such as FM, PCM, physical model, formant synthesis, MP3, encoding method or data compression method may be adopted. Further, all or a part of the sound source 6 may be configured by dedicated hardware, or may be configured by software processing by the CPU 1 or DSP (Digital Signal Processor).

  The communication interface (I / F) 7 is an interface for transmitting and receiving various information such as a control program and various data between the apparatus and an external device (not shown). The communication interface 7 may be, for example, a MIDI interface, a LAN, the Internet, a telephone line, or the like, and may include both wired and wireless ones.

  The storage device 8 stores various information such as a prepared harmony table (see FIG. 2 described later) and various control programs executed by the CPU 1. Or you may enable it to memorize | store the input audio | voice signal input, the produced | generated harmony sound signal, etc. When the control program is not stored in the ROM 2 described above, the control program is stored in the storage device 8 (for example, a hard disk) and is read into the RAM 3 to store the control program in the ROM 2. It is possible to cause the CPU 1 to execute the same operation as in FIG. In this way, control programs can be easily added and upgraded. The storage device 8 is not limited to a hard disk (HD), but may be a flexible disk (FD), a compact disk (CD-ROM / CD-RAM), a magneto-optical disk (MO), or a DVD (Digital Versatile Disk). Any storage device may be used as long as it uses a storage medium in the form. Alternatively, a semiconductor memory such as a flash memory may be used.

In the above-described sound signal generation device (electronic music device), the input operation unit 4, the display unit 5, the sound source 6 and the like are not limited to those built in one device body, but each is configured separately, and has a MIDI interface, Needless to say, the apparatus may be configured to connect each device using a communication interface such as various networks.
The sound signal generating device (electronic music device) and the program according to the present invention may be any device / equipment such as a karaoke device, an electronic musical instrument, a personal computer, a portable communication terminal such as a mobile phone, or a game device. You may apply to. When applied to a portable communication terminal, not only when a predetermined function is completed with only the terminal, but also a part of the function is provided on the server side, and the predetermined function is realized as a whole system composed of the terminal and the server. You may make it do.

  The sound signal generation device (electronic music device) shown in FIG. 1 detects the pitch by analyzing the frequency of the input sound signal input via a microphone or the like (corresponding to one of the musical pitch names in the end). 1 to a plurality of target pitches (similarly to music pitch names) based on the specified pitch and chord information input from the keyboard or the like. And a harmony sound generation (addition) function for automatically generating a harmony sound signal having the determined target pitch.

  Here, the target pitch is prepared in advance based on a specific pitch corresponding to one of musical pitches obtained by frequency analysis of the input voice signal and chord information input from a keyboard or the like. According to the harmony table (pitch determination table) shown in FIG. 2, any one of twelve scale names (sound names) is determined (so-called chord input method). FIG. 2 is a conceptual diagram showing the data structure of the harmony table. However, here, a table referred to when the code name “C major” is designated as the code information is shown in FIG. 2A, and a table referred to when the code name “C minor” is designated as the code information. 2B, each table is data for generating a plurality of series of harmony sound signals composed of chord constituent chords corresponding to specific pitches obtained from the input sound signal. A configuration is shown as an example.

  In the harmony table, a plurality of tables, for example, one table for each code name is stored in advance, and one corresponding table is specified according to the code information (here, the code name). As can be understood from FIGS. 2 (A) and 2 (B), the harmony table is generated for each specific pitch (input pitch) corresponding to one of the musical pitches obtained by frequency analysis of the input voice signal. The target pitches of a plurality of harmony sequences composed of chord constituent sounds corresponding to the chord information are defined. However, in FIGS. 2A and 2B, the pitch names “C, C #, D, D #, E, F, F #, G, G #, A, A #, B” for the input pitches. In addition, the target pitch is also indicated by the pitch name.

  Regarding the pitch notation of the target pitch, for example, the target pitch “G” indicates a “G” tone in the same octave region as the input pitch, and the target pitch “C + 1” is one level higher than the input pitch. Indicates that the target pitch “C + 2” is a “C” sound in the octave region two levels higher than the input pitch. Although not described here, for example, the target pitch “E-1” is an “E” tone in the octave region one level lower than the input pitch, and the target pitch “E-2” is input. This indicates that the “E” sound is in the octave region two places below the pitch. Therefore, in the example of FIG. 2A, for example, when the input pitch is “E3”, the target pitches of the harmony sound signal are “G3”, “C4”, “E4”, and the input pitch is “ In the case of “A2 #”, the target pitch of the harmony sound signal is determined as “E3”, “G3”, and “C4”, respectively. In this embodiment, an example in which the octave region is divided between “C” and “B” is shown.

  In the above-described embodiment, the code input method for determining the pitch of the harmony sound signal based on the chord information (more specifically, the harmony table) is shown. However, the present invention is not limited to this. Other known methods for determining the pitch of the harmony sound signal may be used. For example, two pitches (for example, four semitones (long 3 degrees) and seven semitones (completely 5 degrees)) separated by a predetermined pitch predetermined with respect to the pitch of the input audio signal. Or the like) may be adopted to uniformly determine the pitch of the harmony signal (so-called fixed method).

  The electronic music apparatus shown in FIG. 1 can generate a harmony sound signal having a constant pitch (target pitch) without pitch fluctuations, and further, the input audio signal has pitch fluctuations (pitch fluctuations of less than 100 cents). In the case where it is provided, it is possible to generate a harmony sound signal in which the pitch fluctuation is reflected in any different magnitude. Therefore, a harmony sound generation function for generating one or more harmony sound signals that arbitrarily reflects the pitch fluctuation of the input audio signal will be described with reference to FIGS. 3 and 4 are flowcharts showing an embodiment of “harmonic sound generation processing” in which the above-described harmony sound generation function is realized by the CPU 1. However, for the sake of illustration, the first half of the harmony sound generation process is shown in FIG. 3, and the latter half of the process subsequent to the first half process of FIG. 3 is shown separately in FIG. This process is started in response to an instruction to start a performance in accordance with a user operation of the performance start / stop button, for example, and is repeatedly executed until an instruction to stop the performance is given. Hereinafter, the above process will be described with reference to FIG. 10 as appropriate. FIG. 10 is a conceptual diagram showing a specific example for explaining the harmony sound generation processing.

  As shown in FIG. 3, step S1 performs initial setting. In this initial setting, for example, a code buffer for storing chord information, a read buffer for storing input pitch, a note buffer for storing harmony pitch (target pitch), and a differential pitch between the input pitch and the target pitch are stored. Clearing various buffer areas such as a residual buffer, selecting a harmony pitch determination method (for example, the above-described chord input method or fixed method) according to a user operation, and the like. In step S2, the differential pitch addition ratio is edited and set. For example, the user edits / sets the differential pitch addition ratio table as shown in FIG. The difference pitch addition ratio (pitch adjustment information) is a parameter that determines the magnitude of pitch fluctuation to be added to the harmony pitch based on the difference pitch (here, as an example, it is indicated by a ratio of 0 to 100%, for example). A plurality of differential pitch addition ratios (pitch adjustment information) are defined as the differential pitch addition ratio table shown in FIG. This differential pitch addition ratio table is referred to when generating a harmony sound signal, so that a pitch adjustment (a pitch adjustment of less than 100 cents) for a plurality of harmony pitches can be performed individually (details will be described later). To do).

  Here, the difference pitch addition ratio table will be described. FIG. 5 is a conceptual diagram showing an example of the data configuration of the differential pitch addition ratio table. The difference pitch addition ratio table shown in FIG. 5 (A) is a predetermined pitch order (lowest order or first order) of the first sound, the second sound, the third sound (corresponding to numbers 1 to 3 in the figure). The difference pitch addition ratio (pitch adjustment information) to be applied to each harmony sound is defined in the order of higher order or other appropriate order). This differential pitch addition ratio is pitch adjustment information for adjusting a difference (corresponding to the magnitude of pitch fluctuation) between a specific pitch described later and a normalized pitch. In the example shown here, for example, when the harmony target pitches are specified as “E3”, “G3”, “C4”, each of these pitches “E3”, “G3”, “C4” is harmonized. The pitch fluctuation of the sound is “20%”, “25%”, and “15%” when the magnitude of the pitch fluctuation (that is, the difference pitch) of the input audio signal is 100%. Adjust the difference between the specific pitch and the normalized pitch.

On the other hand, the differential pitch addition ratio table shown in FIG. 5B is a differential pitch to be applied according to the magnitude (level) of pitch fluctuation added to the harmony sound (four levels of large, medium, small and none in this example). This defines the addition ratio (pitch adjustment information).
Note that which of the two tables is used is determined according to a harmony sound generation rule (described in detail later). In addition, when using the difference pitch addition ratio for each level, which level of the difference pitch addition ratio is applied to which harmony sound is previously associated by the harmony sound generation rule (details will be described later). Processing).
Needless to say, the differential pitch addition ratio is not limited to the above-described setting mode, and may be a specific numerical value, for example.

Returning to FIG. 3, step S3 determines whether or not the end of the performance has been detected. If it is determined that the end of the performance has been detected (YES in step S3), for example, an end process (step S28) such as muting the lead sound and / or the harmony sound being generated is executed, and the process ends. On the other hand, when it is determined that the end of the performance has not been detected (NO in step S3), it is determined whether or not the input sound has been turned off (step S4).
In parallel with the harmony sound generation process, a known pitch detection process as shown in FIG. 11 is performed sequentially (for example, at a constant interrupt cycle). Specifically, an input voice signal input via a microphone or the like is digitized by an A / D conversion circuit. In the pitch detection process, a specific pitch of the digitized input voice signal is “frequency detected”. A specific frequency signal (specific pitch information) is obtained by detection by the processing (S70). The specific pitch is a pitch before being rounded to a pitch name (normalized pitch). In the “vowel section detection” process (S71), the vowel section of the input speech signal is detected and the input speech signal is divided into different vowel sections. The “frequency detection” process may use any known technique such as the zero-cross method that is a well-known technique in the field of speech analysis, for example, and will not be described in detail here. When one (same) vowel interval lasts, it is considered that one input sound is on. In step S4 in FIG. 3, it is determined whether or not the input sound has been detected by checking whether or not the same vowel segment continues.
Returning to FIG. 3, when it is determined that the input sound is not detected to be off, that is, when the same vowel section continues, step S4 is NO and the process jumps to step S7. If it is determined that the input sound has been detected to be off, that is, if one vowel section has been completed, step S4 is YES, and a process for muting the lead sound generated based on the reproduction of the input sound signal is executed. Along with (Step S5), a process of muting the harmony sound generated based on the reproduction of the harmony sound signal is executed (Step S6).

  Step S7 determines whether or not a new input sound (input voice signal in a new vowel section) has been detected. If it is determined that a new input sound has not been detected, that is, if one vowel section has not ended (NO in step S7), the process jumps to the process in step S19 shown in FIG. When it is determined that a new input sound has been detected, that is, when one vowel section ends and becomes a different vowel section (YES in step S7), the frequency information of the input voice signal is quantized in units of pitch names. The input pitch is specified (step S8). That is, the frequency signal converted by the “frequency detection” process is “flattened” to flatten the change in the frequency signal (also referred to as smoothing). The flattened frequency signal is discretized into one of twelve-tone scale names (pitch names) every predetermined time by “class name detection” processing. In this way, the leveled frequency signal is rounded to a predetermined pitch corresponding to one of a plurality of musical pitch names defined in units of semitones (100 cents). It is specified as one of the pitches corresponding to the pitch names on music (this is called the input pitch or normalized pitch). The specified input pitch (normalized pitch) is stored in the read buffer. At this time, processing for storing the waveform element data for one period cut out by the window function corresponding to the detected normalized pitch is performed. The storage update of the waveform element data for one cycle may be performed sequentially.

  FIG. 10A shows an example of a specific pitch change of a series of input audio signals. In this example, the first vowel section in which the specific pitch of the input voice signal indicates a small pitch fluctuation (less than a semitone and about several to several tens of cents) with respect to the pitch of the pitch name “C”. To a second vowel section that shows a small pitch fluctuation (also about several to several tens of cents) on the basis of the pitch of the pitch name “D”. For example, when the phoneme of the lyrics sung by the input speech is “ai”, the vowel phoneme “a” of the syllable “a” corresponds to the first vowel segment, and the second vowel The vowel phoneme “i” of the syllable “I” corresponds to the section. When the frequency information of the input voice signal having such pitch fluctuation is quantized in units of pitch names (that is, a normalized pitch is detected), as shown in FIG. The input pitch (normalized pitch) can be specified for the pitch of the pitch name “C” and the pitch of the pitch name “D” for the second vowel interval, respectively.

  Step S9 generates a difference pitch (difference information related to the difference) between the frequency information (specific pitch) of the input voice signal and the specified input pitch (normalized pitch). The difference pitch (difference information related to the difference) is information indicating a cent value expression, that is, a pitch difference. An example of the difference pitch generated in each vowel section is shown in FIG. 10C (see solid line). As can be understood from this figure, the above processing generates a differential pitch that directly reproduces the pitch fluctuation of the input audio signal with the differential pitch “0” as a reference. In addition, since a pitch difference of 100 cents or more should be detected as another normalized pitch, the maximum value of the generated difference pitch should be less than 100 cents. In other words, a temporary pitch difference of 100 cents or more that occurs instantaneously in a very short time that could not be detected as a normalized pitch, for example, is ignored in the difference pitch (for example, rounded to 99 cents). Or a pitch difference of less than 100 cents just before that may be used). Of course, the present invention is not limited to this, and a temporary pitch difference of 100 cents or more may be generated as a difference pitch as it is. That is, the differential pitch is not limited to less than 100 cents. In FIG. 10C, the difference pitch addition ratio (limited to less than 100 cents) defined in the difference pitch addition ratio table (see FIG. 5) edited and set in step S2 is adjusted. The subsequent differential pitch is also indicated by a dotted line for convenience (however, only one is illustrated here).

In step S10, the input sound signal is reproduced to generate a lead sound. Note that the lead sound may be generated so that the pitch fluctuation in the original input sound signal can be reproduced exactly by sequentially reproducing the input sound signal temporarily stored in the buffer. Alternatively, using the stored and sequentially updated waveform element data for one period, the pitch fluctuation in the original input audio signal can be reproduced with the combination of the normalized pitch and the differential pitch. A lead sound may be generated. Alternatively, a lead sound may be generated using an arbitrary sound source waveform so that pitch fluctuations in the original input audio signal can be reproduced with a combination of the normalized pitch and the differential pitch. .
Step S11 determines whether or not the harmony pitch is determined based on the chord information, that is, whether or not the chord input method is selected as a method for determining the pitch of the harmony signal. When it is determined that the chord input method has not been selected (NO in step S11), a predetermined pitch (for example, 4 semitones above (long 3) is determined in advance from the input pitch (normalized pitch) according to the fixed method). (Degree), 7 semitones (completely 5 degrees), etc.) are determined as target pitches (step S14). The determined target pitch is stored in the note buffer. On the other hand, if it is determined that the code input method is selected (YES in step S11), it is determined whether the code information stored in the code buffer is valid (step S12).

  If it is determined that the code information is not valid, that is, if no code information is input and no code information is stored (NO in step S12), the process jumps to step S19 shown in FIG. When it is determined that the code information is valid, that is, when some code information is input and the code information is stored (YES in step S12), the code information stored in the code buffer and the read buffer are stored. Based on the stored input pitch, the corresponding harmony table stored in the ROM 2 or the storage device 8 is referred to, and one or more pitches are determined as the pitch (target pitch) of the harmony sound signal. (Step S13). For example, if the input chord information is “C major” and the harmony table shown in FIG. 2 is used, the target pitches in the first vowel section shown in FIG. 10A are “E”, “G”, “ C + 1 ”, and the target pitches in the second vowel section are“ G ”,“ C + 1 ”, and“ E + 1 ”. The determined one or more target pitches are stored in a note buffer. A step S15 executes a difference pitch addition ratio determination process.

  Here, the difference pitch addition ratio determination process (see step S15) will be described. FIG. 6 is a flowchart showing the difference pitch addition ratio determination process. As shown in FIG. 6, in this difference pitch addition ratio determination process, the difference pitch addition ratio is reflected in accordance with a pre-selected harmony sound generation rule (for example, six rules from rule 1 to rule 6 described later). The target harmony sound (target pitch) is specified, and the difference pitch addition ratio applied to the specified harmony sound is determined. Therefore, whether rule 1 is selected in step S31, rule 2 is selected in step S32, rule 3 or 4 is selected in step S33, or rule 5 or 6 is selected in step S34. The determination is made in stages, and the difference pitch addition ratio is determined according to the rule determined to be selected according to each determination.

  Here, as the rule, for example, six rules, specifically, one to a plurality of differential pitch addition ratios (that is, a differential pitch addition ratio table) set for each input pitch (normalized pitch) ( Rule 1 using FIG. 5 (A))), rule 2 using a higher difference pitch addition ratio for higher harmonic sounds, and a differential pitch having a higher ratio to the harmonic sound of the third harmonic of the harmony target pitch. Rule 3 using additional ratio, Rule 4 to stabilize the harmony sound of 3rd of the target pitch of harmony (Do not cause pitch fluctuation), Hit note for input pitch (normalized pitch) Rule 5, which uses a differential pitch addition ratio with a high ratio to the harmony sound of the pitch, for tension notes against the input pitch (normalized pitch) Upcoming pitch to stabilize the harmony sound (not to cause the pitch shaking) there is a rule 6. These rules are used for each purpose to generate a harmonious sound that is easy to listen to, a harmonious sound that has an atmosphere (majority or minor feeling, etc.) according to the tune, or a tension that is strong and weak. Prepared together.

  Returning to FIG. 6, if it is determined in step S31 that rule 1 is selected (YES in step S31), a difference pitch addition ratio table for each harmony sound as rule 1 processing (see FIG. 5A). ), For example, to identify the harmony sound to be reflected by reflecting the pitch fluctuation of the input signal in the order of low pitch (or high pitch), and set the difference pitch addition ratio for each of the identified harmony sounds. Determine (step S35). If it is determined in step S32 that rule 2 is selected (step S31 is NO and S32 is YES), a rule 2 process described later is executed (step S36). If it is determined in step S33 that rule 3 or 4 is selected (steps S31 and S32 are NO and S33 is YES), the later-described rule 3 or 4 process is executed (step S37). If it is determined in step S34 that rule 5 or 6 is selected (steps S31 to S33 are NO and S34 is YES), the later-described rule 5 or 6 process is executed (step S38). If it is determined in step S34 that rule 5 or 6 is not selected (steps S31 to S35 are all NO), the rule 1 process is applied to apply a difference pitch addition ratio table for each harmony sound (FIG. 5A). )))), For example, the harmony sounds to be reflected are specified in the order of low pitch (or high order), and the difference pitch addition ratio is determined for each specified harmony sound (step S39).

  The rule 2 process (see step S36) will be described with reference to FIG. FIG. 7 is a flowchart showing the rule 2 process. In step S41, it is determined whether or not the number of generated harmony sounds is one. If it is determined that the number of generated harmony sounds is one (YES in step S41), one harmony sound to be reflected is specified, and a difference pitch addition ratio table for each harmony sound (FIG. 5A) ) See))) to determine the difference pitch addition ratio for the identified harmony sound (step S46). When it is determined that the number of generated harmony sounds is not one (NO in step S41), it is determined whether the number of generated harmony sounds is two (step S42). When it is determined that the number of harmony sounds to be generated is two (YES in step S42), two harmony sounds to be reflected are specified, and a difference pitch addition ratio table for each level (FIG. 5B). The difference pitch addition ratio for each harmony sound is determined based on (see step S45). However, in this case, the difference pitch addition ratio with a level greater than that of the higher harmony sound and with no level for the other harmony sound with a lower pitch is determined.

  When it is determined that the number of generated harmony sounds is not two, that is, three or more (NO in step S42), an intermediate pitch between the highest pitch and the lowest pitch among the harmony sounds is determined. Specify (step S43). In step S44, all the harmony sounds to be reflected are specified, and the difference pitch addition ratio for each harmony sound is determined based on the level-specific difference pitch addition ratio table (see FIG. 5B) (step S44). However, in this case, for example, the specified intermediate level except for the harmony sound having the highest pitch, the level being higher than the harmony sound having the highest pitch, and having no level for the harmony sound having the lowest pitch. A difference pitch addition ratio is determined which is in the level with respect to the harmony sound having a pitch higher than the pitch, and with a small level with respect to the other harmony sounds.

  The rule 3 or 4 process (see step S37) will be described with reference to FIG. FIG. 8 is a flowchart showing rule 3 or 4 processing. In step S51, it is determined whether or not a harmony sound having a pitch corresponding to the third of the harmony target pitches is present. When it is determined that there is no harmony sound having a pitch of 3 (NO in step S51), the harmony sound to be reflected is determined based on the difference pitch addition ratio table for each harmony sound (see FIG. 5A). At the same time, the differential pitch addition ratio is determined (step S55). If it is determined that there is a harmony sound with a pitch corresponding to the third (YES in step S51), it is determined whether rule 3 is selected (step S52). When it is determined that the rule 3 is selected (YES in step S52), all the harmony sounds to be reflected are specified, and each difference pitch addition ratio table for each level (see FIG. 5B) is used. The difference pitch addition ratio for each harmony sound is determined (step S53). However, in this case, for example, a difference pitch addition ratio having a high level with respect to a harmony sound having a pitch of 3 degrees and a small level with respect to other harmony sounds is determined. If it is determined that rule 3 has not been selected, that is, rule 4 has been selected (NO in step S52), all the target harmony sounds are identified, and a differential pitch addition ratio table for each level (FIG. 5 (B)), the difference pitch addition ratio for each harmony sound is determined (step S54). However, in this case, for example, the difference pitch addition ratio in the level with respect to the other harmony sounds without the level with respect to the harmony sound having a pitch of 3 times is determined.

  The rule 5 or 6 process (see step S38) will be described with reference to FIG. FIG. 9 is a flowchart showing rule 5 or 6 processing. In step S61, it is determined whether or not there is a harmony sound having a pitch corresponding to the tension note with respect to the input pitch (normalized pitch). If it is determined that there is no harmony sound of the pitch corresponding to the tension note (NO in step S61), the harmony sound to be reflected is calculated based on the difference pitch addition ratio table (see FIG. 5A) for each harmony sound. At the same time, the differential pitch addition ratio is determined (step S65). If it is determined that there is a harmony sound with a pitch corresponding to the tension note (YES in step S61), it is determined whether or not rule 5 is selected (step S62).

  If it is determined that the rule 5 is selected (YES in step S62), all the harmony sounds to be reflected are specified, and each level is added based on the differential pitch addition ratio table for each level (see FIG. 5B). The difference pitch addition ratio for each harmony sound is determined (step S63). However, in this case, for example, a difference pitch addition ratio having a high level with respect to a harmonic sound having a pitch corresponding to a tension note and a low level with respect to other harmony sounds is determined. If it is determined that rule 5 has not been selected, that is, rule 6 has been selected (NO in step S62), all the target harmony sounds are identified, and a differential pitch addition ratio table for each level (FIG. 5 (B)), the difference pitch addition ratio for each harmony sound is determined (step S64). However, in this case, for example, the difference pitch addition ratio in the level with respect to the other harmony sounds without the level with respect to the harmony sound of the pitch corresponding to the tension note is determined.

  Returning to FIG. 3, in step S16, if there is a harmony sound being produced, it is muted. Step S17 compares the target pitch stored in the note buffer with the input pitch stored in the read buffer, and outputs a difference (corresponding to the pitch shift amount used to generate the harmony sound signal in the conventional apparatus). And one or more pitch shift amounts are calculated by adding one or more difference pitches to the obtained difference. However, the difference pitch to be added at this time is changed after adjusting the pitch change of the difference pitch stored in the residual buffer according to one or more difference pitch addition ratios (pitch adjustment information) determined in step S15. These are one or more differential pitches later. A step S18 pitch-shifts the input audio signal (specifically, the stored waveform element data of one cycle) based on the calculated one or more pitch shift amounts, and inputs the reference based on the target pitch. Reflecting the pitch fluctuation of the audio signal, one or more pitch-modulated harmony sound signals are generated and reproduced to generate one or more harmony sounds.

  In FIG. 10 (D), the pitch shift amount calculated according to the changed differential pitch after the change in pitch is adjusted according to the differential pitch addition ratio is indicated by a solid line, and the fluctuation according to the original differential pitch is indicated by a dotted line for reference. Shown (however, only one is illustrated here). As shown in FIG. 10D, for example, for the first vowel section determined to be the target pitch “E”, the target pitch “G” is determined based on the pitch shift amount “+400”. For the second vowel section, a pitch shift amount in which the pitch fluctuates up and down with respect to the pitch shift amount “+500” is obtained. As shown in FIG. 10E, the input speech signal (stored waveform element data) is pitch-shifted in each section in accordance with a plurality of pitch shift amounts obtained for each vowel section in this manner. It is possible to generate a plurality of harmony signals having different pitch fluctuations while reflecting the pitch fluctuations of the input audio signal.

  As can be understood from FIG. 10 (E), conventionally, a plurality of harmony sound signals having constant pitches such as “E”, “G”, “C + 1”, and “G”, “C + 1”, “E + 1” are generated. Although generated (indicated by dotted lines), in the present embodiment, a plurality of harmony sound signals having different pitch fluctuations reflecting the pitch fluctuations of the input audio signal are generated. In other words, depending on the difference pitch addition ratio determined for each individual harmony sound, the magnitude of the pitch fluctuation of each of the multiple harmony sound signals can be made larger or smaller than the magnitude of the pitch fluctuation of the input audio signal. It can be adjusted. For example, when the difference pitch addition ratio is 100%, the pitch fluctuation magnitude of the harmony sound signal is set in advance to be the same as the pitch fluctuation magnitude of the input audio signal, and the difference pitch addition ratio is 100%. When set to a smaller value, the pitch fluctuation magnitude of the harmony sound signal appears smaller than the pitch fluctuation magnitude of the input audio signal as the differential pitch addition ratio approaches 0%. A harmony sound signal having a constant pitch that does not have the same pitch fluctuation as in FIG.

  Returning to FIG. 4, in step S19, it is determined whether or not the chord information input by the user operating the keyboard or the like (or may be automatically provided along with the karaoke accompaniment) is acquired. judge. If it is determined that the code information has not been acquired (NO in step S19), the process jumps to the process in step S2. If it is determined that the code information has been acquired (YES in step S19), the code information is extracted and stored in the code buffer (step S20). A step S21 decides whether or not the input pitch stored in the read buffer is valid. If it is determined that the input pitch is not valid (NO in step S21), the process returns to step S2 shown in FIG. If it is determined that the input pitch is valid (YES in step S21), it is determined whether or not a chord input method is selected as a method for determining the pitch of the harmony sound signal (step S22). If it is determined that the code input method is not selected (NO in step S22), the process jumps to the process in step S2.

  If it is determined that the chord input method is selected (YES in step S22), the ROM 2 or the storage device 8 is based on the chord information stored in the chord buffer and the input pitch stored in the read buffer. Referring to the corresponding harmony table stored in (1), the pitches (target pitches) of one or more harmony sound signals are determined (step S23). A step S24 executes a difference pitch addition ratio determination process (see FIG. 6). In step S25, if there is a harmony sound being pronounced, it is muted. In step S26, the target pitch stored in the note buffer and the input pitch stored in the read buffer are compared to obtain a difference (conventional pitch shift amount), and the difference pitch is added to the obtained difference. To calculate the pitch shift amount. However, the difference pitch to be added at this time is the difference pitch after the change after adjusting the pitch change of the difference pitch stored in the residual buffer according to the preset difference pitch addition ratio, as already described. It is. Step S27 shifts the pitch of the input voice signal (specifically, the stored waveform element data of one cycle) based on the calculated pitch shift amount, and the pitch fluctuation of the input voice signal based on the target pitch. A harmony sound signal reflecting the above is generated and reproduced to produce a harmony sound.

  As described above, in the present invention, the pitch of the voice signal detected by analyzing the input voice signal and the pitch name specified for each predetermined section of the input voice signal according to the pitch detection are supported. A difference pitch which is a difference from any of the pitches obtained is obtained. Further, the difference pitch is changed according to any different difference pitch addition ratio to generate a plurality of different difference pitches. Then, with respect to the pitch shift amount (conventional pitch shift amount) required when the input audio signal is pitch-shifted to the pitch of one or more harmony sounds determined according to the specified pitch, The pitch shift amount is changed so as to add the pitch fluctuation due to the changed differential pitch. Since the difference pitch indicates a pitch fluctuation (pitch fluctuation component) based on a certain pitch originally included in the input audio signal, the pitch change to which the fluctuation due to the difference pitch after the change is added is added. The later pitch shift amount takes into account the pitch fluctuation that the input audio signal had. Therefore, if the input audio signal is pitch-shifted based on the pitch shift amount after the pitch change, one or more harmony sound signals having pitch fluctuations are generated based on the determined one or more pitches. Can do. In this way, the user can reflect the pitch fluctuation of the input audio signal in a plurality of harmony sound signals with an arbitrary magnitude by a simple operation, and it is the same as the input audio signal that does not make the user feel uncomfortable. It becomes possible to generate a harmony sound signal having a pitch fluctuation of.

As mentioned above, although an example of embodiment was demonstrated based on drawing, this invention is not limited to this, It cannot be overemphasized that various embodiment is possible. For example, as in the prior art, the input sound signal is pitch-shifted by the difference obtained by comparing the target pitch and the input pitch (conventional pitch shift amount) as in the prior art, so that there is no constant pitch (target pitch). The pitch fluctuation of the input audio signal is generated by performing a pitch modulation control in which a difference pitch adjusted by the difference pitch addition ratio is simply generated. You may make it reflect in a harmony sound signal.
In this specification, “pitch fluctuation” or pitch modulation is not limited to periodic pitch changes such as vibrato, but may include non-periodic pitch changes such as bend up and bend down. In addition, a fine pitch change that cannot be recognized by the user as a performance style expression may be included.

Note that the chord information input to generate the harmony sound signal is detected from the input information input in response to a user operation from a performance operator such as a keyboard connected to the apparatus or the apparatus as described above. May be obtained in a format in which chord names are sequentially input, or may be automatically provided along with karaoke accompaniment.
Needless to say, when chord input is performed by a performance operation by the user, chord detection is performed from the key depression state. However, the chord detection method at this time is a so-called fingered system in which chords are designated by depressing all keys of actual chord constituent sounds, and one to three keys are depressed based on a predetermined rule. Thus, any method such as a so-called single finger method for designating a code may be used. Alternatively, a method of designating the route and type of each code by a predetermined switch operation arranged on the operation panel may be used.

Note that the pitch detection result of the input sound signal is not limited to determining the pitch of the harmony sound signal by directly using the pitch detection result of the sound signal input as described above. For example, the pitch detection result of the input sound signal is 1 octave or 3 semitones. You may make it determine the pitch of a harmony sound signal using what converted pitch, such as going up and down only a predetermined pitch.
In addition, although the input audio | voice signal used as the origin for producing | generating a harmony sound signal in the Example mentioned above was demonstrated to the example of the user's audio | voice input via the microphone, it is not restricted to this. For example, a musical instrument performance sound input via a microphone may be used, or a stored audio signal or an audio signal distributed from the outside as appropriate may be used.

  Note that the pitch adjustment information is not limited to the predetermined differential pitch addition ratio (%) as in the above-described embodiment, and the differential pitch addition ratio (%) may be calculated by calculation. In that case, some rules that are the basis of calculation may be prepared in advance and selected from them. Furthermore, after the pitch adjustment information is automatically calculated, the user may be able to edit the calculated pitch adjustment information. In addition, the magnitude of pitch fluctuation of the input audio signal may be automatically detected, and pitch adjustment information for each harmony sound may be determined according to the detected magnitude of pitch fluctuation. Furthermore, the user himself / herself may be able to specify the harmony sound to be pitch adjusted based on the pitch adjustment information.

Note that the number of harmony sounds to be generated may be designated in advance. For example, in the above-described embodiment, the pitch for generating the three harmony sounds is determined based on the harmony table (see FIG. 2A) specified according to the chord information. In the case of such a table, for example, when two sounds are specified as the number of sounds, the pitch is determined by combining any two sounds of the three sounds such as the two sounds on the low sound side (or the high sound side) of the table. Good.
It should be noted that a plurality of difference pitch addition ratio tables of the same type (for each harmony sound / by level, etc.) may be prepared, and selected from among them and switched. Such table switching may be performed at any time during the course of a song.
When using the difference pitch addition ratio table for each level (see FIG. 5B), the same difference pitch addition ratio is not applied when there are a plurality of harmony sounds even at the same level, and the values are slightly different. It is good to be able to adjust (for example, to shift 2).

DESCRIPTION OF SYMBOLS 1 ... CPU, 2 ... ROM, 3 ... RAM, 4 ... Input operation part, 5 ... Display part, 6 ... Sound source, 6A ... Sound system, 7 ... Communication interface, 8 ... Storage device, 1D ... Data and address bus

Claims (5)

A sound signal generation device that generates one or more harmony sound signals whose pitches are controlled following the pitch variation of an input sound signal,
An input means for inputting an audio signal;
Pitch detection means for sequentially detecting a specific pitch of the input voice signal and detecting a normalized pitch corresponding to a pitch name from the specific pitch;
Difference generating means for obtaining differential pitch information related to the difference between the specific pitch and the normalized pitch;
Target pitch determining means for determining a plurality of pitches having different pitches relative to the normalized pitch as target pitches of a plurality of sound signals to be generated;
For each of the plurality of target pitches, in order to adjust the degree of pitch fluctuation added to the target pitch, setting means for respectively setting pitch adjustment information indicating a difference pitch addition ratio ;
A sound signal comprising sound signal generating means for generating , for each of the plurality of target pitches, a sound signal having a pitch obtained by modulating the target pitch in accordance with the difference pitch information in which the addition ratio is adjusted according to the pitch adjustment information. Generator. The sound signal generation unit generates pitch information indicating a pitch obtained by modulating the target pitch according to the differential pitch information in which an addition ratio is adjusted according to the pitch adjustment information, and based on the generated pitch information, the sound signal is generated. The sound signal generation device according to claim 1, wherein the sound signal generation device generates a signal. The sound signal generating means generates a sound signal having the target pitch, and modulates the generated sound signal having the target pitch according to the differential pitch information whose addition ratio is adjusted according to the pitch adjustment information. The sound signal according to claim 1, wherein the sound signal having a pitch obtained by modulating the target pitch according to the difference pitch information in which an addition ratio is adjusted according to the pitch adjustment information is generated. Signal generator. The sound signal generating means outputs a sound signal having a waveform characteristic of the input sound signal for each of the plurality of target pitches according to the difference pitch information in which an addition ratio is adjusted according to the pitch adjustment information. 4. The sound signal generating apparatus according to claim 1, wherein the sound signal generating apparatus generates the sound with a pitch modulated . A computer-executable program for generating one or more harmony sound signals whose pitches are controlled following the pitch fluctuation of an input audio signal, the program being stored in a computer,
Input audio signal,
A step of sequentially detecting a specific pitch of the input voice signal and detecting a normalized pitch corresponding to a pitch name from the specific pitch;
Obtaining differential pitch information related to the difference between the specific pitch and the normalized pitch ;
Determining a plurality of pitches having different pitches relative to the normalized pitch as target pitches of a plurality of sound signals to be generated;
For each of the plurality of target pitches, in order to adjust the degree of pitch fluctuation to be added to the target pitch, a procedure for setting each pitch adjustment information indicating a difference pitch addition ratio ,
Wherein for each of a plurality of target pitch, a program for executing the steps of generating a sound signal having a pitch obtained by modulating the target pitch according to the difference pitch information adding proportion is adjusted according to the pitch adjustment information.

Images