JP4613923B2 - Musical sound processing apparatus and program - Google Patents

Description

  The present invention relates to a technique for generating music data that reflects the tone of music.

Conventionally, a technique for automatically generating a desired accompaniment sound by selectively using a plurality of automatic accompaniment data designating the form of the accompaniment sound of the music has been proposed. For example, Patent Document 1 discloses a technique of selecting automatic accompaniment data according to a beat point mode (rhythm) designated by a user and outputting the data together with a code detected based on the content of a key pressed to the keyboard. Yes.
JP-A-10-222166

  However, the technique disclosed in Patent Document 1 has a problem that the user is required to perform a complicated task of designating a beat point mode that matches the musical tone. In addition, when a user unfamiliar with music specifies the mode of beat points, automatic accompaniment data that appropriately reflects the tone of the music may not be selected. Against the background of the above circumstances, an object of the present invention is to solve the problem of making it unnecessary to specify beat points by the user.

In order to solve the above-described problems, a musical sound processing apparatus according to the present invention stores a plurality of accompaniment mode data (for example, the storage circuit 20 in FIG. 1) for specifying a mode of an accompaniment sound, and a musical tone of music. A feature amount extracting unit that sequentially extracts feature amounts from the musical tone signal shown, a beat point detecting unit that detects beat points from the musical tone signal, and a plurality of unit sections are defined based on the beat points detected by the beat point detecting unit. The storage means stores the section defining means, the comparison means for comparing the feature quantity of the accompaniment sound indicated by each of the plurality of accompaniment mode data and the feature quantity specified by the feature quantity extraction means for the musical tone signal in the unit section. A selection unit that sequentially selects accompaniment mode data of accompaniment sounds similar to a musical tone signal in a unit section among a plurality of accompaniment mode data for each unit section based on a comparison result by the comparison unit, and a selection unit, Selected companion The unit data including the mode data or the identifier of the accompaniment mode data and an output means for outputting for each unit section. According to the above configuration, since the beat point detecting means detects the beat point from the musical tone signal, the labor of the user who designates the beat point is reduced. Also, since the accompaniment mode data is selected based on the result of comparison between the feature amount of the accompaniment sound and the feature amount of the musical tone signal indicated by each of the accompaniment mode data, for example, the timing of the accompaniment sound and the musical tone specified by the accompaniment mode data Compared to a configuration in which accompaniment mode data is selected based only on the result of comparison with the timing of the tone of the signal, it is possible to generate accompaniment tone unit data whose tone is similar to the tone signal.
In a preferred aspect of the present invention, the feature quantity extracting means sequentially extracts the feature quantity of the musical tone signal for each frame, and the comparing means includes the feature quantity for each frame of the accompaniment sound indicated by each of the plurality of accompaniment style data. The feature amount of each frame of the musical tone signal in the unit section is compared. In a preferred aspect of the present invention, the selection means includes accompaniment processing means for specifying a spectrogram of the accompaniment sound for each of the plurality of accompaniment aspect data, and the feature amount of the spectrogram specified by the accompaniment processing means for each accompaniment aspect data The accompaniment mode in which the spectrogram of the accompaniment sound is similar to the spectrogram of the tone signal in the unit interval among a plurality of accompaniment mode data based on the result of the comparison means comparing the characteristic amount of the spectrogram of the musical sound signal in the unit interval Select data.

A musical tone processing apparatus according to a preferred aspect of the present invention includes a pitch specifying unit that specifies a pitch (for example, a chord name or a bass tone) corresponding to a musical tone signal for each unit section, and the output unit includes: Unit data including the selected accompaniment mode data or the identifier of the accompaniment mode data and the pitch specified by the pitch specifying means is output. According to the above aspect, since the unit data including the pitch corresponding to the musical tone signal is output, the musical tone (accompaniment tone) generated based on the unit data can be approximated to the musical tone tone. It is.
In a further preferred aspect, the pitch specifying means includes a chord specifying means for specifying a chord of each unit section and a base specifying means for specifying a base sound of each unit section, and the output means is specified by the chord specifying means. The chord and the bass sound specified by the bass specifying means are included in the unit data and output. According to this aspect, unit data indicating an accompaniment sound that sufficiently reflects the tone of the tone signal can be generated.

In a preferred aspect of the present invention, the beat point detecting means detects, as a beat point, a time point at which a component belonging to each of a plurality of frequency bands corresponding to performance sounds of different types of musical instruments is generated. According to the above aspect, for example, beat points can be detected with high accuracy compared to a configuration in which a time point when a component belonging to one frequency band is generated is used as a beat point.

In a preferred aspect of the present invention, the selection means includes an accompaniment processing means for specifying the characteristic amount of the accompaniment sound indicated by each of the plurality of accompaniment aspect data, a characteristic amount specified by the accompaniment processing means, and a musical sound signal within the unit section. Comparing means for comparing with the feature amount is selected, and accompaniment mode data of accompaniment sounds similar to the musical tone signal in the unit interval is selected based on the comparison result by the comparing means. In a further preferred aspect, the accompaniment processing means is configured to perform an accompaniment sound specified by the accompaniment aspect data so that each accompaniment sound specified by the accompaniment aspect data corresponds to a beat point detected by the beat point detecting means on the time axis. The position on the time axis is adjusted (for example, the expansion / contraction process in FIG. 6). According to this aspect, the accompaniment mode data is adjusted so that each accompaniment sound specified by the accompaniment mode data corresponds to the beat point detected by the beat point detecting means on the time axis. It is possible to precisely compare the feature amount of the musical sound and the feature amount of the musical sound signal.

  The musical sound processing apparatus according to a preferred aspect of the present invention includes a configuration determining unit that detects a repeated section in which a performance is repeated in the music, and the selecting unit is the same accompaniment mode for each repeated section detected by the configuration determining unit. Select data. In the above aspect, there is an advantage that the process of selecting the accompaniment aspect data by the selection means for the repetitive section is reduced and the musical uniformity of each repetitive section is maintained.

The musical sound processing apparatus according to the present invention is realized by hardware (electronic circuit) such as a DSP (Digital Signal Processor) dedicated to each process, and a general-purpose arithmetic processing apparatus such as a CPU (Central Processing Unit) and a program It is also realized through collaboration with. The program according to the present invention includes a feature amount extraction process for sequentially extracting feature amounts from a musical sound signal indicating a musical tone of music, a beat point detection process for detecting a beat point from the musical sound signal, and a beat detected by the beat point detection process. About the musical tone signal in the unit section by the section defining process for defining a plurality of unit sections based on the points, and the accompaniment sound feature amount and the feature amount extraction process indicated by each of the plurality of accompaniment aspect data for specifying the accompaniment sound aspect Comparing processing for comparing the specified feature value, and accompaniment sound data of accompaniment sounds similar to the musical tone signal in the unit interval among a plurality of accompaniment mode data, for each unit interval based on the result of the comparison processing The computer is caused to execute selection processing for selecting sequentially and output processing for outputting unit data including the accompaniment mode data selected in the selection processing or the identifier of the accompaniment mode data for each unit section. With the above program, the same operations and effects as the musical sound processing apparatus according to the present invention are exhibited. The program of the present invention is provided to the user in a form stored in a computer-readable recording medium and installed in the computer, or is provided from the server device in the form of distribution via a communication network. To be installed.

The present invention is also specified as a method for processing musical sounds. The musical sound processing method of the present invention is a method using storage means for storing a plurality of pieces of accompaniment mode data for specifying the mode of the accompaniment sound, wherein the feature amount is sequentially extracted from the musical sound signal indicating the musical tone of the music. Each of the extraction process, the beat point detection process for detecting the beat point from the musical sound signal, the section defining process for defining a plurality of unit sections based on the beat points detected in the beat point detection process, and the plurality of accompaniment mode data Including a comparison process in which the feature amount of the accompaniment sound shown and the feature amount extraction means compare the feature amount specified for the musical tone signal in the unit section, and among the plurality of accompaniment mode data stored in the storage means , the musical tone in the unit section accompaniment mode data of the accompaniment tone similar to the signal, based on the results of the comparison process, a selection process that sequentially selects each unit interval, the identifier of the accompaniment mode data or the accompaniment mode data selected in the selection process including The unit data and an output step of outputting for each unit section. According to the above method, the same operation and effect as the musical sound processing apparatus according to the present invention are exhibited.

<A: Configuration of musical tone processing device>
FIG. 1 is a block diagram showing a configuration of a musical tone processing apparatus according to one embodiment of the present invention. As shown in the figure, a signal generation device 12 and an output processing device 14 are connected to the musical tone processing device 100.

  The signal generation device 12 is a means for generating a musical sound signal V indicating a temporal waveform of a musical tone (performance sound) of music. A reproduction apparatus that sequentially acquires data from a recording medium (for example, a music CD) that records musical sounds of music and generates and outputs a musical sound signal V is suitably employed as the signal generating apparatus 12. The musical tone signal V generated by the signal generating device 12 is supplied to the musical tone processing device 100.

  The musical tone processing apparatus 100 generates and outputs musical tone data DOUT from the musical tone signal V. The musical sound data DOUT is a data string that specifies accompaniment sounds similar to the music indicated by the musical sound signal V in time series. That is, the musical tone processing apparatus 100 according to the present embodiment functions as a device that encodes the musical tone signal V into the musical tone data DOUT.

  The musical tone data DOUT generated by the musical tone processing device 100 is supplied to the output processing device 14. The output processing device 14 is a device that outputs sound waves according to the musical sound data DOUT. The output processing device 14 may be directly connected to the musical tone processing device 100, or may be indirectly connected to the musical tone processing device 100 via a communication network such as the Internet. Further, the output processing device 14 and the musical tone processing device 100 may be integrated.

  As shown in FIG. 1, the musical sound processing apparatus 100 includes a storage circuit 20 and a control circuit 30. The storage circuit 20 stores a program executed by the control circuit 30 and various data used by the control circuit 30. Any storage device such as a semiconductor storage device or a magnetic storage device is employed as the storage circuit 20. The control circuit 30 is an arithmetic processing unit such as a CPU that functions as each element (functional body) in FIG. 1 by executing a program. The control circuit 30 is also realized by an electronic circuit such as a DSP dedicated to processing the musical tone signal V. Further, each element of the control circuit 30 illustrated in FIG. 1 may be distributed and arranged in a plurality of integrated circuits.

  The feature quantity extraction unit 32 in FIG. 1 is means for extracting N types of feature quantities F (FS, FW) for each of a plurality of frames obtained by dividing the musical sound signal V on the time axis (N is a natural number). Successive frames overlap on the time axis. The feature amount extraction unit 32 of this embodiment includes a frequency analysis unit 321, a spectrum feature amount extraction unit 323, and a waveform feature amount extraction unit 325.

  The frequency analysis unit 321 specifies the power spectrum Q of each frame by executing frequency analysis including FFT (Fast Fourier Transform) processing for each frame of the musical sound signal V. The spectrum feature amount extraction unit 323 extracts the feature amount FS of the power spectrum Q specified by the frequency analysis unit 321. For example, as shown in FIG. 2, the spectrum feature amount extraction unit 323 extracts a combination S1 of each frequency at which a peak appears in the power spectrum Q and the intensity of the peak (hereinafter referred to as “frequency bin”) as a feature amount FS. . Further, the spectrum feature amount extraction unit 323 extracts an HPCP (Harmonics Pitch Class Profile) from the power spectrum Q. As shown in FIG. 2, HPCP is a feature value FS indicating the intensity of each of the twelve semitones (C, C #, D,..., A #, B) constituting one octave.

  The waveform feature quantity extraction unit 325 in FIG. 1 extracts the waveform feature quantity FW on the time axis of the musical sound signal V for each frame. For example, as shown in FIG. 3, the waveform feature amount extraction unit 325 calculates the intensity (instantaneous value) W2 of the envelope W1 of the musical sound signal V for each frame, and differentiates the time series of the intensity W2 in each frame. The gradient W3 is calculated as the feature value FW. Instead of the gradient W3, a moving average of the intensity W2 of each frame may be calculated.

  The N types of feature values F (FS, FW) extracted by the feature value extraction unit 32 are stored in the feature value storage unit 34 of FIG. FIG. 4 is a conceptual diagram showing how the feature quantity F is stored in the feature quantity storage unit 34. As shown in the figure, N types of feature quantities F1 to FN extracted for each frame of the musical tone signal V over the entire musical piece are held in the feature quantity storage unit 34. However, the configuration may be such that only the feature value F of some frames of the music is stored in the feature value storage unit 34.

  The beat point detection unit 52 in FIG. 1 is means for detecting the beat point of the music from the feature value F of the musical sound signal V. More specifically, the beat point detection unit 52 detects a time point when a percussion instrument performance sound is generated as a beat point. In this embodiment, the beat point P is detected based on the performance sounds of the hi-hat cymbal (HH), snare drum (SD), and bass drum (BD).

  FIG. 5 is a conceptual diagram illustrating a specific example of an operation in which the beat point detection unit 52 detects a beat point. As shown in the figure, the beat point detection unit 52 identifies a remarkable attack part A (A1, A2,...) Of each musical tone in the music. For example, among a plurality of frames in which the gradient W3 (feature value FW) illustrated in FIG. 3 increases steeply, the interval between beat points in a normal music piece (for example, 1 corresponding to 30 BPM (Beat Per Minute) to 240 BPM in 4 beats) A frame having a period in the range of seconds to 8 seconds) is selected as the attack part A.

  Further, the beat point detection unit 52 determines the beat point by analyzing the power spectrum Q (frequency bin S1 included in the feature value FS) of each frame corresponding to the attack part A. More specifically, the beat point detection unit 52 detects an attack part A generated in a plurality of preselected frequency bands among the plurality of attack parts A as a beat point. Each of the plurality of frequency bands corresponds to a sound range of a different type of percussion instrument.

  FIG. 5 schematically shows a spectrogram G0 of the tone signal V. In each frame corresponding to the attack parts A3 and A7 in FIG. 5, the intensity in the frequency band BSD corresponding to the sound range of the snare drum is high. Therefore, the beat point detector 52 determines that the attack parts A3 and A7 are caused by the performance of the snare drum. Similarly, the beat point detector 52 determines that the attack parts A1 and A5 having a high frequency band BBD (band including the bass drum sound band) are caused by the performance of the bass drum, and the frequency band BHH (high hat) It is determined that the attack parts A1 to A7 having a high intensity of the band including the cymbal sound range are caused by the performance of the hi-hat cymbal.

  On the other hand, there is a musical tendency between the order of each beat point included in the measure of the music and the type of percussion instrument played at each beat point. For example, in a 4-beat music, the snare drum tends to be played on the second and fourth beats, and the bass drum tends to be played on the first and third beats. The beat point detection unit 52 determines the beat point P from the plurality of attack parts A on the condition of the musical tendency as described above. For example, when focusing on the attack parts A3 and A7 corresponding to the performance of the snare drum in FIG. 5, the beat point detecting part 52 determines the attack part A3 as the second beat point P of the measure and the attack part A7 as the measure. The fourth beat point P is determined. Similarly, the beat point detector 52 determines the attack part A1 as the first beat point P and the attack part A5 as the third beat point P. Furthermore, the beat point detection unit 52 sequentially specifies the reciprocal of the interval between successive beat points P as the instantaneous tempo TMP0.

  The section demarcating section 54 in FIG. 1 is means for dividing the music into a plurality of unit sections T based on the beat point P detected by the beat point detecting section 52. For example, a section including a predetermined number of beat points P detected by the beat point detection unit 52 is defined as the unit section T. As shown in FIG. 5, the section defining unit 54 of this embodiment defines one measure including four beat points P from the first beat to the fourth beat as a unit section T.

  The pitch specifying unit 42 in FIG. 1 is a means for specifying a pitch (a chord name and a bass tone) according to the feature amount F for each unit section T of the musical tone signal V. The pitch specifying unit 42 of this embodiment includes a chord specifying unit 421 and a bass specifying unit 423. The chord specifying unit 421 specifies a chord name (chord) N1 corresponding to the tone indicated by the tone signal V of each unit section T. For example, the code specifying unit 421 averages the HPCP (feature amount FS) stored in the feature amount storage unit 34 over a plurality of frames in the unit section T, and refers to a table that associates the HPCP with the code name N1. The code name N1 corresponding to HPCP is specified. The specification of the code name N1 using HPCP is disclosed in Japanese Patent Application Laid-Open No. 2000-298475, which is a prior application of the present applicant.

  The bass identifying unit 423 identifies the bass sound N2 corresponding to the musical tone indicated by the musical tone signal V of each unit section T. For example, the base specifying unit 423 specifies the frequency of the lowest frequency peak among the plurality of peaks whose intensity exceeds the threshold in the power spectrum Q indicated by the frequency bin S1 of the feature quantity FS as the base sound N2. Note that the method for specifying the bass sound N2 by the bass specifying unit 423 is arbitrary. For example, the base sound N2 may be specified by calculating the period (pitch) of the zero-crossing point after performing filter processing for selectively extracting only the components of 1 kHz or less from the musical sound signal V.

  The chord name N1 and the bass note N2 calculated by the pitch specifying unit 42 for each unit section T are stored in the pitch storage unit 44. A semiconductor storage device or a magnetic storage device is employed as the feature amount storage unit 34 or the pitch storage unit 44. The feature amount storage unit 34 and the pitch storage unit 44 may be separate storage areas set in one storage device, or may be separate storage devices. In addition, at least one of the feature amount storage unit 34 and the pitch storage unit 44 may be a storage area set in the storage circuit 20.

  The storage circuit 20 of FIG. 1 stores a plurality of accompaniment mode data DA each corresponding to a separate musical piece. One piece of accompaniment mode data DA is data for designating the mode of the accompaniment sound within one measure of the music. The mode of the accompaniment sound (the type of musical instrument that plays the accompaniment sound and the time of occurrence of each accompaniment sound) differs for each accompaniment mode data DA. As shown in FIG. 1, one accompaniment mode data DA is composed of an identifier AID uniquely assigned to the accompaniment mode data DA and a plurality of tracks TR each designating the time point of performance of a separate percussion instrument. The The accompaniment mode data DA of this embodiment includes a track TR_HH corresponding to a hi-hat cymbal, a track TR_SD corresponding to a snare drum, and a track TR_BD corresponding to a base drum. One track TR is MIDI (Musical Instrument Digital Interface) data in which event data E for instructing performance of a percussion instrument and time data Δ for designating an interval of each event (performance) are arranged in time series.

  1 selects one of a plurality of accompaniment mode data DA stored in the storage circuit 20 based on the feature value F specified by the feature value extraction unit 32 for the musical tone signal V in the unit section T. It is means for selecting each section T sequentially. The selection unit 62 of this embodiment includes an accompaniment processing unit 64 and a comparison unit 66.

  FIG. 6 is a conceptual diagram for explaining the operation of the accompaniment processing unit 64 and the comparison unit 66. In the figure, a spectrogram G0 of the musical tone signal V in the unit interval T to be processed by the selection unit 62 is shown. As shown in FIG. 6, the accompaniment processing unit 64 executes an expansion / contraction process and an accompaniment sound specifying process for each of the plurality of accompaniment mode data DA stored in the storage circuit 20.

  In the expansion / contraction process, the accompaniment mode data DA is set so that the time points of the accompaniment sounds specified by the accompaniment mode data DA and the beat points P detected by the beat point detection unit 52 for the unit section T coincide on the time axis. This is a process of adjusting the position on the time axis of each accompaniment sound to be designated. The accompaniment processing unit 64, for example, the relative ratio of the average tempo TMP0 detected by the beat point detection unit 52 for each beat point P in the unit interval T and the tempo TMP of the accompaniment sound specified by the accompaniment mode data DA (TMP0 / TMP) is multiplied by the time data Δ of the accompaniment mode data DA.

  The accompaniment sound specifying process is a process of specifying the feature amount FA of the accompaniment sound specified by the accompaniment mode data DA. First, the accompaniment processing unit 64 specifies a spectrogram GA (power spectrum time series) for the accompaniment sound specified by the accompaniment mode data DA. For example, the accompaniment processing unit 64 holds a spectrogram g of performance sound for each of a plurality of percussion instruments assumed as sound sources of accompaniment sounds specified by the accompaniment mode data DA. Then, the spectrogram g of the percussion instrument is arranged at the time when the generation of the performance sound of one percussion instrument is instructed by the accompaniment mode data DA, and the generation of the accompaniment sound of a plurality of percussion instruments is instructed by the accompaniment mode data DA. Specifies the spectrogram GA by placing the addition of the spectrogram g of each percussion instrument. Secondly, the accompaniment processing unit 64 extracts N types of feature values FA from the spectrogram GA for each frame by the same processing as the feature value extraction unit 32. Note that the feature quantity extraction unit 32 may be shared for extracting the feature quantity FA. In the above description, the spectrogram GA and the feature amount FA are specified from the accompaniment mode data DA. However, the spectrogram GA and the feature amount FA may be stored in the storage circuit 20 as the accompaniment mode data DA.

  As shown in FIG. 6, the comparison unit 66 performs accompaniment processing for each of a plurality of accompaniment mode data DA stored in the storage circuit 20 from the feature amount F in the unit interval T of the musical tone signal V and the accompaniment mode data DA. The similarity index value C is calculated by comparing the feature amount FA specified by the unit 64. The similarity index value C is a numerical value that serves as an index of similarity between the feature value F of the musical sound signal V and the feature value FA specified by the accompaniment processing unit 64 (the similarity between the spectrogram G0 and the spectrogram GA in FIG. 6). . For example, N types of feature values F extracted from the musical tone signal V are specified as coordinate values in the N-dimensional space, and N types of feature values FA extracted from the accompaniment mode data DA are used as coordinate values in the N-dimensional space. The Euclidean distance from the point specified by is calculated as the similarity index value C. Therefore, it can be said that the smaller the similarity index value C, the higher the similarity between the feature value F and the feature value FA of the tone signal V. However, a configuration for calculating the similarity index value C having a property that increases as the similarity between the feature amount F and the feature amount FA increases is also adopted.

  The musical tone signal V includes musical sounds other than the accompaniment sounds, whereas the accompaniment mode data DA includes only the designation of the accompaniment sounds. Therefore, it is desirable to calculate the similarity index value C for components other than the accompaniment sound in the musical sound signal V so that the influence on the similarity with the feature amount FA is reduced. For example, a component in which the musical sound other than the accompaniment sound is dominant among the characteristic amount F extracted from the musical sound signal V (for example, the characteristic amount F in the frequency band in which the musical sound other than the accompaniment sound exists in the power spectrum Q). A configuration in which the similarity index value C is calculated after being removed is adopted.

  The selection unit 62 selects the accompaniment mode data DA (accompaniment mode data DA that designates the accompaniment sound most similar to the musical tone signal V) that minimizes the similarity index value C calculated by the comparison unit 66, and the accompaniment mode data. The identifier AID included in DA is acquired from the storage circuit 20 and is sequentially output to the output unit 70 for each unit interval T. That is, the identifier AID is output for the accompaniment mode data DA of the accompaniment sound that approximates the tone of the music signal V.

  The output unit 70 in FIG. 1 generates and outputs unit data U for each unit section T defined by the section defining unit 54. As shown in FIG. 1, the unit data U corresponding to one unit section T includes the identifier AID of the accompaniment mode data DA selected by the selection unit 62 for the unit section T and the pitch specifying unit 42 for the unit section T. Includes the chord name N1 and the bass note N2. A time series of unit data U for each unit section T is supplied to the output processing device 14 as musical sound data DOUT.

  The output processing device 14 includes a control circuit 141, a storage circuit 143, a sound source circuit 145, and an output device 147. The storage circuit 143 sequentially stores the musical tone data DOUT supplied from the musical tone processing apparatus 100, and stores a plurality of performance data DB each corresponding to separate accompaniment mode data DA. The performance data DB is data for designating the performance mode within one measure, as with the accompaniment mode data DA.

  FIG. 7 is a conceptual diagram showing the structure of one piece of performance data DB. As shown in FIG. 7, performance data DB corresponding to one accompaniment mode data DA includes the accompaniment mode data DA, a common identifier AID, and a plurality of tracks TR each corresponding to a separate part. The performance data DB of this embodiment includes a track TR_CD for specifying the chord performance mode and a track for specifying the bass performance mode in addition to the common track TR (TR_HH, TR_SD, TR_BD) with the accompaniment mode data DA. Including TR_BS. The track TR_CD is a data string in which event data E for instructing performance of a chord and time data Δ for designating an interval between events are arranged in time series. The event data E of the track TR_CD does not include a code name designation. Similarly, the track TR_BS is a data string of event data E (not including bass sound designation) that instructs performance of the bass sound and time data Δ.

  The control circuit 141 in FIG. 1 generates performance data DC by assigning the chord name N1 and the bass sound N2 included in each unit data U of the musical sound data DOUT to the performance data DB. That is, the unit data U of the musical tone data DOUT is sequentially obtained from the storage circuit 143 in the order of arrangement, and the performance data including the identifier AID in the unit data U among the plurality of performance data DB stored in the storage circuit 143. Select DB. Then, the control circuit 141 adds the chord name N1 included in the unit data U to each event data E of the track TR_CD in the performance data DB, and also adds the bass sound N2 included in the unit data U to the track in the performance data DB. Performance data DC is generated by adding to each event data E of TR_BS. Further, the control circuit 141 sequentially outputs the event data E of each track TR of the performance data DC to the sound source circuit 145 at the time designated by the time data Δ in the track TR.

  The tone generator circuit 145 is a MIDI tone generator that generates a signal indicating a tone waveform corresponding to each event data E supplied from the control circuit 141 and outputs the signal to the output device 147. The output device 147 emits a sound wave corresponding to the signal output from the sound source circuit 145. The musical sound output from the output device 147 is an accompaniment sound (percussion instrument sound, chord and bass sound) that approximates the musical tone of the musical sound signal V.

  As described above, according to the present embodiment, the musical sound signal V output from the signal generating device 12 is encoded into the musical sound data DOUT having a small data amount. Since the position of the beat point P is automatically detected from the feature amount F when the music signal V is encoded into the music data DOUT, the user's effort to specify the beat point P for the music signal V is reduced. There is an advantage. Further, since the unit data U is generated (selecting the accompaniment mode data DA and adding the chord name N1 and the bass sound N2) with the unit section T (for example, one measure) as a unit, for example, the beat point P of the musical sound signal V There is an advantage that the unit data U of performance sound with a musical sense of unity can be generated as compared with the configuration in which encoding is performed every time.

<B: Modification>
Various modifications can be made to the above embodiment. An example of a specific modification is as follows. In addition, the structure which combined arbitrary two or more aspects illustrated below is also employ | adopted.

(1) Modification 1
In the above embodiment, the configuration in which the feature value F (FS, FW) of the musical tone signal V is extracted for each frame at a predetermined interval is exemplified, but the timing for extracting the feature value F is appropriately changed. The accompaniment feature data F is particularly important for the selection of the accompaniment mode data DA (the necessity of extracting the musical feature F other than the accompaniment sound is low). A configuration in which the feature amount F is extracted only for a section in which sound generation is estimated is preferable.

  As shown in FIG. 3, the waveform feature amount extraction unit 325 in one aspect sequentially selects the time t (the time when the accompaniment sound is generated) t of the musical tone signal V from the feature amount FW of the musical tone signal V. Identify. The frequency analysis unit 321 calculates the power spectrum Q only for the section extending from the time point t specified by the waveform feature amount extraction unit 325 to a predetermined time length. According to the above configuration, there is an advantage that the processing amount of the frequency analysis unit 321 and the spectrum feature amount extraction unit 323 is reduced. Further, if the feature amount extraction unit 32 calculates the feature amount F by using a section corresponding to the interval of the beat points P detected by the beat point detection unit 52 in the musical sound signal V as a unit, the feature for each beat point P is obtained. The quantity F can be calculated.

(2) Modification 2
A configuration in which the accompaniment mode data DA is selected in consideration of the configuration of the music indicated by the musical tone signal V is also employed. FIG. 8 is a block diagram showing the configuration of the musical tone processing apparatus 100 according to the modification. The configuration determination unit 82 shown in the figure is means for detecting a repeated section in which an equivalent performance is repeated in the music. FIG. 9 is a conceptual diagram showing the contents of processing by the configuration determining unit 82. As shown in the figure, the configuration determining unit 82 is based on the feature amount F for the musical similarity for all combinations of selecting two sections from a plurality of sections (for example, the unit section T) into which the music is divided. To calculate. In FIG. 9, the arrangement of the unit sections T from the start point to the end point of one piece of music is shown on the vertical axis and the horizontal axis. Black spots are written at points corresponding to combinations of two sections whose similarity exceeds a predetermined threshold. A black dot on the straight line L in FIG. 9 means that the similarity calculated for one common section is maximized (matched). When the result as illustrated in FIG. 9 is calculated, the configuration determining unit 82 determines that the section a and the section b are repetitive sections (sections similar to each other).

  The selection unit 62 selects the same accompaniment mode data DA for the repeated section detected by the configuration determination unit 82. That is, in the case of FIG. 9, the accompaniment mode data DA common to the i-th (i is a natural number) unit section T in the section a and the i-th unit section T in the section b is selected. According to the above configuration, the accompaniment mode data DA can be selected without executing the process of FIG. 6 for the section b which is a repetition of the section a, so that there is an advantage that the processing amount of the selection unit 62 is reduced. Further, in the configuration in which the process of FIG. 6 is performed for each unit section T, there is a possibility that separate accompaniment mode data DA is selected even though it is a repeated section. On the other hand, according to the configuration of FIG. 8, the same accompaniment sound is automatically generated in the repeated section, so that musical uniformity can be ensured.

  Further, the characteristic verification unit 84 in FIG. 8 includes the chord name N1 and the bass tone N2 specified by the pitch specifying unit 42 and the beat point P detected by the beat point detecting unit 52 (or the unit section T defined by the section defining unit 54). ) And the accompaniment mode data DA selected by the selection unit 62 is verified and reflected in each operation. For example, the characteristic verification unit 84 determines whether or not the chord name N1 over a plurality of unit sections T is musically appropriate, and determines that the chord name N1 of any unit section T is inappropriate The code identification unit 421 is caused to correct the code name N1 of the unit section T. That is, when the code name N1 is determined as “Cm” in only one unit section T among the many unit sections T and the code name N1 is determined as “C” in all other unit sections T, the characteristic verification is performed. The unit 84 instructs the code specifying unit 421 to correct the code name N1 (Cm → C) for the unit section T in which the code name N1 is determined to be “Cm”. When the chord name N1 specified by the chord specifying unit 421 and the bass sound N2 specified by the base specifying unit 423 are not in harmony musically, the characteristic verification unit 84 corrects the chord name N1 or the bass sound N2 with a pitch. The specifying unit 42 is instructed. According to the above configuration, it is possible to generate an accompaniment sound in which musical incongruity is suppressed.

(3) Modification 3
In some cases, a peak corresponding to the percussion instrument performance sound occurs on the low frequency side of the power spectrum Q. Therefore, in the form in which the lowest frequency at which the peak occurs is specified as the bass sound N2, the performance sound of the percussion instrument may be mistaken for the bass sound N2. On the other hand, the bass sound N2 that should basically be identified by the bass identifying unit 423 has a harmonic structure (harmonic structure) in which the performance sound continues for a long time compared to the percussion instrument and overtones appear at a frequency that is an integral multiple of the fundamental tone. ) Voiced sound. Therefore, the base specifying unit 423 in the preferred embodiment specifies, as a base tone N2, a tone that continues for a time length exceeding a predetermined value and has a harmonic structure recognized among the tones corresponding to the low-frequency peak. According to the above configuration, it is possible to specify the bass sound N2 with high accuracy even in the music in which the bass sound N2 and the percussion instrument performance sound are mixed. However, the method by which the bass identifying unit 423 identifies the bass sound N2 is arbitrary in the present invention.

(4) Modification 4
The method in which the code specifying unit 421 specifies the code name N1 based on the feature amount F is arbitrary in the present invention. In addition, a configuration in which the chord identifying unit 421 identifies the order (voicing) of the arrangement of each musical sound constituting the chord is also employed. For example, the chord specifying unit 421 recognizes the level of each tone constituting the chord from the shape of the power spectrum Q (feature amount FS), and specifies the chord name N1 of the chord in which each tone is arranged in the order of the level. In addition, a configuration in which the chord identifying unit 421 identifies the chord name N1 based on the bass sound N2 identified by the bass identifying unit 423 is also employed. For example, the chord specifying unit 421 specifies the chord name N1 of the fraction code (on chord) that includes the bass sound N2 as the lowest scale.

(5) Modification 5
In the above embodiment, the configuration in which the extraction of the feature amount FA (expansion / contraction processing and accompaniment sound specifying processing) is performed on all the accompaniment mode data DA stored in the storage circuit 20 has been illustrated, but specific accompaniment mode data DA A configuration is also adopted in which the feature amount FA is selectively extracted for. For example, a plurality of accompaniment mode data DA in the memory circuit 20 is classified according to the genre of music, and only the accompaniment mode data DA corresponding to the genre specified from the musical tone signal V or the genre specified by the user is a candidate for selection. A configuration that contrasts with the musical sound signal V is also adopted. Further, accompaniment mode data DA (for example, accompaniment mode data DA in which the tempo of the accompaniment sound is significantly different from the musical sound signal V) that can be determined to be clearly different from the accompaniment sound of the musical sound signal V is selected from the selection candidates. An excluded configuration is also adopted. According to the above form, since the extraction of the feature value FA can be omitted for the accompaniment mode data DA excluded by narrowing down, there is an advantage that the processing amount of the selection unit 62 is reduced.

(6) Modification 6
In the above embodiment, the configuration in which the identifier AID of the accompaniment mode data DA is included in the unit data U and output from the musical sound processing apparatus 100 is exemplified, but the contents of the unit data U are changed as appropriate. For example, according to the configuration in which the accompaniment mode data DA is output as the unit data U together with the chord name N1 and the bass sound N2, the output processing device 14 holds the percussion instrument track TR (TR_HH, TR_SD, TR_BD) in the performance data DB. There is an advantage that it is not necessary.

  Further, a configuration in which the data (performance data DB in FIG. 7) obtained by adding the chord track TR_CD and the bass sound track TR_BS to the percussion instrument track TR is stored in the storage circuit 20 as the accompaniment mode data DA is also employed. The output unit 70 generates performance data DC by inserting the chord name N1 into the event data E of the chord track TR_CD and the bass sound N2 into the event data E of the bass track TR_BS in the accompaniment mode data DA. And output to the output processing device 14. According to the above configuration, there is an advantage that it is not necessary to hold the performance data DB in the output processing device 14.

  Further, a file in which the musical sound signal V of the unit section T is added to the unit data U of each unit section T may be sequentially transmitted from the musical sound processing apparatus 100 to the output processing apparatus 14. According to this configuration, on the output processing device 14 side, not only the accompaniment sound can be reproduced based on the unit data U, but also the musical sound signal V corresponding to the accompaniment sound is used as a material for creating various musical pieces. Is possible.

  As shown in FIG. 1, according to the configuration in which the identifier AID is included in the unit data U and output, the output processing from the musical sound processing device 100 is performed compared to the configuration in which the data of each track TR is included in the unit data U and output. There is an advantage that the amount of data transmitted to the device 14 is reduced. Therefore, for example, in a configuration in which the musical sound processing device 100 and the output processing device 14 are connected via a communication network, unit data U including an identifier AID (unit data not including a track TR is used from the viewpoint of reducing communication traffic. The configuration in which U) is transmitted to the output processing device 14 is particularly suitable.

(7) Modification 7
The type of feature value F extracted from the musical sound signal V (or feature value FA extracted from the accompaniment mode data DA) is not limited to the above examples. For example, the gradient of the straight line approximating the power spectrum Q, the center of gravity (centroid) of the power spectrum Q, the Mel Cepstrum Coefficients (MFCC), the average value of the intensity at the peak of the power spectrum Q and the intensity of the intensity other than the peak The spectrum feature quantity extraction unit 323 can extract various feature quantities F such as a relative ratio with the average value. That is, the chord name N 1 and the bass sound N 2 are specified by the pitch specifying unit 42, the beat point P is detected by the beat point detecting unit 52 (the unit section T is defined by the section defining unit 54), and the accompaniment mode is performed by the selecting unit 62. All feature quantities F that can be used for selection of data DA (comparison between accompaniment sound indicated by musical tone signal V and accompaniment sound indicated by accompaniment mode data DA) are applied.

(8) Modification 8
The method for detecting the beat point P is appropriately changed. For example, in the above embodiment, the configuration in which the beat point P is detected based on the tendency of the type of percussion instrument played at each beat point P is illustrated, but the specific frequency corresponding to the percussion instrument among the plurality of attack parts A An attack part A having a high component intensity in a band (for example, band BSD in FIG. 5) may be specified as beat point P. Further, since the performance sound of the percussion instrument is instantaneous (does not continue), a configuration is also adopted in which the attack portion A in which the performance sound does not continue among the plurality of attack portions A is specified as the beat point P.

(9) Modification 9
The signal generation device 12 and the musical sound processing device 100 may constitute an integrated device. Further, the source of the musical sound signal V is not limited to the signal generator 12. For example, a musical tone signal V obtained by A / D converting an analog signal collected by a microphone is supplied to the musical tone processing apparatus 100, or a musical tone signal V is supplied to the musical tone processing apparatus 100 via a communication network such as the Internet. A configuration is also adopted.

It is a block diagram which shows the structure of the musical tone processing apparatus which concerns on one form of this invention. It is a conceptual diagram for demonstrating operation | movement of a spectrum feature-value extraction part. It is a conceptual diagram for demonstrating operation | movement of a waveform feature-value extraction part. It is a conceptual diagram which shows a mode that a feature-value memory | storage part memorize | stores a feature-value. It is a conceptual diagram which shows the specific example of the operation | movement which a beat point detection part detects a beat point. It is a conceptual diagram for demonstrating operation | movement of an accompaniment process part and a comparison part. It is a conceptual diagram which shows the structure of performance data DB. It is a block diagram which shows the structure of the musical tone processing apparatus which concerns on a modification. It is a conceptual diagram for demonstrating operation | movement of a structure determination part.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 ... Musical sound processing device, 12 ... Signal generation device, 14 ... Output processing device, 141 ... Control circuit, 143 ... Memory circuit, 145 ... Sound source circuit, 147 ... Output device, 20 ... Memory circuit , 30... Control circuit, 32... Feature amount extraction unit, 321... Frequency analysis unit, 323... Spectrum feature amount extraction unit, 325. ... Pitch specifying part, 421 ... Chord specifying part, 423 ... Bass specifying part, 44 ... Pitch storage part, 52 ... Beat point detection part, 54 ... Section demarcation part, 62 ... Selection part, 64 ... accompaniment processing section, 66 ... comparison section, 70 ... output section.

Claims (5)

Storage means for storing a plurality of accompaniment mode data for specifying the mode of the accompaniment sound;
Feature quantity extraction means for sequentially extracting feature quantities from a musical tone signal indicating the musical tone of the music;
Beat point detecting means for detecting a beat point from the musical sound signal;
Section defining means for defining a plurality of unit sections based on the beat points detected by the beat point detecting means;
Wherein said plurality of accompaniment aspects of data each indicating accompaniment sound feature amount and the feature amount extracting means includes a comparison means for comparing the feature amounts specified for the tone signal in the unit section, said storage means stores Selection means for sequentially selecting accompaniment sound data of accompaniment sounds similar to the musical tone signal in the unit interval among a plurality of accompaniment aspect data for each unit interval based on a result of comparison by the comparison means;
A musical tone processing apparatus comprising: output means for outputting the accompaniment mode data selected by the selection unit or unit data including an identifier of the accompaniment mode data for each unit section. The feature amount extraction means sequentially extracts the feature amount of the musical sound signal for each frame,
2. The musical sound processing apparatus according to claim 1, wherein the comparing means compares the characteristic amount of each accompaniment sound frame indicated by each of the plurality of accompaniment mode data with the characteristic amount of each frame of the musical sound signal in the unit section . The selection means includes accompaniment processing means for specifying a spectrogram of an accompaniment sound for each of the plurality of accompaniment mode data, and the characteristic amount of the spectrogram specified by the accompaniment processing means for each accompaniment mode data and a musical tone in the unit section Based on the result of comparison by the comparing means with the spectrogram feature value of the signal, the accompaniment mode data whose accompaniment sound spectrogram is similar to the spectrogram of the musical tone signal in the unit section is selected from the plurality of accompaniment mode data The musical tone processing apparatus according to claim 1 or 2. The beat point detecting means detects a time point at which a component belonging to each of a plurality of frequency bands corresponding to performance sounds of different types of musical instruments is generated as a beat point. Any of the musical tone processing apparatuses. On the computer,
A feature amount extraction process for sequentially extracting feature amounts from a musical tone signal indicating the musical tone of the music;
Beat point detection processing for detecting a beat point from the musical sound signal;
Section defining process for defining a plurality of unit sections based on the beat points detected by the beat point detecting process,
A comparison process for comparing the feature amount of the accompaniment sound indicated by each of a plurality of accompaniment mode data specifying the form of the accompaniment sound and the feature amount specified for the musical tone signal in the unit interval in the feature amount extraction process, A selection process for sequentially selecting accompaniment mode data of accompaniment sounds similar to the musical tone signal in the unit section among a plurality of accompaniment mode data for each unit section based on the result of the comparison process ;
A program for executing an output process for outputting the accompaniment mode data selected in the selection process or unit data including an identifier of the accompaniment mode data for each unit section.

Images