JP5229998B2 - Code name detection device and code name detection program - Google Patents

Description

  The present invention relates to a code name detection device and a code name detection program.

  In popular music, chords are a very important element. When playing music of such a genre in a small band, do not use the score on which the individual notes to be played are written. Or it is usual to use a musical score called a lead sheet in which only a melody and chord progression are written. Therefore, it is necessary to record the chord progression of a song in order to play a commercially available song such as a CD in a band, but this work can only be performed by experts with special musical knowledge. It was impossible. Therefore, there has been a demand for an automatic musical transcription device that detects a chord name based on a music acoustic signal output from a sound source such as a music CD on which performance sounds are recorded using a commercially available personal computer.

  As an apparatus for detecting a chord from such a music sound signal, there is a configuration of Patent Document 1 below. In the configuration of this document, a fundamental frequency candidate is extracted from the calculation result of the power spectrum, and what is considered to be a harmonic is removed from the fundamental frequency candidate to detect note information, and a chord is detected from this note information. .

Similarly, as a device for detecting a chord from a music acoustic signal, there is a configuration of Patent Document 2 described later. In the configuration of Patent Document 2, digital filtering processing with different characteristics is performed on input acoustic signals in a time-sharing manner to detect the level of each scale sound, and the same scale relation within the octave is detected among the detected levels. The chords are detected by using a predetermined number having a large value among the integrated levels by integrating certain levels. Since this method does not detect individual note information contained in the acoustic signal, the problem described in Patent Document 1 described later does not occur.
Japanese Patent No. 28768861 Japanese Patent No. 3156299

  However, in the configuration shown in Patent Document 1, the above-described operation for removing overtones includes the difference in overtone structure depending on the type of musical instrument, the difference in how harmonics are generated due to keystroke strength, the power change of overtones over time, and the same frequency overtones. It is known that it is very difficult due to the problem of phase interference between sounds possessed as components. That is, it is not considered that the step of detecting the note information functions correctly with a sound source such as a general music CD in which many musical instruments and singing are mixed.

  On the other hand, in the configuration shown in Patent Document 2 for detecting a chord from a music acoustic signal, the levels of the scales are integrated for each of the 12 pitch names having the same scale relationship within the octave. A plurality of chords composed of the same constituent sound, for example, Am7 composed of la, de, mi and so, and C6 composed of de, mi, so and la cannot be discriminated.

  In addition, the chord detection device of Patent Document 2 does not have a tempo or measure detection function, and chord detection is performed at predetermined timings. In other words, it is assumed that the tempo of the song is set in advance and the performance is performed in accordance with a metronome that produces the tempo. The chord name can be detected, but the tempo and measure are not detected, so it cannot be output in the form of a score in which the chord name of each measure such as a chord score or lead sheet is written. .

  Even if the tempo of the song is given, the tempo of the performance recorded on the music CD is generally not constant and slightly fluctuates, so that the chord for each measure cannot be detected correctly.

  Also, it is very difficult for beginners to perform at an accurate tempo that matches a metronome that is pronounced at a constant tempo, and generally the performance tempo usually fluctuates. .

  Furthermore, in the configuration of Patent Document 2, a configuration in which digital filtering processing with different characteristics is performed in time division on an input acoustic signal is adopted. The reason for the adoption of this configuration is that the FFT operation has a low frequency range. The frequency resolution is bad. However, it is possible to obtain a certain level of frequency resolution even at low frequencies by down-sampling the input acoustic signal and performing FFT, and the digital filtering process requires an envelope extraction unit to determine the level of the filter output signal On the other hand, in FFT, the power itself after the FFT represents the level at each frequency, so that is not necessary, and the frequency resolution can be selected by appropriately selecting the parameters of the number of FFT points and the shift amount. And the time resolution can be set freely.

  Even if a chord can be detected, if there are pitches such as 7th and 9th in the detected chord, the chord may be detected as a four-chord chord although it may actually be a three-chord. In addition, the intensity of the characteristic pitch is relatively small, and a desired chord may not enter the candidate. Therefore, even if automatic music recording can be performed, it is necessary that the user can freely set the recorded results to some extent.

  Even if you are not an expert with special musical knowledge, it does not detect individual note information from music sound signals (audio signals) that are mixed with multiple instrument sounds such as music CDs and that have fluctuations in the performance tempo. In addition, there is provided a chord name detection device capable of detecting chord names (chord names) from the overall sound, and a configuration in which the user can freely modify the scored results in the result. desired.

  Even when such a configuration is provided, it is actually necessary to listen to the performance sound of the detected chord and the original sound signal at the same time and to check whether the detected chord and bass sound are correct. is there. Even if a chord detection device as described above is provided, even if the sound signal and the detected chord performance are listened to separately or at the same time, at the same time, the volume of the sound signal and the chord performance is adjusted separately. In order to determine whether or not the detected bass sound or chord is correct, it takes time and effort to adjust the volume, and it is difficult to make a simple listening comparison.

  The present invention was devised in view of the above-described problems, and both volume levels can be adjusted by a single operation for listening comparison between an acoustic signal and a detected chord performance while performing chord detection. The present invention also proposes a computer program that can implement these configurations on a computer.

Therefore, the configuration of the code name detection device according to the present invention is as follows.
An input means for inputting an acoustic signal;
First scale sound level detection means for performing FFT calculation using a parameter suitable for beat detection at predetermined time intervals from the input acoustic signal, and obtaining the level of each scale sound for each predetermined time;
The increment value of each scale sound level for each predetermined time is summed for all the scale sounds to obtain a total of level increment values indicating the degree of change in the overall sound for each predetermined time. Beat detection means for detecting the average beat interval and the position of each beat from the sum of the incremental values of the level indicating the degree of change in the overall sound for each,
The average value of the scale level for each beat is calculated, and the increment value of the average level of each scale sound for each beat is added for all the scale sounds to indicate the degree of change in the overall sound for each beat. A bar detecting means for obtaining a value and detecting a time signature and a bar line position from a value indicating a change degree of the whole sound for each beat;
From the input acoustic signal, an FFT operation is performed using a parameter suitable for chord detection at a predetermined time interval different from that at the time of the previous beat detection, and the level of each scale sound for each predetermined time is calculated. Second scale level detection means to be obtained;
Bass sound detection means for detecting a bass sound from the level of the low-frequency scale sound in each measure out of the detected scale levels,
Chord name determining means for determining the chord name of each measure from the detected bass sound and the level of each scale sound;
For each detected chord, it is obtained from the chord position and the level of the scale sound in the base detection range during the base detection period (calculate the average level of each scale sound in the base detection range during the base detection period (Obtained by calculating the average level of each scale tone in the chord detection period) (obtained by calculating the average level of each tone in the chord detection period) Chord information storage means for storing chord scale sound intensity, chord constituent sound, chord constituent sound number, chord name,
Display means for displaying the chord name or chord constituent sound and / or bass sound determined by the chord name determining means;
Performance means for playing the input acoustic signal and the determined chord;
Adjusting means for adjusting the volume balance of both outputs of the performance means;
By increasing or decreasing the threshold value that is set for the intensity of the scale sound that is the constituent sound, the number of chord constituent sounds can be increased or decreased in descending order of the chord scale sound intensity. Change means to change the sound;
Based on the change of the chord name or chord constituent sound by the changing means, the chord constituent sound, the chord constituent sound number, and the chord name are referred to by referring to the chord scale intensity and chord constituent sound stored in the chord information storing means. The basic feature is that the name determining means is changed, and the change information is re-stored in the code information storage means, and at least the display means has a resetting means for re-displaying the chord name or chord constituting sound.

  According to the above configuration, the scale level for each predetermined time is obtained from the acoustic signal input to the input means by the scale level detecting means, and the beat detection means determines the scale sound for each predetermined time. The level increment values are summed for all scales to obtain the total level increment value indicating the degree of change in the overall sound for each predetermined time, and the beat detection means also performs the overall sound for this predetermined time. The average beat interval (that is, tempo) and the position of each beat are detected from the sum of the level increments indicating the degree of change in the level, and then the measure detection means measures the tone of each scale note for each beat. The average value of the level is calculated, and the increment value of the average level of each scale sound for each beat is summed up for all the scale sounds, and the above value indicating the degree of change in the overall sound for each beat is obtained, and for each beat. all From a value that indicates the degree of change in tone, thereby detecting the beat and measure line position (first beat position).

  That is, the level of each scale sound for each predetermined time is obtained from the input sound signal, and the average beat (beat) interval (that is, tempo) and each beat are determined from the change in the level of each scale sound for each predetermined time. Next, the time signature and bar line position (position of the first beat) are detected from the change in the level of each scale tone for each beat.

  In the bass sound detecting means, when a plurality of bass sounds are detected in a measure, the chord name determining means divides the measure into several chord detection ranges according to the bass sound detection result. The chord name in each chord detection range is determined from the base sound and the level of each scale sound in each chord detection range.

  According to the above configuration, the first acoustic scale level detection means first performs an FFT operation with a parameter suitable for beat detection on the input sound signal input from the input means at a predetermined time interval. The level of each scale sound for each time is obtained, and the beat detection means detects the average beat interval and the position of each beat from the change in the level of each scale sound for each predetermined time. Next, the measure and the bar line position are detected from the change in the level of each scale sound for each beat by the measure detecting means. Furthermore, the chord name detection apparatus according to the present invention is suitable for chord detection at a predetermined time interval different from the time of the previous beat detection with respect to the input acoustic signal by the second scale sound level detection means. An FFT operation is performed with the parameters, thereby obtaining the level of each scale sound for each predetermined time. The bass sound detecting means detects the bass sound of each measure from the scale sound level on the low frequency side, and the chord name determining means detects the bass sound and the level of each scale sound. The chord name of each measure will be determined from

  In addition, as described above, when a plurality of bass sounds are detected in the measure by the bass sound detecting means, the chord name determining means determines that the measure is divided into several chord detection ranges according to the bass sound detection result. The chord name in each chord detection range is determined from the bass sound and the level of each tone in the chord detection range.

  As described above, in the configuration of the code name detection device of the present invention, processing that requires time resolution of beat detection with only a simple configuration (so-called tempo detection device configuration) and frequency resolution of chord detection are possible. Can be performed simultaneously (a configuration that can further detect chords based on the configuration of the tempo detection device).

  Further, for each detected chord, the chord information storage means stores the base range tone intensity, the base tone, and the chord detection period obtained from the chord position and the level of the scale tone of the base detection range in the base detection period. The chord scale sound intensity, the chord constituent sound, the chord constituent sound number, and the chord name obtained from the scale sound level of the chord detection tone range in FIG. And the chord name thru | or the chord constituent sound and the bass sound determined by the chord name determining means are displayed on the display means.

  It is not possible to determine whether or not the detection result is actually correct only by displaying the chord name, chord constituent sound and bass sound. For this purpose, the apparatus according to the present invention is configured such that the input sound signal and the determined chord are played by the playing means.

  However, the volume balance of both outputs of the performance means can be adjusted by the adjustment means as described above. That is, the volume can be adjusted without changing the overall volume by operating with a single operator, such as a slide resistor, and operating it once.

If the user who listened to both sounds with appropriate volume adjustments feels that the chord name is not appropriate in the result, or if he / she feels unnatural when he / she hears the chord progression, he / she wants to correct it. use means, on the user side, it is possible to change the code name to chord member (may change in the number of chord member). That is, as the number of changes of chord member, when changing the adoption number, triad, and the like becomes possible to four chords controls.

  When such a change is made, the chord scale sound intensity and chord constituent sound stored in the chord information storing means are referred to by the resetting means, and the chord constituent sound, the chord constituent sound number, the chord name, The code name determining means is changed, the change information is re-stored in the code information storage means, and at least the display means re-displays the chord name or chord constituting sound.

When the number of chord constituent sounds is changed, the chord scale intensity stored in the chord information storage means and the chord constituent sounds are referred to because a threshold is provided for the intensity of the scale sound as the constituent sound. This is because the number of chord constituent sounds can be easily increased or decreased in descending order of the chord scale sound intensity. Depending on the change in the number of chords, the chord name will also change.

  Further, the second configuration of the present invention defines a program itself that can be executed by the computer in order to cause the computer to execute the configuration of the first invention. In other words, as a configuration for solving the above-described problems, the above-described means is realized by using the configuration of a computer, and is a program that can be read and executed by the computer. In this case, the computer may include a general-purpose computer configuration including the configuration of the central processing unit, or may include a dedicated machine directed to a specific process, and the configuration of the central processing unit. If it accompanies, there will be no limitation in particular.

  When a program for realizing the above means is read by the computer, the same function realizing means as the function realizing means defined as the configuration of the first invention is achieved.

A more specific configuration of the second invention is as follows:
By being read and executed by a computer, the computer is
An input means for inputting an acoustic signal;
First scale sound level detection means for performing FFT calculation using a parameter suitable for beat detection at predetermined time intervals from the input acoustic signal, and obtaining the level of each scale sound for each predetermined time;
The increment value of each scale sound level for each predetermined time is summed for all the scale sounds to obtain a total of level increment values indicating the degree of change in the overall sound for each predetermined time. Beat detection means for detecting the average beat interval and the position of each beat from the sum of the incremental values of the level indicating the degree of change in the overall sound for each,
The average value of the scale level for each beat is calculated, and the increment value of the average level of each scale sound for each beat is added for all the scale sounds to indicate the degree of change in the overall sound for each beat. A bar detecting means for obtaining a value and detecting a time signature and a bar line position from a value indicating a change degree of the whole sound for each beat;
From the input acoustic signal, an FFT operation is performed using a parameter suitable for chord detection at a predetermined time interval different from that at the time of the previous beat detection, and the level of each scale sound for each predetermined time is calculated. Second scale level detection means to be obtained;
Bass sound detection means for detecting a bass sound from the level of the low-frequency scale sound in each measure out of the detected scale levels,
Chord name determining means for determining the chord name of each measure from the detected bass sound and the level of each scale sound;
For every detected chord, from the chord position, the base tone scale intensity obtained from the scale sound level of the bass detection range during the base detection period, the base tone, and the scale sound level of the chord detection range during the chord detection period Chord information storage means for storing the required chord scale sound intensity, chord constituent sound, chord constituent sound number, chord name;
Display means for displaying the chord name or chord constituent sound and / or bass sound determined by the chord name determining means;
Performance means for playing the input acoustic signal and the determined chord;
Adjusting means for adjusting the volume balance of both outputs of the performance means;
By increasing or decreasing the threshold value that is set for the intensity of the scale sound that is the constituent sound, the number of chord constituent sounds can be increased or decreased in descending order of the chord scale sound intensity. Change means to change the sound;
Based on the change of the chord name or chord constituent sound by the changing means, the chord constituent sound, the chord constituent sound number, and the chord name are referred to by referring to the chord scale intensity and chord constituent sound stored in the chord information storing means. Code name detection characterized in that it is changed to a name determining means, and the change information is re-stored in the code information storage means, and at least the display means functions as a resetting means for re-displaying a chord name or a chord constituent sound. Program.

  With the program configuration as described above, by using this program using the existing hardware resources, each device of the present invention as a new application can be easily realized with the existing hardware. Become.

  In the aspect of this program, it becomes possible to easily use, distribute, and sell it using communication or the like. In addition, by using this program using existing hardware resources, the apparatus of the present invention as a new application can be easily executed with the existing hardware.

  It should be noted that some of the functions realizing means of the configuration of the second invention may be functions incorporated in a computer (functions incorporated in the hardware of a computer, operating functions incorporated in the computer) It may be a function realized by a system or other application program), and the program may include an instruction for calling or linking a function achieved by the computer.

  This is because a part of each function realizing means defined as the configuration of the first invention is substituted by a part of the function achieved by, for example, an operating system, and a program or a module for realizing the function is Although it does not exist directly, if a part of the function of the operating system that achieves these functions is called or linked, the configuration is substantially the same.

According to the chord name detection apparatus and program defined in the present invention, a music sound having a mixture of a plurality of instrumental sounds such as a music CD and a fluctuation in performance tempo, even if not an expert having special musical knowledge. In addition to being able to detect the chord name (chord name) from the overall sound without detecting individual note information from the signal (audio signal), the chord name (chord name) can be checked when checking the judgment result. Since it is not possible to determine whether or not the detection result is actually correct only by displaying the name or chord constituent sound or bass sound, in the configuration of the present invention, the input sound signal and the determined chord are It is configured to be played, and the above-mentioned adjusting means allows the volume balance of the sound signal and chord performance output of the playing means to be adjusted and is operated only once. , Without changing the overall volume, not only to the way it is these volume control, listening to such output, if not satisfied with the result of detection, the user side, by using the changing unit code By changing the name and chord constituent sound, there is also an effect that the music transcription result can be freely changed .

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

(Assumption example)
FIG. 1 is an overall block diagram of a tempo detection apparatus as a premise of the present invention. The tempo detection device is configured by a computer that can read and execute a tempo detection program. When the computer is divided into tempo detection functional blocks, an audio signal is input as shown in FIG. An input unit 1, a scale sound level detection unit 2 that performs an FFT operation at predetermined time intervals from the input sound signal, and obtains the level of each scale sound for each predetermined time, and this predetermined time interval The increment value of each scale sound level is summed for all the scale sounds to obtain the sum of the level increment values indicating the degree of change in the overall sound for each predetermined time. The beat detector 3 for detecting the average beat interval and the position of each beat, and the average value of the level of each scale tone for each beat are calculated from the sum of the incremental values indicating the degree of change. The value indicating the degree of change in the overall sound for each beat is obtained by summing up the average level increments of all the scales for each beat for all scales to obtain the value indicating the degree of change in the overall sound for each beat. To a bar detector and a bar detecting unit 4 for detecting bar line positions.

  As will be described later, the input unit 1 includes a microphone connected to a computer and an I / O interface such as a MIDI interface connected to the computer. As described above, the input unit 1 has a function of inputting an acoustic signal. have.

  The scale sound level detection unit 2 is constituted by a CPU of a computer that reads and executes a tempo detection program and performs the following processing. As described above, it has a function of performing an FFT operation at a predetermined time interval from the input acoustic signal and obtaining the level of each scale sound for each predetermined time.

  The beat detection unit 3 is similarly configured by a CPU of a computer that reads and executes a tempo detection program and performs the following processing. As described above, the sum of the increments of each scale sound level for each predetermined time is summed for all the scale sounds, and the sum of the level increment values indicating the degree of change of the overall sound for each predetermined time is obtained. It has a function of detecting the average beat interval and the position of each beat from the sum of the incremental values of the level indicating the degree of change of the entire sound every predetermined time.

  The bar detection unit 4 is similarly configured by a CPU of a computer that reads and executes a tempo detection program and performs the following processing. As mentioned above, it calculates the average value of each scale note level for each beat, and sums the increments of the average level of each scale note for each beat for all the scale notes. A value indicating the degree of change in sound is obtained, and the time and bar line position are detected from the value indicating the degree of change in the overall sound for each beat.

  The input unit 1 for inputting a music sound signal is a part for inputting a music sound signal to be subjected to tempo detection. An analog signal input from a device such as a microphone may be converted into a digital signal by an A / D converter (not shown). In the case of digitized music data such as a music CD, it is directly taken in as a file ( Ripping), it may be specified and opened. When the input digital signal is stereo, it is converted to monaural in order to simplify subsequent processing.

  This digital signal is input to the scale sound level detector 2. This scale sound level detection unit is composed of each unit shown in FIG.

  Among them, the waveform preprocessing unit 20 is configured to downsample the sound signal from the input unit 1 of the music sound signal to a sampling frequency suitable for future processing.

  The downsampling rate is determined by the range of the instrument used for beat detection. In other words, in order to reflect the performance sound of high-frequency rhythm instruments such as cymbals and hi-hats in beat detection, it is necessary to set the sampling frequency after down-sampling to a high frequency, but bass sounds, bass drums, snare drums, etc. When beat detection is mainly performed from instrument sounds and middle instrument sounds, the sampling frequency after downsampling need not be so high.

  For example, when the highest sound to be detected is A6 (C4 is in the middle), the basic frequency of A6 is about 1760 Hz (when A4 = 440 Hz), so the sampling frequency after downsampling is a Nyquist frequency of 1760 Hz or higher. It may be 3520 Hz or higher. From this, the downsampling rate may be about 1/12 when the original sampling frequency is 44.1 kHz (music CD). At this time, the sampling frequency after downsampling is 3675 Hz.

  In the downsampling process, data is skipped after passing through a low-pass filter that cuts off components above the Nyquist frequency (1837.5 Hz in this example), which is usually half the sampling frequency after downsampling (now In this example, 11 out of 12 waveform samples are discarded).

  The purpose of downsampling in this way is to reduce the FFT computation time by lowering the number of FFT points necessary to obtain the same frequency resolution in the subsequent FFT computation.

  When a sound source has already been sampled at a fixed sampling frequency, such as a music CD, such downsampling is necessary. However, the music acoustic signal input unit 1 is input from a device such as a microphone. When an analog signal is converted into a digital signal by an A / D converter, the waveform pre-processing unit is naturally set by setting the sampling frequency of the A / D converter to the sampling frequency after downsampling. It can be omitted.

  When the downsampling by the waveform preprocessing unit 20 is completed in this manner, the output signal of the waveform preprocessing unit is subjected to FFT (Fast Fourier Transform) by the FFT calculation unit 21 at predetermined time intervals.

  The FFT parameters (the number of FFT points and the shift amount of the FFT window) are values suitable for beat detection. In other words, if the number of FFT points is increased in order to increase the frequency resolution, the size of the FFT window increases, and one FFT is performed from a longer time, resulting in the FFT characteristic that the time resolution decreases. (In other words, it is better to increase the time resolution at the expense of the frequency resolution when detecting beats.) There is a method in which the time resolution is not deteriorated even if the number of FFT points is increased by setting the waveform data to only a part of the window and filling the rest with 0 without using the waveform as long as the window size. There is a certain number of waveform samples in order to correctly detect the power on the bass side (here, the power agrees with the above level; the same applies hereinafter).

  Considering the above, in this embodiment, the number of FFT points is 512, the window shift is 32 samples, and no zero padding is set. When FFT calculation is performed with such settings, the time resolution is about 8.7 ms and the frequency resolution is about 7.2 Hz. It can be seen that the time resolution of about 8.7 ms is a sufficient value considering that the tune has a tempo of quarter note = 300 and the length of the 32nd note is 25 ms.

  In this way, the FFT operation is performed at predetermined time intervals, the power is calculated from the square root of the sum of the squares of the real part and the imaginary part, and the result is sent to the level detector 22.

  The level detector 22 calculates the level of each scale sound from the power spectrum calculated by the FFT calculator 21. Since the FFT only calculates the power of a frequency that is an integer multiple of the sampling frequency divided by the number of FFT points, in order to detect the level of each scale tone from this power spectrum, the following processing is performed. I do. In other words, for all the sounds (C1 to A6) for which the scale sound is calculated, the maximum power in the power spectrum corresponding to the frequency in the range of 50 cents above and below the fundamental frequency of each sound (100 cents is a semitone) is obtained. Let the power of the spectrum it has be the scale level.

  When the levels are detected for all the scale sounds, the levels are stored in a buffer, the waveform reading position is advanced by a predetermined time interval (32 samples in the previous example), and the processing of the FFT calculation unit 21 and the level detection unit 22 is performed. Repeat until the end of the waveform.

  As described above, the level of each scale sound for each predetermined time of the acoustic signal input to the music acoustic signal input unit 1 is stored in the buffer 23.

  Next, the configuration of the beat detection unit 3 in FIG. 1 will be described. The beat detection unit 3 is executed in the process flow as shown in FIG.

  The beat detection unit 3 uses an average beat (beat) interval (based on a change in the level of each scale sound for each predetermined time (hereinafter, this one predetermined time is referred to as one frame) output from the scale sound level detection unit. That is, the tempo) and the beat position are detected. For this purpose, first, the beat detection unit 3 sums up the level increment values of each scale sound (the sum of the level increment values from the previous frame for all the scale sounds. When the level is reduced from the previous frame Is added as 0) (step S100).

That is, when the level of the i-th scale sound at the frame time t is L _i (t), the level increment value L _addi (t) of the i-th scale sound is as shown in the following equation 1, Using L _addi (t), the sum L (t) of the level increments of each scale tone at the frame time t can be calculated by the following equation (2). Here, T is the total number of scale sounds.

  The total L (t) value represents the degree of change in sound for each frame. This value suddenly increases at the beginning of sounding, and becomes larger as more sounds begin to sound at the same time. Since music often starts to sound at the beat position, there is a high possibility that the place where this value is large is the beat position.

  As an example, FIG. 4 shows a diagram of the sum of the waveform of a part of a certain song, the level of each musical note, and the level increment value of each musical note. The upper row is the waveform, the middle is the tone level of each scale in each frame (lower is lower, the upper is higher. In this figure, the range is C1 to A6), and the lower is each scale. Shows the sum of level increments. Since the level of each scale sound in this figure is output from the scale sound level detector, the frequency resolution is about 7.2 Hz, and the level cannot be calculated for some scale sounds below G # 2. In this case, since the purpose is to detect beats, it is not a problem that the level of a part of the lower tone cannot be measured.

  As seen in the lower part of the figure, the sum of the level increments of each scale sound has a peak periodically. This regular peak position is the beat position.

  In order to obtain the beat position, the beat detector 3 first obtains the periodic peak interval, that is, the average beat interval. The average beat interval can be calculated from the autocorrelation of the total level increment value of each scale note (FIG. 3; step S102).

  When the total level increment value of each scale tone in a certain frame time t is L (t), this autocorrelation φ (τ) is calculated by the following equation (3).

ここで、Ｎは総フレーム数、τは時間遅れである。

Here, N is the total number of frames, and τ is a time delay.

  A conceptual diagram of autocorrelation calculation is shown in FIG. As shown in this figure, when the time delay τ is an integral multiple of the peak period of L (t), φ (τ) takes a large value. Therefore, if the maximum value of φ (τ) is obtained for a certain range of τ, the tempo of the music can be obtained.

  The range of τ for obtaining the autocorrelation may be changed according to the assumed tempo range of the song. For example, if the range of quarter note = 30 to 300 is calculated with a metronome symbol, the range for calculating the autocorrelation is 0.2 second to 2 seconds. The conversion formula from time (seconds) to frame is as shown in the following equation (4).

  Τ with the maximum autocorrelation φ (τ) in this range may be set as the beat interval, but τ when autocorrelation is maximum in all songs is not necessarily the beat interval, so the autocorrelation is the maximum value. It is preferable to obtain beat interval candidates from τ at a certain time (FIG. 3; step S104), and let the user determine the beat interval from these multiple candidates (FIG. 3; step S106).

When the beat interval is determined in this way (the determined beat interval is set to τ _max ), the head beat position is first determined.

A method for determining the first beat position will be described with reference to FIG. The upper part of FIG. 6 is a total L (t) of the level increment values of each tone at the frame time t, and the lower part M (t) is a function having a value at the determined beat interval τ _max . This is expressed by the following equation (5).

The cross correlation between L (t) and M (t) is calculated while shifting this function M (t) in the range of 0 to τ _max −1.

  The cross-correlation r (s) can be calculated by the following equation 6 from the characteristic of M (t).

  In this case, n may be determined appropriately according to the length of the first silent portion (n = 10 in the example of FIG. 6).

If r (s) is obtained in the range of s from 0 to τ _max −1, and s at which r (s) is maximized is obtained, this s frame is the first beat position.

  When the first beat position is determined, the subsequent beat positions are determined one by one (FIG. 3; step S108).

The method will be described with reference to FIG. Assume that the first beat is found at the position of the triangle in FIG. The second beat position is determined from a position where L (t) and M (t) are most correlated in the vicinity of the temporary beat position at a position separated by a beat interval τ _max from the first beat position. That is, when the leading beat position is b ₀ , the value of s is determined so that r (s) in the following expression is maximized. In this equation, s is a deviation from the temporary beat position, and is an integer in the range of Equation 7 below. F is a fluctuation parameter, and a value of about 0.1 is appropriate. However, a larger value may be used for a song with a large tempo fluctuation. n may be about 5.

  k is a coefficient that changes in accordance with the value of s, and has a normal distribution as shown in FIG. 8, for example.

If the value of s that maximizes r (s) is obtained, the second beat position b ₁ is calculated by the following equation (8).

  Thereafter, the third and subsequent beat positions can be obtained in the same manner.

  For songs with almost the same tempo, the beat position can be obtained until the end of the song in this way.

  When the position of each beat is determined as described above, the result is stored in the buffer 30 and the detected result is displayed.

  Next, the detection of time signature and measure will be described.

  Since the position of the beat has been determined by the processing so far, the degree of change in sound for each beat is obtained next time. The degree of change in sound for each beat is calculated from the level of each scale sound for each frame output from the scale sound level detector.

When the number of frames of the j-th beat is b _j and the frames of the beats before and after the j-th beat are b _j−1 and b _{j + 1} , the degree of change in sound for each beat of the j-th beat is from the frame b _j−1. The average of the levels of each scale in the frames up to b _j −1 and the average of the levels of each scale in the frames from b _j to b _{j + 1} −1 are calculated. The degree of change in sound can be obtained and calculated by summing up all the scales.

That is, when the level of the i-th scale sound at the frame time t is L _i (t), the average level L _avg i (j) of the i-th scale sound level of the j-th beat is expressed by the following equation (9). Therefore, the sound change degree B _addi (j) for each beat of the i-th tone of the j-th beat is expressed by the following equation (10).

  Therefore, the sound change degree B (j) for each beat of the j-th beat is as shown in the following equation (11). Here, T is the total number of scale sounds.

  The bottom row in FIG. 9 shows the degree of change in sound for each beat. The time signature and the position of the first beat are obtained from the degree of change in sound for each beat.

  The time signature is obtained from the autocorrelation of the degree of sound change for each beat. In general, it is considered that the sound often changes in the first beat, so the time signature can be obtained from the autocorrelation of the sound change degree for each beat. For example, the autocorrelation φ (τ) of the sound change degree B (j) for each beat is determined in the range of 2 to 4 from the formula for obtaining the autocorrelation φ (τ) shown in the following equation (12). The delay τ that maximizes the autocorrelation φ (τ) is defined as the number of beats.

  N is the total number of beats, and φ (τ) is calculated in the range of τ = 2 to 4, and τ at which φ (τ) is the maximum is the number of beats.

Next, the first beat is obtained. This is the position where the sound change degree B (j) for each beat is the largest. That is, when phi (tau) is maximum tau and tau _max, the k of X (k) is maximum the following equation number 13 and _{k max, k max} th beat becomes the position of the first first beat Thereafter, the beat position obtained by adding τ _max is the first beat.

ｎ_ｍａｘは、τ_ｍａｘ・ｎ＋ｋ＜Ｎの条件で最大となるｎ

n _max is the _maximum n under the condition of τ _max · n + k <N

  When the time signature and the position of the first beat (bar line position) are determined as described above, the result is stored in the buffer 40. Of course, the detected result may be displayed on the screen.

  With the above premise configuration, the average tempo and accurate beat (beat) position of the entire song, and the time signature and the first beat position of the song are detected from the acoustic signal of the performance of the tempo performed by a human. It becomes possible.

  FIG. 10 is an overall block diagram of the code name detection apparatus of the present invention. In the figure, the configuration of beat detection and measure detection is basically the same as the premise configuration, and in the same configuration, the tempo detection and chord detection configurations are different from those of the premise configuration. Therefore, the same description overlaps except for mathematical formulas and the like, and is shown below.

  According to the figure, the configuration of the code name detection apparatus is configured to perform an FFT operation using a parameter suitable for beat detection at a predetermined time interval from an input acoustic signal and an input acoustic signal. And a beat detection scale level detector 2 for obtaining the scale level for each predetermined time, and adding the increment value of each scale level for each predetermined time for all the scales The sum of the level increments indicating the degree of change in the overall sound for each time period is obtained, and from the sum of the level increment values indicating the degree of change in the overall sound for each predetermined time, the average beat interval and each The beat detection unit 3 that detects the position of the beat, the average value of the level of each scale sound for each beat, and the increment value of the average level of each scale sound for each beat are summed for all the scale sounds. , Changes in overall sound for each beat A value indicating the match is obtained, and from the value indicating the degree of change in the overall sound for each beat, the measure detecting unit 4 for detecting the time signature and the position of the measure line, and the time of the previous beat detection from the input acoustic signal. A chord detection scale sound level detection unit 5 that performs an FFT operation using a parameter suitable for chord detection at a predetermined time interval different from the above and obtains the level of each scale sound for each predetermined time; Among the levels of each scale, the bass sound detection unit 6 that detects a bass sound from the level of the low-frequency scale sound in each measure, and the chord name of each measure from the detected bass sound and the level of each scale sound Chord information for storing a chord position, a bass scale tone intensity, a bass tone, a chord scale tone intensity, a chord constituent tone, a chord constituent tone number, and a chord name for every chord detected Storage unit 8 A display unit 9 for displaying the chord name or chord component sound and / or bass sound determined by the chord name determination unit 7, a performance unit 10 for playing the input sound signal and the determined chord, and the performance An adjustment unit 11 that adjusts the volume balance of both outputs of the unit 10; a change unit 12 that changes a chord name or chord constituent sound; and a chord information storage unit based on a change of the chord name or chord constituent sound by the change unit 11 The chord scale sound intensity and chord constituent sound stored in FIG. 8 are referred to, the chord constituent sound, the number of chord constituent sounds, and the chord name are changed by the chord name determining section 7, and the change information is sent to the chord information storing section 8. And a resetting unit 13 for re-displaying the chord name or chord constituent sound on the display unit 9. In the figure, the acoustic signal input to the input unit 1 is also stored separately in the buffer 10a and used for performance by the subsequent performance unit 10.

  The chord information storage unit 8 is configured by an internal or external storage device of a computer, and as described above, for each detected chord, chord position, bass range tone intensity, bass tone, chord tone intensity Have a function of storing a chord constituent sound, the number of chord constituent sounds, and a chord name.

  The display unit 9 is composed of a computer display, and has a function of displaying the chord name or chord constituent sound determined by the chord name determining unit 7 and / or the bass sound as described above. Yes.

  The performance section 10 is composed of an audio output section of a computer (or an external musical sound output section from a MIDI interface connected to the computer), and is input from the input section 1 and stored in the buffer 10 as described above. The sound signal and the code determined by the code name determination unit 7 are output together as a sound.

  The adjustment unit 11 includes an operator having a slider (see FIG. 15 described later) displayed on a display by a program that also runs on the computer, and a pointing device connected to the computer. It has a function of adjusting the volume balance between the 10 input acoustic signals and the output of both chords (see FIG. 17 described later).

  The change unit 12 includes a cursor displayed on the computer display and a pointing device connected to the computer. As described above, the change unit 12 changes the chord name or the chord sound displayed on the display unit 9. (Many change the number of constituent sounds).

  The resetting unit 13 is configured by a CPU of a computer that reads and executes a code name detection program and performs the processing described below. The chord scale sound intensity and chord constituent sound stored in the chord information storage section 8 are referred to, the chord constituent sound, the chord constituent sound number, and the chord name are changed by the chord name determining section 7, The change information is re-stored in the chord information storage unit 8 and at least the display unit 9 has a function of re-displaying the chord name or chord constituent sound. In addition, an acoustic signal and a chord may be played again by the performance unit 10.

  The input unit 1 for inputting a music acoustic signal is a part for inputting a music acoustic signal to be subjected to chord detection, but the basic configuration is the same as that of the input unit 1 of the first embodiment. Description is omitted. However, if vocals normally localized at the center are disturbed by later code detection, vocal cancellation may be performed by subtracting the waveform of the right channel and the waveform of the left channel.

  This digital signal is input to the beat detection scale level detector 2 and the chord detection scale level detector 5. Each of these scale sound level detection units is composed of the respective units shown in FIG. 2 and has the same configuration. Therefore, the same components can be reused by changing only the parameters.

  The waveform pre-processing unit 20 used as the configuration has the same configuration as described above, and down-samples the acoustic signal from the input unit 1 of the music acoustic signal to a sampling frequency suitable for future processing. However, the sampling frequency after downsampling, that is, the downsampling rate, may be changed for beat detection and chord detection, or may be the same in order to save time for downsampling.

  In the case of beat detection, the downsampling rate is determined by the range used for beat detection. In order to reflect the performance sound of high-frequency rhythm instruments such as cymbals and hi-hats in beat detection, the sampling frequency after down-sampling needs to be set to a high frequency, but the bass sound and instrument sounds such as bass drum and snare drum In the case of detecting beats mainly from instrument sounds in the middle range, the same downsampling rate as that in the following chord detection may be used.

  The down-sampling rate of the chord detection waveform pre-processing unit varies depending on the chord detection range. The chord detection tone range is a tone range used when chord detection is performed by the chord name determination unit. For example, if the chord detection sound range is C3 to A6 (C4 is the center), the basic frequency of A6 is about 1760 Hz (when A4 = 440 Hz), so the sampling frequency after downsampling is a Nyquist frequency of 1760 Hz or higher. It may be 3520 Hz or higher. From this, the downsampling rate may be about 1/12 when the original sampling frequency is 44.1 kHz (music CD). At this time, the sampling frequency after downsampling is 3675 Hz.

  In the downsampling process, data is skipped after passing through a low-pass filter that cuts off components above the Nyquist frequency (1837.5 Hz in this example), which is usually half the sampling frequency after downsampling (now In this example, 11 out of 12 waveform samples are discarded). This is for the same reason as described in the premise configuration.

  When the downsampling by the waveform preprocessing unit 20 is completed in this manner, the output signal of the waveform preprocessing unit is subjected to FFT (Fast Fourier Transform) by the FFT calculation unit 21 at predetermined time intervals.

  The FFT parameters (the number of FFT points and the shift amount of the FFT window) are different values at the time of beat detection and code detection. This is because if the number of FFT points is increased to increase the frequency resolution, the size of the FFT window increases, and one FFT is performed from a longer time, resulting in a decrease in time resolution. (In other words, it is better to increase the time resolution at the expense of frequency resolution when detecting beats). A method that does not deteriorate the time resolution even if the number of FFT points is increased by setting waveform data to only a part of the window and filling the rest with 0 without using a waveform with the same length as the window size. However, in the case of the present embodiment, a certain number of waveform samples are necessary in order to correctly detect the power on the bass side.

  In consideration of the above, in this embodiment, the number of FFT points is 512 at the time of beat detection, the window shift is 32 samples, 0 padding is not performed, the number of FFT points is 8192 at the time of code detection, and the window shift is 128 samples. Then, 1024 samples were used for the waveform sample in one FFT. When FFT calculation is performed with such a setting, the time resolution is about 8.7 ms and the frequency resolution is about 7.2 Hz when the beat is detected, and the time resolution is about 35 ms and the frequency resolution is about 0.4 Hz when the code is detected. Since the scale tone for which the level is to be obtained is in the range from C1 to A6, the frequency resolution of about 0.4 Hz at the time of detecting the chord is the difference between the basic frequency of C1 and C # 1 having the smallest frequency difference, about 1 .9 Hz is also supported. Considering that the length of a 32nd note is 25 ms in a song with a tempo of quarter note = 300, it can be seen that the time resolution of about 8.7 ms at the time of beat detection is a sufficient value.

  In this way, the FFT operation is performed at predetermined time intervals, the power is calculated from the square root of the sum of the squares of the real part and the imaginary part, and the result is sent to the level detector 22.

  The level detector 22 calculates the level of each scale sound from the power spectrum calculated by the FFT calculator 21. Since FFT only calculates the power of a frequency that is an integer multiple of the value obtained by dividing the sampling frequency by the number of FFT points, in order to detect the level of each scale tone from this power spectrum, the same as in the premise configuration Perform proper processing. That is, for all the sounds (C1 to A6) for which the scale sound is calculated, the maximum power in the power spectrum corresponding to frequencies in the range of 50 cents above and below the fundamental frequency of each sound (100 cents is a semitone) is obtained. Let the power of the spectrum it has be the scale level.

  When the levels are detected for all the scale sounds, this is stored in the buffer, and the waveform read position is advanced by a predetermined time interval (32 samples at the time of beat detection and 128 samples at the time of chord detection in the previous example), and FFT is performed. The processing of the calculation unit 21 and the level detection unit 22 is repeated until the end of the waveform.

  As described above, the level of each scale sound of the sound signal input to the music sound signal input unit 1 for each predetermined time is stored in the two types of buffers 23 and 50 for beat detection and chord detection.

  Next, since the configurations of the beat detection unit 3 and the bar detection unit 4 in FIG. 10 are the same as those of the beat detection unit 3 and the bar detection unit 4 of the premise example configuration, detailed description thereof will be omitted here.

  Since the position of the bar line (frame number of each bar) has been determined with the same configuration and procedure as the precondition example, the bass sound of each bar is detected this time.

  The bass sound is detected from the scale level of each frame output by the chord detection scale level detector 5.

  FIG. 11 shows the scale level of each frame output by the chord detection scale level detector 5 of the same part of the same composition as in FIG. As shown in this figure, since the frequency resolution in the chord detection scale sound level detector 5 is about 0.4 Hz, the levels of all the scale sounds C1 to A6 are extracted.

  Since the bass sound may be different between the first half and the second half of the measure, the bass sound detection unit 6 detects the first half and the second half of each measure. When the first half and the second half are the same, this is confirmed as the bass of the measure, and the chord is also detected in the entire measure. When different bass sounds are detected in the first half and the second half, the chord is also detected separately in the first half and the second half. In some cases, the range to be divided may be further reduced to half (up to a quarter of the bar).

  The bass sound is obtained from the average intensity of the scale sound level in the bass detection range during the bass detection period. That is, this is the intensity of the bass sound.

When the level of the i-th note in the scale at frame time t and _L i (t), the average level of the i th scale notes of _{f e} from the frame _{f s L avgi (f s,} f e) is the following formula It can be calculated by Equation 14.

  This average level is calculated in the bass detection range, for example, in the range from C2 to B3, and the bass sound detection unit 6 determines the scale tone having the highest average level as the bass sound. An appropriate threshold is set to prevent the bass sound from being erroneously detected in songs or silences that do not include sound in the bass detection range, and if the average level of the detected bass sound is below this threshold, Sound may not be detected. In addition, when the bass sound is important in later chord detection, it is checked whether the detected bass sound keeps a certain level or more continuously during the bass detection period, and only the more reliable ones are checked. You may make it detect as a bass sound. Further, instead of determining the scale tone having the highest average level in the bass detection range as the base tone, the average level of each pitch name is averaged for every 12 pitch names, The pitch name having the highest level may be determined as the bass pitch name, and the scale tone having the highest average level among the scale sounds in the bass detection range having the pitch name may be determined as the bass tone.

  When the bass sound is determined, the result may be stored in the buffer 60, and the bass detection result may be displayed on the screen so that the user can correct it if it is incorrect. Further, since the bass range may be changed depending on the song, the user may be able to change the bass detection range.

  FIG. 12 shows a display example of the bass detection result by the bass sound detection unit 6.

  Next, the chord detection process by the chord name determination unit 7 is also determined by calculating the average level of each tone in the chord detection period. In other words, this is the strength of the cord.

  In this embodiment, the code detection period and the base detection period are the same. The average level in the chord detection period, for example, the C3 to A6 scales in the chord detection period is calculated, and several pitch names are detected in order from the scale that has the largest value, and the pitch names of the bass sounds Extract code name candidates from.

  The code information storage unit 8 is configured to store the detected data, and is configured by an internal or external storage device of the computer.

  In the chord name detection process, in the chord name detection process, the chord position stores the first frame number (chord 1, chord 2,...) Of the bass tone detection period, The average of the intervals of the 12 sounds in the bass sound range from C to B in the bass sound detection period is stored. Since the frame number can also be converted into a sample, it may be stored as a sample number. The bass sound stores the scale number of the bass sound detected during the bass sound detection period. The chord scale sound intensity stores a section average of the intensity of 12 sounds from C to B in the base detection period, and the chord constituent sound stores the chord constituent sound extracted in the base detection period. In the code name, the determined code name (or a corresponding number) may be stored.

  The code name determination unit 7 refers to the code information storage unit 8 to determine the code name. The chord name determination unit 7 searches for one chord name and its chord constituent sound from a chord name database in which the chord type (m, M7, etc.) and the pitch from the root tone of the chord constituent sound are stored. The average intensity of the chord constituent sound is calculated from the chord scale sound intensity of the chord information storage unit 8. The chord name having the highest chord constituent sound average intensity of all chords is determined as the chord name of the section. At this time, the chord root sound (five tone) and the fifth sound may be omitted in the musical instrument playing the chord, so that even if they are not included, they are extracted as chord name candidates. When a bass tone is detected, the pitch name of the bass tone is added to the chord name of this chord name candidate. In other words, if the chord root sound and the bass sound have the same pitch name, they can be left as they are. Alternatively, a plurality of chord constituent sound average intensities may be displayed so that the user can select them.

  In the above method, when there are too many code name candidates to be extracted, limitation by bass sound may be performed. That is, when a bass sound is detected, chord name candidates whose root sound is not the same as the base sound are deleted.

  Furthermore, musical knowledge may be introduced into the calculation of the chord constituent sound average intensity. For example, the level of each musical note is averaged over all frames, and is averaged for every 12 pitch names to calculate the strength of each pitch name, and the key of the song is detected from the distribution of the strength. Then, the key diatonic chord is multiplied by a certain constant so that the average intensity of the chord constituent sound is increased, or the chord that includes the sound deviating from the sound on the key diatonic scale is included in the tone of the off sound. It is conceivable that the chord constituent sound average intensity is reduced according to the number. In addition, by storing a pattern of common chord progressions as a database and comparing it with the ones that are frequently used among chord candidates, a certain constant is applied so that the average intensity of chord constituent sounds increases. You may do it.

  In any case, when the code name is determined by the code name determination unit 7, the result (change information) is re-stored in the code information storage unit 8 via the buffer 70.

  In this embodiment, the input acoustic signal is a signal having a sampling frequency of 44.1 kHz. In this embodiment, it is assumed that the time signature is 4/4 and the bar position is detected as shown in FIG. The unit of the bar line position in the figure is Sample.

  At this time, the tempo of the song can be calculated from the bar line position, time signature, and sampling frequency as follows. The tempo is expressed as the number of quarter notes per minute.

Bar _n is n-th measure line position of the above formula number 15, Beat the meter of the molecule, it fs is the sampling frequency. From the above formula 15, Tempo is 120 in this embodiment.

  FIG. 14 shows the result of determining the code name. This figure shows a part of the code information storage unit 8 in this embodiment. The numbers of Bass and chord constituent sounds are represented by 0 to 11 from pitch names C to B. -1 means that there is no chord constituent sound. The maximum number of chord constituent sounds is seven. The unit of the code position is assumed to be Sample.

  When the chord name is detected, the display unit 9 displays the input sound signal, the level increment value, the beat position, the bar position, and the chord as shown in FIG.

  In addition, the chord name determination unit 7 creates chord performance information corresponding to the detected chord and stores it in the buffer 70.

  Next, creation of performance information will be described.

  The data structure of the performance information is shown in FIG. This conforms to SMF (standard MIDI file), and its description is omitted. In the figure, the header data includes an identifier indicating the header data, the number of tracks, and a quarter note resolution.

  The track header data includes an identifier indicating the track header data, a data size, and event data. This event data consists of event time, status, and data.

  The number of data varies depending on the status. For example, in the case of pronunciation, status = 0 × 90, data 1 represents the pitch, and data 2 represents the strength of the pronunciation (volume). In 0x90, 0 represents a MIDI channel, and 9 represents sound generation. Further, when sound generation is stopped, status = 0 × 80, data 1 is the pitch, and data 2 is the sound generation stop strength. (The strength of sound generation stop represents the speed at which the pressed keyboard returns on the keyboard instrument.)

  The event time represents an elapsed time from the start of performance. The unit is Tick, and the quarter note resolution defined by the header data is used. A step time (elapsed time from the previous event) may be used instead of the event time.

  First, the code start position and code end position are converted from Sample to Tick by the following equation (16).

Chord _n indicates the chord start or end position of the nth chord, fs indicates the sampling frequency, Tempo indicates the tempo of the song, and Div indicates the quarter note resolution.

  The start position of each code is converted to 0, 1920, 3840, 5760, 7680, and 9600 Tick, respectively. Since the Tick needs to be an integer, the end position of each code becomes 1920, 3840, 5760, 7680, 9600, and 11520, respectively, by rounding off after the decimal point.

  In the event data of chord C, sounding events of pitches 60 (center), 64 (mi), 67 (so) and sounding stop events are written.

  The event time of each sounding event is the chord start position converted into Tick, obtained from the above equation (16).

  The status of each sounding event is 0x90 according to MIDI.

  Since the data 1 of each sounding event represents the pitch, it becomes 0 × 3C, 0 × 40, and 0 × 48, respectively.

  The data 2 of each sounding event is an arbitrary value because it is sounding intensity.

  The event time of the sound generation stop event is set to be about 90% from the start to the end of the chord. This is obtained by the following equation (17).

ChordS _n indicates the start position of the _nth code, and ChordE _n indicates the end position of the nth code. Since the first stop event time is also Tick, round off the decimal point. Then, the event time of each sound generation stop event is 1728. The status of each sound generation stop event is set to 0 × 80 according to MIDI. Since the data 1 of each sound generation stop event represents a pitch, they are 0 × 3C, 0 × 40, and 0 × 48, respectively. The data 2 of each sounding event is an arbitrary value because it is the sounding stop strength.

  Create performance data for all chords as described above.

  After the performance information is created by the above operation, the display unit 9, the performance unit 10, the adjustment unit 11, the change unit 12 and the resetting unit 13 confirm with the user whether the chord detection so far has been performed correctly. And make changes if necessary.

  When the user presses the performance start button 100 shown in FIG. 15 after detection, the input sound signal input through the buffer 10a and the chord read out from the chord information storage unit 8 are output as musical sounds by the performance unit 10. Is done. In this case, the chord is played in the range from the performance start position indicated by S in the bar acoustic signal of bar number 2 in FIG. 15 to the performance end position indicated by E in the bar acoustic signal of bar number 6. To do. These can be moved to any position by clicking the bar number display area with the mouse.

  When the performance start button 100 is pressed, the performance of the input sound signal and chord is started, and the performance ends when the performance reaches the performance end position or the performance end button 102 is pressed.

  A functional block diagram of the chord information storage unit 8, the performance unit 10, and the adjustment unit 11 is shown in FIG.

  When the performance start button 100 is pressed to start the performance, the performance unit 10 reads the input acoustic signal from the buffer 10a storing the input acoustic signal of FIG. Accordingly, the chord performance information is read from the chord performance information storage unit 8 by the chord reading unit 104 of the performance unit 10, and the musical tone signal is read from the tone generator unit 106. The chord reading unit 104 responds accordingly. A chord performance sound signal is generated and output.

  The amplitude of the read input acoustic signal is adjusted by the input acoustic signal volume 112. Similarly, the chord performance sound signal is adjusted in amplitude by the chord performance sound signal volume 110. The adjusting unit 11 described above includes the chord performance sound signal volume 110 and the input sound signal volume 112.

  The amplitude-adjusted input sound signal and chord performance sound signal are added by the adder 114 and written to the reproduction sound signal buffer 108.

  The playback sound signal is D / A converted by the sound output means of the computer system, is sounded to the outside through a speaker or the like, and is played back.

  As described above, the adjustment unit 11 includes the chord performance sound signal volume 110 and the input sound signal volume 112. More specifically, the adjustment section 11 is linked to the chord performance sound signal volume 110 and the input sound signal volume 112, respectively. By operating the operator (slider) of the adjustment unit 11 displayed on the display unit 9, the values of both the chord performance sound signal volume 110 and the input sound signal volume 112 change.

  By listening and comparing both of these sounds, the user can ascertain whether the chord has been detected in error.

  If the user listens and compares many times and determines that the code is wrong, the change unit 12 moves the cursor to the code display position displayed on the display unit 9 of the computer, and displays the code name or code displayed there. Change chord composition sound.

  The resetting unit 13 refers to the chord scale sound intensity and the chord constituent sound stored in the chord information storage unit 8 based on the change of the chord name or the chord constituent sound by the changing unit 12, and performs the chord constituent sound and chord constituent sound. The number of sounds and the chord name are changed by the chord name determination unit 7, the change information is re-stored in the chord information storage unit 8, and at least the display unit 9 has a function of re-displaying the chord name or chord constituent sound. doing. In response to the user's request, the performance unit 10 can play the sound signal and the chord again.

  If only the chord name detected in this way is displayed on the screen, it cannot be determined whether it is correct just by looking at the displayed chord name or chord constituent sound, but the input sound signal and the detected chord are reproduced. In this case, the output is adjusted by the adjustment unit 11 so that the user can easily determine both sounds. This is very effective for confirming the code detection.

According to the configuration of the present embodiment described above, processing that requires time resolution of beat detection with the simple configuration only (the same as the configuration of the tempo detection device of the above premise example) and processing that requires frequency resolution of code detection ( (A configuration capable of further detecting a chord name based on the configuration of the tempo detection device), and a plurality of musical instrument sounds such as a music CD, even if not an expert having special musical knowledge. In addition to being able to detect the chord name from the overall reverberation without detecting individual note information, it is possible not only to discriminate even if the constituent sounds are the same chord, user heard by playing unit 10 and the detection result is a chord progression on unnatural chord notes, when you want to make corrections on the detection result by using the changing unit 12, the user side, the code You can change or chord member, so that it is determined the appropriate code name. Alternatively, the user can make corrections so that they feel natural when listening to the chord progression. Further, when listening to the reproduction of such an input sound signal and the detected code, the adjustment unit 11 can easily adjust the output so that the user can easily determine both sounds.

  The code name detection apparatus of the present invention and the program that can realize the code name detection apparatus are not limited to the illustrated examples described above, and various changes can be made without departing from the scope of the present invention. .

  The code name detection device and the code name detection program of the present invention can be performed by video editing processing for synchronizing an event in a video track with a beat time in a music track when creating a music promotion video, or by beat tracking. Audio editing processing that finds beat positions and cuts and pastes the sound signal waveform of music, controls elements such as lighting color, brightness, direction, special effects, etc. in sync with human performances, audience clapping, cheers, etc. It can be used in various fields, such as live stage event control for automatically controlling music, computer graphics synchronized with music, and the like.

1 is an overall block diagram of a tempo detection device as a premise of the present invention. It is a block diagram of a structure of a scale sound level detection part. 4 is a flowchart showing a flow of processing of a beat detection unit 3. It is a graph which shows the figure of the sum total of the waveform of the part of a certain music, the level of each scale sound, and the level increment value of each scale sound. It is explanatory drawing which shows the concept of autocorrelation calculation. It is explanatory drawing explaining the determination method of the first beat position. It is explanatory drawing which shows the method of determining the position of the beat after it after the first beat position determination. It is a graph which shows the distribution state of the coefficient k changed according to the value of s. It is a screen display figure which shows the example of the confirmation screen of a bar detection result. BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is an overall block diagram of a code name detection apparatus according to a first embodiment of the present invention. It is a graph which shows the level of the scale sound of each flame | frame output from the chord detection scale level detection part 5 of the same part of a music. It is a graph which shows the example of a display of the bass detection result by the bass sound detection part. It is explanatory drawing which shows an example of the bar line position detected in the present Example. It is explanatory drawing which shows the data storage state which shows the state by which the code name was determined in the present Example. It is explanatory drawing which shows the example of a screen display which performs display of a detection result, and performance operation. It is explanatory drawing which shows the data structure of performance information. 3 is an explanatory diagram showing functional blocks of a performance unit 10 and an adjustment unit 11. FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Input part 2 Beat detection scale sound level detection part 3 Beat detection part 4 Measure detection part 5 Chord detection scale sound level detection part 6 Bass sound detection part 7 Code name determination part 8 Code information storage part 9 Display part 10 Performance part DESCRIPTION OF SYMBOLS 11 Adjustment part 12 Change part 13 Reset part 20 Waveform pre-processing part 21 FFT operation part 22 Level detection part 10a, 23, 30, 40, 50, 60, 70, 108 Buffer 100 Performance start button 102 Performance end button 110 Chord performance Acoustic signal volume 112 Input acoustic signal volume 114 Adder

Claims (2)

An input means for inputting an acoustic signal;
First scale sound level detection means for performing FFT calculation using a parameter suitable for beat detection at predetermined time intervals from the input acoustic signal, and obtaining the level of each scale sound for each predetermined time;
The increment value of each scale sound level for each predetermined time is summed for all the scale sounds to obtain a total of level increment values indicating the degree of change in the overall sound for each predetermined time. Beat detection means for detecting the average beat interval and the position of each beat from the sum of the incremental values of the level indicating the degree of change in the overall sound for each,
The average value of the scale level for each beat is calculated, and the increment value of the average level of each scale sound for each beat is added for all the scale sounds to indicate the degree of change in the overall sound for each beat. A bar detecting means for obtaining a value and detecting a time signature and a bar line position from a value indicating a change degree of the whole sound for each beat;
From the input acoustic signal, an FFT operation is performed using a parameter suitable for chord detection at a predetermined time interval different from that at the time of the previous beat detection, and the level of each scale sound for each predetermined time is calculated. Second scale level detection means to be obtained;
Bass sound detection means for detecting a bass sound from the level of the low-frequency scale sound in each measure out of the detected scale levels,
Chord name determining means for determining the chord name of each measure from the detected bass sound and the level of each scale sound;
For every detected chord, from the chord position, the base tone scale intensity obtained from the scale sound level of the bass detection range during the base detection period, the base tone, and the scale sound level of the chord detection range during the chord detection period Chord information storage means for storing the required chord scale sound intensity, chord constituent sound, chord constituent sound number, chord name;
Display means for displaying the chord name or chord constituent sound and / or bass sound determined by the chord name determining means;
Performance means for playing the input acoustic signal and the determined chord;
Adjusting means for adjusting the volume balance of both outputs of the performance means;
By increasing or decreasing the threshold value that is set for the intensity of the scale sound that is the constituent sound, the number of chord constituent sounds can be increased or decreased in descending order of the chord scale sound intensity. Change means to change the sound;
Based on the change of the chord name or chord constituent sound by the changing means, the chord constituent sound, the chord constituent sound number, and the chord name are referred to by referring to the chord scale intensity and chord constituent sound stored in the chord information storing means. Code name detection characterized by having a name determining means for changing, re-storing the changed information in the code information storage means, and at least a resetting means for causing the display means to re-display the chord name or the chord sound. apparatus. By being read and executed by a computer, the computer is
An input means for inputting an acoustic signal;
First scale sound level detection means for performing FFT calculation using a parameter suitable for beat detection at predetermined time intervals from the input acoustic signal, and obtaining the level of each scale sound for each predetermined time;
The increment value of each scale sound level for each predetermined time is summed for all the scale sounds to obtain a total of level increment values indicating the degree of change in the overall sound for each predetermined time. Beat detection means for detecting the average beat interval and the position of each beat from the sum of the incremental values of the level indicating the degree of change in the overall sound for each,
The average value of the scale level for each beat is calculated, and the increment value of the average level of each scale sound for each beat is added for all the scale sounds to indicate the degree of change in the overall sound for each beat. A bar detecting means for obtaining a value and detecting a time signature and a bar line position from a value indicating a change degree of the whole sound for each beat;
From the input acoustic signal, an FFT operation is performed using a parameter suitable for chord detection at a predetermined time interval different from that at the time of the previous beat detection, and the level of each scale sound for each predetermined time is calculated. Second scale level detection means to be obtained;
Bass sound detection means for detecting a bass sound from the level of the low-frequency scale sound in each measure out of the detected scale levels,
Chord name determining means for determining the chord name of each measure from the detected bass sound and the level of each scale sound;
For every detected chord, from the chord position, the base tone scale intensity obtained from the scale sound level of the bass detection range during the base detection period, the base tone, and the scale sound level of the chord detection range during the chord detection period Chord information storage means for storing the required chord scale sound intensity, chord constituent sound, chord constituent sound number, chord name;
Display means for displaying the chord name or chord constituent sound and / or bass sound determined by the chord name determining means;
Performance means for playing the input acoustic signal and the determined chord;
Adjusting means for adjusting the volume balance of both outputs of the performance means;
By increasing or decreasing the threshold value that is set for the intensity of the scale sound that is the constituent sound, the number of chord constituent sounds can be increased or decreased in descending order of the chord scale sound intensity. Change means to change the sound;
Based on the change of the chord name or chord constituent sound by the changing means, the chord constituent sound, the chord constituent sound number, and the chord name are referred to by referring to the chord scale intensity and chord constituent sound stored in the chord information storing means. Code name detection characterized in that it is changed to a name determining means, and the change information is re-stored in the code information storage means, and at least the display means functions as a resetting means for re-displaying a chord name or a chord constituent sound. Program.

Images