JP5153517B2 - Code name detection device and computer program for code name detection - Google Patents

Description

  The present invention relates to a chord name detecting device and a chord name detecting computer program for detecting a chord name being played from a music sound signal (audio signal) such as a music CD.

  So far, a technique for automatically recording a chord name from an audio signal has been disclosed, and the present applicant has filed a similar configuration.

  Usually, music is composed of repeated passages. A passage is a single unit in a music format, usually about 8 bars. Often, terms like A melody, B melody, and chorus are used, but A melody, B melody, and chorus are one syllable unit.

  However, when one long piece of music is chord-detected, the same passage will appear many times, but with conventional chord name detection devices and programs with the same purpose, there are errors in bar division and chord name detection results. Had to be fixed.

  In the conventional code name detection device, due to its characteristics, the same detection error often occurs in the same passage. Therefore, it is necessary for the user to repeat the same correction many times, which is a very time-consuming work.

  The present invention was devised in view of the above-mentioned problems, and while improving the detection accuracy of the code name detection result and making corrections at one place by utilizing the technology for improving the accuracy, it immediately The code name detecting apparatus and the code name detecting computer program capable of performing the same correction are also provided.

Therefore, in the present invention, a configuration has been devised in which after detecting a code, the same passage portion is detected from the similarity of the detected code. Its specific configuration is
In the chord name detection device for detecting beats, detecting bars, and detecting / determining chords from the acoustic signals for the input acoustic signals,
A performance means for performing the acoustic signal;
A first input means for receiving a break of a passage including a plurality of measures according to the performance;
The number of measures for each passage is determined from the positions of the above-mentioned passages, and the similarities are checked by comparing the character strings of the code names or the constituent sounds of the code names detected from those with the same number of measures. A passage similarity detection means for assigning a unique ID to the passages having a similarity equal to or greater than a predetermined threshold,
Display means for displaying to the user the performance status of the acoustic signal, the section separation status, and the similarity detection status including the passage with the unique ID assigned;
A second input means for accepting determination of the similarity detection status by the user;
When it is input that the determination of the similarity detection status is not fixed, the threshold value is changed, and the similarities of the sections are re-detected by the section similarity detecting means, and the determination is confirmed. The basic feature is that, when input, the same section portion has section similarity determination means for redetecting the same section division and the same code.

  In the above configuration, after detecting the code, the same passage portion is detected by the passage similarity detection means from the similarity of the detected code. At this time, the user inputs the passage of the passage by the first input means. There is only a delimiter position, and there is no need to be aware of whether it is A melody or B melody. The same number of measures are compared with each other from the specified segment breaks, and if the chord progression is the same, the same passage is detected. In determining the similarity, as described above, the section similarity detection means calculates the number of bars for each section, and the character string of the code name detected from those having the same number of bars or The constituent sounds of the chord names are compared, and whether or not the similarity representing the similarity is equal to or higher than a predetermined threshold is determined.

  After the similarity is checked, a unique ID is assigned to the passages whose similarity representing the similarity is equal to or more than a predetermined threshold by the above-described passage similarity detection means. This is for facilitating the work of confirming as it is after the change of the similarities of the passages whose similarity is equal to or higher than a specific threshold by changing at a stroke or at once.

  Of course, since the similarity detection of the passage is not necessarily correct by the above-mentioned passage similarity detection means, in the present configuration, the detection status is displayed by the display means before confirmation (and if necessary, the performance is performed). The original sound signal is played by the means), and finally the determination by the user is accepted using the second input means for the similarity detection situation.

  If the user inputs that the determination of the similarity detection status is not confirmed by the second input means, the passage similarity determination means changes the threshold value to the passage similarity detection means. The similarity of each section will be rediscovered.

  On the other hand, when the similarities of the individual passages are not detected again, or after such a redetection is made and the above determination is confirmed, the same passage portion (the same unique ID) is input when input by the second input means. The above-mentioned section similarity determination means re-detects the same section division and the same code. This is because, since it is determined that there is similarity between the sections, the same section should be divided and arranged in the same section, and the codes in each section should be the same.

  In addition, when the same section portion is re-detected so as to have the same measure division and the same code by the above-mentioned measure similarity determination means, it is further provided with a correction means capable of dividing the bar and / or correcting the code. good. The second invention proposes the configuration as described above. That is, by the above-mentioned section similarity determination means, the same section portion should have the same measure division and the same code, but if it is wrong, the correction means can correct these errors by the user. To. In that case, the portion determined to be the same passage by the passage similarity determination means is corrected by the correction means so that it becomes the same bar division and the same code by the above-mentioned passage similarity determination means. With such a configuration, it is possible to correct the same measure of the same passage at the same passage portion only by correcting the erroneous portion, and to greatly reduce the trouble of correction.

  In the above-described configuration, as described above, after detecting a code, the same section portion is detected by the section similarity detecting means from the similarity of the detected code. In this case, the user inputs the first section It is only the section position of the passage by the input means, and there is no need to be aware of whether it is A melody or B melody. The same number of measures are compared with each other from the specified segment breaks, and if the chord progression is the same, the same passage is detected. However, if the user knows in advance that the passage is A melody, B melody, or chorus, the overall structure of the entire song will be clear from the beginning, and the detection accuracy will be further improved. It is desirable that the section of the passage received by the first input means further receives the input of a passage such as A melody, B melody, and chorus. The third invention defines such a configuration.

Furthermore, among the above configurations, as an example of a configuration that performs beat detection, measure detection, and chord detection / determination from an input acoustic signal,
An input means for inputting an acoustic signal;
First scale sound level detection means for performing FFT calculation using a parameter suitable for beat detection at predetermined time intervals from the input acoustic signal, and obtaining the level of each scale sound for each predetermined time;
The increment value of each scale sound level for each predetermined time is summed for all the scale sounds to obtain a total of level increment values indicating the degree of change in the overall sound for each predetermined time. Beat detection means for detecting the average beat interval and the position of each beat from the sum of the incremental values of the level indicating the degree of change in the overall sound for each,
The average value of the scale level for each beat is calculated, and the increment value of the average level of each scale sound for each beat is added for all the scale sounds to indicate the degree of change in the overall sound for each beat. A bar detecting means for obtaining a value and detecting a time signature and a bar line position from a value indicating a change degree of the whole sound for each beat;
From the input acoustic signal, an FFT operation is performed using a parameter suitable for chord detection at a predetermined time interval different from that at the time of the previous beat detection, and the level of each scale sound for each predetermined time is calculated. Second scale level detection means to be obtained;
Bass sound detection means for detecting a bass sound from the level of the low-frequency scale sound in each measure out of the detected scale levels,
Chord name determining means for determining the chord name of each measure from the detected bass sound and the level of each scale sound;
For every detected chord, from the chord position, the base tone scale intensity obtained from the scale sound level of the bass detection range during the base detection period, the base tone, and the scale sound level of the chord detection range during the chord detection period It is necessary to have at least a chord information storage means for storing the required chord scale sound intensity, chord constituent sound, chord constituent sound number, and chord name.

  In the above-described configuration, the scale level for each predetermined time is obtained by the scale level detection means from the acoustic signal input to the input means, and the level of each scale tone for each predetermined time is determined by the beat detection means. The increment value is summed up for all the scale sounds to obtain the sum of the increment value of the level indicating the change degree of the whole sound every predetermined time, and the change of the whole sound every predetermined time is also obtained by the beat detection means. The average beat (beat) interval (that is, tempo) and the position of each beat are detected from the sum of the level increments indicating the degree, and then the measure of each scale tone for each beat is detected by the above bar detecting means. The average value is calculated, and the average level increment of each scale note for each beat is summed for all scale sounds to obtain a value indicating the degree of change in the overall sound for each beat. sound From the value indicating the degree of change, it will detect the time signature and bar line position (first beat position).

  That is, the level of each scale sound for each predetermined time is obtained from the input sound signal, and the average beat (beat) interval (that is, tempo) and each beat are determined from the change in the level of each scale sound for each predetermined time. Next, the time signature and bar line position (position of the first beat) are detected from the change in the level of each scale tone for each beat.

  In the bass sound detecting means, when a plurality of bass sounds are detected in a measure, the chord name determining means divides the measure into several chord detection ranges according to the bass sound detection result. The chord name in each chord detection range is determined from the base sound and the level of each scale sound in each chord detection range.

  According to the above configuration, the first acoustic scale level detection means first performs an FFT operation with a parameter suitable for beat detection on the input sound signal input from the input means at a predetermined time interval. The level of each scale sound for each time is obtained, and the beat detection means detects the average beat interval and the position of each beat from the change in the level of each scale sound for each predetermined time. Next, the measure and the bar line position are detected from the change in the level of each scale sound for each beat by the measure detecting means. Furthermore, the chord name detection apparatus according to the present invention is suitable for chord detection at a predetermined time interval different from the time of the previous beat detection with respect to the input acoustic signal by the second scale sound level detection means. An FFT operation is performed with the parameters, thereby obtaining the level of each scale sound for each predetermined time. The bass sound detecting means detects the bass sound of each measure from the scale sound level on the low frequency side, and the chord name determining means detects the bass sound and the level of each scale sound. The chord name of each measure will be determined from

  In addition, as described above, when a plurality of bass sounds are detected in the measure by the bass sound detecting means, the chord name determining means determines that the measure is divided into several chord detection ranges according to the bass sound detection result. The chord name in each chord detection range is determined from the bass sound and the level of each tone in the chord detection range.

  As described above, in the configuration of the code name detection device of the present invention, processing that requires time resolution of beat detection with only a simple configuration (so-called tempo detection device configuration) and frequency resolution of chord detection are possible. Can be performed simultaneously (a configuration that can further detect chords based on the configuration of the tempo detection device).

  With the above configuration, it is possible to detect beat intervals, beat positions, time signatures, and measures (positions of the first beat), and the power spectrum of each scale sound at predetermined intervals from the acoustic signal input to the input means. Is obtained by the scale sound power detecting means, and the power increment value calculating means sums the increment value of the power of each scale sound for every predetermined time (for each frame) for all the scale sounds, and the whole for every predetermined time. The beat increment is used to calculate the average beat (beats) from the sum of the power increments indicating the degree of change in the entire sound every predetermined time. ) Detect the interval (that is, tempo) and the position of each beat, then calculate the average power of each scale sound for each beat by the above-mentioned measure detection means, Add the average power increment value for all scales to obtain the above value indicating the overall sound change rate for each beat, and use the value indicating the overall sound change rate for each beat to determine the time signature and bar line position. (The position of the first beat) is detected.

  On the premise of that, if the structure analysis of the music is performed by the above-described configuration of the first to third inventions, the chord name can be determined comprehensively from the highly similar portions. As a result, the code detection accuracy can be increased. At the same time, in the configuration of the second invention (and the configuration of the third invention when the third invention has the configuration of the second invention), the technique for improving the accuracy is used once. As soon as this correction is made, the same correction can be made in other places.

  The configurations of the fifth to eighth inventions define a program that can be executed by the computer in order to cause the computer to execute the configurations of the first to fourth inventions. In other words, as a configuration for solving the above-described problems, the above-described means is realized by using the configuration of a computer, and is a program that can be read and executed by the computer. In this case, the computer may include a general-purpose computer configuration including the configuration of the central processing unit, or may include a dedicated machine directed to a specific process, and the configuration of the central processing unit. If it accompanies, there will be no limitation in particular.

  By reading and executing the program for realizing the above means by the computer, the same function realizing means as the function realizing means defined in the first to fourth inventions is achieved. It will be.

Of these, the more specific configuration of the fifth invention is:
By being read and executed by a computer, the computer is
For the input acoustic signal, from the acoustic signal, function as a configuration for detecting beats, detecting bars and detecting / determining chords,
A performance means for performing the acoustic signal;
A first input means for receiving a break of a passage including a plurality of measures according to the performance;
The number of measures for each passage is determined from the positions of the above-mentioned passages, and the similarities are checked by comparing the character strings of the code names or the constituent sounds of the code names detected from those with the same number of measures. A passage similarity detection means for assigning a unique ID to the passages having a similarity equal to or greater than a predetermined threshold,
Display means for displaying to the user the performance status of the acoustic signal, the section separation status, and the similarity detection status including the passage with the unique ID assigned;
A second input means for accepting determination of the similarity detection status by the user;
When it is input that the determination of the similarity detection status is not fixed, the threshold value is changed, and the similarities of the sections are re-detected by the section similarity detecting means, and the determination is confirmed. When inputted, the same passage portion is a code name detection computer program having a function as a passage similarity determining means for redetecting the same measure division and the same code.

A more specific configuration of the sixth invention is:
By being read and executed by a computer, the computer is
For the input acoustic signal, from the acoustic signal, function as a configuration for detecting beats, detecting bars and detecting / determining chords,
A performance means for performing the acoustic signal;
A first input means for receiving a break of a passage including a plurality of measures according to the performance;
The number of measures for each passage is determined from the positions of the above-mentioned passages, and the similarities are checked by comparing the character strings of the code names or the constituent sounds of the code names detected from those with the same number of measures. A passage similarity detection means for assigning a unique ID to the passages having a similarity equal to or greater than a predetermined threshold,
Display means for displaying to the user the performance status of the acoustic signal, the section separation status, and the similarity detection status including the passage with the unique ID assigned;
A second input means for accepting determination of the similarity detection status by the user;
When it is input that the determination of the similarity detection status is not fixed, the threshold value is changed, and the similarities of the sections are re-detected by the section similarity detecting means, and the determination is confirmed. A passage similarity determination unit that, when input, causes the same section portion to be re-detected to have the same measure division and the same code;
Code name having a function as a correction means that can perform bar division and / or code correction when the same section portion is re-detected to have the same bar division and the same code by the above-mentioned section similarity determination means A computer program for detection.

  A more specific configuration of the seventh invention is characterized in that an input of a passage such as an A melody, a B melody, or a chorus is further received at the section of the passage received by the first input means.

A more specific configuration of the eighth invention is:
By being loaded and executed on a computer,
An input means for inputting an acoustic signal;
First scale sound level detection means for performing FFT calculation using a parameter suitable for beat detection at predetermined time intervals from the input acoustic signal, and obtaining the level of each scale sound for each predetermined time;
The increment value of each scale sound level for each predetermined time is summed for all the scale sounds to obtain a total of level increment values indicating the degree of change in the overall sound for each predetermined time. Beat detection means for detecting the average beat interval and the position of each beat from the sum of the incremental values of the level indicating the degree of change in the overall sound for each,
The average value of the scale level for each beat is calculated, and the increment value of the average level of each scale sound for each beat is added for all the scale sounds to indicate the degree of change in the overall sound for each beat. A bar detecting means for obtaining a value and detecting a time signature and a bar line position from a value indicating a change degree of the whole sound for each beat;
From the input acoustic signal, an FFT operation is performed using a parameter suitable for chord detection at a predetermined time interval different from that at the time of the previous beat detection, and the level of each scale sound for each predetermined time is calculated. Second scale level detection means to be obtained;
Bass sound detection means for detecting a bass sound from the level of the low-frequency scale sound in each measure out of the detected scale levels,
Chord name determining means for determining the chord name of each measure from the detected bass sound and the level of each scale sound;
For every detected chord, from the chord position, the base tone scale intensity obtained from the scale sound level of the bass detection range during the base detection period, the base tone, and the scale sound level of the chord detection range during the chord detection period Functions as chord information storage means for storing the required chord scale sound intensity, chord constituent sound, chord constituent sound number, chord name, beat detection, measure detection and chord detection / determination from the input acoustic signal The computer program for detecting a code name applicable to the configuration according to any one of the fifth to seventh inventions further provided in the computer as a configuration for performing the above.

  With the program configuration as described above, by using this program using the existing hardware resources, each device of the present invention as a new application can be easily realized with the existing hardware. Become.

  In the aspect of this program, it becomes possible to easily use, distribute, and sell it using communication or the like. In addition, by using this program using existing hardware resources, the apparatus of the present invention as a new application can be easily executed with the existing hardware.

  It should be noted that some of the functions realizing means according to any one of the fifth to eighth inventions are functions incorporated in a computer (even functions incorporated in a computer in hardware). It may be a function realized by an operating system or other application program incorporated in the computer, and the program includes an instruction for calling or linking a function achieved by the computer. May be.

  This is because a part of each function realization means defined in the first invention to the fourth invention is replaced with a part of a function achieved by, for example, an operating system, and a program or a program for realizing the function This is because modules and the like do not exist directly, but if the functions of the operating system that achieve these functions are called and linked, they have substantially the same configuration.

  According to the configuration of the present invention, after detecting a chord, the same passage portion is detected from the similarity of the chord progression, and when it is detected that they are similar and confirmed, the same passage portion is basically divided into the same measure. Since the same chord progression is used, the chord name detection accuracy can be extremely enhanced.

  Even if there is a misrecognition in the code detection, as in the second invention or the sixth invention, if the correction work by the user is possible, one of the parts having the same ID is corrected. However, since other parts are automatically corrected, it is not necessary for the user to repeat the same correction over and over, and the trouble of correction can be greatly reduced.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 shows the configuration of a personal computer to which a preferred embodiment of the present invention is applied. In the configuration shown in FIG. 1, a program that can be used as the code name detection apparatus of the present invention when the CD-ROM 1016a is loaded into a CD-ROM drive 1016, which will be described later, and is read and executed. Are stored in the CD-ROM 1016a. Accordingly, the CD-ROM 1016a is read and executed by the CD-ROM drive 1016, and the code name detection apparatus of the present invention is realized on the personal computer.

  The outline of the circuit of the personal computer shown in FIG. 1 is as follows: a CPU 1002, a ROM 1004, a RAM 1006, a display 1008, an I / O interface 1010, and a hard disk drive 1020 connected via a system bus 1000 via an image control unit (not shown). Are connected to each other, and control signals and data are input / output to / from each device via the system bus 1000.

  The CPU 1002 is a central processing unit that controls the entire code name detection device based on the program read from the CD-ROM 1016a by the CD-ROM drive 1016 and stored in the hard disk drive 1020 to the RAM 1006. Also, a beat detection scale level detection unit 20, a beat detection unit 25, a bar detection unit 30, a chord detection scale level detection unit 40, a bass sound detection unit 50, a chord name determination unit 60, and a passage similarity detection unit which will be described later. 100 and the passage similarity determination unit 120 are constituted by the CPU 1002 in which the above-described program is operated.

  The ROM 1004 is a storage area in which the BIOS of the personal computer is stored.

  In addition to the storage area for this program, the RAM 1006 is a temporary storage area such as a work area, various coefficients, and parameters suitable for beat and chord detection used during FFT calculation (for example, each buffer and each It is used as a temporary memory).

  The display 1008 is controlled by an image control unit (not shown) that performs necessary image processing in accordance with an instruction from the CPU 1002, and displays the image processing result. A display unit 80 described later corresponds to this.

  The I / O interface 1010 is connected to a keyboard 1012, a sound system 1014, a CD-ROM drive 1016, and a mouse 1018 connected to the system bus 1000 through the I / O interface 1010, and these devices connected to the system bus 1000. The control signal and data are input and output between the two.

  The sound system 1014 constitutes an input unit 10 to be described later. In addition, the sound system 1014 constitutes a performance unit 70 to be described later that outputs an input and stored acoustic signal.

  The CD-ROM drive 1016 reads the program, data, and the like from a CD-ROM 1016a in which a code name detection program is stored. The program, data, and the like are stored in the hard disk drive 1020, and the main program is stored in the RAM 1006 and executed by the CPU 1002.

  As described above, the hard disk drive 1020 stores the program itself and necessary data by reading the code name detection program and executing the program. The data includes, for example, parameters suitable for beat detection, parameters suitable for chord detection, various parameters including thresholds, and the like obtained by obtaining the level of each scale tone for each predetermined time by FFT calculation described later. The hard disk drive 1020 stores the above-described buffers, parameters thereof, threshold values, and the like together with the RAM 1006. Furthermore, it also functions as a chord information storage unit 62 and an acoustic signal storage unit 71 described later. The data stored in the hard disk drive is an acoustic signal (such as performance data) equivalent to that input from the sound system 1014 or the CD-ROM drive 1016, or a first input unit 90 or a second input unit described later. 110 or data from the correction unit 130 or the like.

  The code name detection program according to the present embodiment is read into a personal computer (RAM 1006 and hard disk drive 1020) and executed (by the CPU 1002), whereby the code name detection apparatus as shown in FIG. 2 is configured.

  Further, the input devices such as the keyboard 1012 and the mouse 1018 constitute a first input unit 90, a second input unit 110, or a correction unit 130 which will be described later.

  FIG. 2 is an overall block diagram of the code name detection apparatus according to the present invention. According to the figure, the configuration of the code name detection apparatus is suitable for beat detection at a predetermined time interval (predetermined frame; window) from the input unit 10 for inputting an acoustic signal and the input acoustic signal. An FFT calculation is performed using the parameters, and a tone detection level detecting unit 20 for detecting a scale sound for each predetermined time, and an increment value of the level of each scale sound for each predetermined time are calculated for all scales. Summing up the sounds, obtaining a sum of level increments indicating the degree of change in the overall sound at a given time, and from the sum of level increments showing the degree of change in the overall sound at a given time, The beat detection unit 25 for detecting the average beat interval and the position of each beat, the average value of the level of each scale sound for each beat, and the increment value of the average level of each scale sound for each beat For the scale sounds of Then, a value indicating the degree of change in the overall sound for each beat is obtained, and from the value indicating the degree of change in the overall sound for each beat, the bar detection unit 30 for detecting the time signature and the bar line position, and the input sound A chord detection scale that obtains the level of each tone at a predetermined time by performing an FFT operation using a parameter suitable for chord detection at a predetermined time interval different from the time of the previous beat detection from the signal. A sound level detection unit 40; a bass sound detection unit 50 that detects a bass sound from the level of a low-frequency tone within each measure; and a detected bass sound and each tone A chord name determination unit 60 that determines the chord name of each measure from the level of the chord, and the bass range sound obtained from the chord position and the level of the scale tone of the bass detection range in the base detection period for every detected chord A chord information storage unit 62 for storing chord scale sound intensity, chord constituent sound, chord constituent sound number, and chord name obtained from the sound intensity, bass sound, and the scale sound level in the chord detection range in the chord detection period. An acoustic signal storage unit 71 for storing the acoustic signal, a performance unit 70 for performing the performance of the acoustic signal, and a first input unit 90 for receiving a break of a passage including a plurality of measures by the user according to the performance. Then, the number of measures in each passage is determined from the position of the section of the above-mentioned passage, and the similarity is checked by comparing the character string of the code name detected from the ones with the same number of measures or the constituent sounds of the code name. The passage similarity detection unit 100 for assigning a unique ID to the passages whose similarity representing the similarity is equal to or greater than a predetermined threshold, and the performance status / section division of the acoustic signal A display unit 80 that displays to the user a similarity detection status including a passage with a unique ID and a unique ID, a second input unit 110 that accepts determination of the similarity detection status by the user, and the similarity When it is input that the determination of the sex detection status is not confirmed, the threshold value is changed, and the similarity detection unit 100 is caused to re-detect the similarity of each passage, and the determination is input when the determination is confirmed. In this case, the same section portion is re-detected so as to have the same measure division and the same code, and the same section portion is changed to the same measure division and the same by the above-described passage similarity determination section 120. A correction unit 130 is provided that can divide bars and / or correct codes when re-detecting the code.

  The input unit 10 for inputting a music sound signal is a part for inputting a music sound signal to be subjected to chord name detection, and is configured by the sound system 1014 as described above. An analog signal input from a device such as a microphone may be converted into a digital signal by an A / D converter (not shown). In the case of digitized music data such as a music CD, it is directly taken in as a file ( Ripping), it may be specified and opened. When the input digital signal is stereo, it is converted to monaural in order to simplify subsequent processing.

  This digital signal is input to the beat detection scale level detector 20. The beat detection scale level detection unit 20 is read and executed by a chord name detection program, and is configured by the CPU 1002. As described above, an FFT calculation is performed at predetermined time intervals from an input acoustic signal. And has a function of obtaining the level of each scale sound for each predetermined time. The configuration is further configured from each part of FIG.

  Among them, the waveform preprocessing unit 21 is configured to downsample the acoustic signal from the input unit 10 of the music acoustic signal to a sampling frequency suitable for future processing.

  The downsampling rate is determined by the range of the instrument used for beat detection. In other words, in order to reflect the performance sound of high-frequency rhythm instruments such as cymbals and hi-hats in beat detection, it is necessary to set the sampling frequency after down-sampling to a high frequency, but bass sounds, bass drums, snare drums, etc. When beat detection is mainly performed from instrument sounds and middle instrument sounds, the sampling frequency after downsampling need not be so high.

  For example, when the highest sound to be detected is A6 (C4 is in the middle), the basic frequency of A6 is about 1760 Hz (when A4 = 440 Hz), so the sampling frequency after downsampling is a Nyquist frequency of 1760 Hz or higher. It may be 3520 Hz or higher. From this, the downsampling rate may be about 1/12 when the original sampling frequency is 44.1 kHz (music CD). At this time, the sampling frequency after downsampling is 3675 Hz.

  In the downsampling process, data is skipped after passing through a low-pass filter that cuts off components above the Nyquist frequency (1837.5 Hz in this example), which is usually half the sampling frequency after downsampling (now In this example, 11 out of 12 waveform samples are discarded).

  The purpose of downsampling in this way is to reduce the FFT computation time by lowering the number of FFT points necessary to obtain the same frequency resolution in the subsequent FFT computation.

  If the sound source is already sampled at a fixed sampling frequency, such as a music CD, such down-sampling is necessary. However, the music acoustic signal input unit 10 is input from a device such as a microphone. When the analog signal is converted into a digital signal by the A / D converter, the waveform preprocessing unit 21 is naturally set by setting the sampling frequency of the A / D converter to the sampling frequency after downsampling. Can be omitted.

  When the downsampling by the waveform preprocessing unit 21 is completed in this manner, the output signal of the waveform preprocessing unit 21 is subjected to FFT (Fast Fourier Transform) by the FFT calculation unit 22 at a predetermined time interval.

  The FFT operation unit 22 is constituted by the CPU 1002 in which the program is operated. The FFT parameters (the number of FFT points and the shift amount of the FFT window) are values suitable for beat detection. In other words, if the number of FFT points is increased in order to increase the frequency resolution, the size of the FFT window increases, and one FFT is performed from a longer time, resulting in the FFT characteristic that the time resolution decreases. Must be taken into account. In other words, at the time of beat detection, it is better to increase the time resolution at the expense of frequency resolution. There is a method in which the time resolution is not deteriorated even if the number of FFT points is increased by setting the waveform data to only a part of the window and filling the rest with 0 without using the waveform as long as the window size. However, a certain number of waveform samples is necessary to correctly detect the power on the bass side.

  Considering the above, in this embodiment, the number of FFT points is 512, the window shift is 32 samples, and no zero padding is set. When FFT calculation is performed with such settings, the time resolution is about 8.7 ms and the frequency resolution is about 7.2 Hz. It can be seen that the time resolution of about 8.7 ms is a sufficient value considering that the tune has a tempo of quarter note = 300 and the length of the 32nd note is 25 ms.

  In this way, the FFT operation is performed at predetermined time intervals, and the level of the power spectrum is calculated from the square root of the sum of the squares of the real part and the imaginary part, and the result is sent to the level detector 23. It is done.

  Similarly, the level detection unit 23 includes the CPU 1002 in which the above-described program is operated, and calculates the level of each scale tone from the power spectrum calculated by the FFT calculation unit 22. Since FFT only calculates the power of a frequency that is an integer multiple of the sampling frequency divided by the number of FFT points, in order to detect the level of each scale tone from this power spectrum, the following processing is performed. Do. That is, all the sounds (C1 to A6) for which the scale sound is calculated have the maximum power in the power spectrum corresponding to the frequency in the range of 50 cents above and below the fundamental frequency of each sound (100 cents is a semitone). Let the power of the spectrum be the level of this scale sound.

  When the levels are detected for all the scale sounds, the levels are stored in the buffer 24, and the waveform read position is advanced by a predetermined time interval (32 samples in the previous example), so that the FFT calculation unit 22 and the level detection unit 23 have waveforms. Repeat until the end.

  As described above, the power of each scale sound for each predetermined time of the acoustic signal input to the music acoustic signal input unit 10 is stored in the buffer 24.

  The beat detection unit 25 is configured by a CPU 1002 of a computer that similarly reads and executes a code name detection program and performs the following processing. As described above, the sum of the increments of each scale sound level for each predetermined time is summed for all the scale sounds, and the sum of the level increment values indicating the degree of change of the overall sound for each predetermined time is obtained. It has a function of detecting the average beat interval and the position of each beat from the sum of the incremental values of the level indicating the degree of change of the entire sound every predetermined time.

  Next, the configuration of the beat detection unit 25 in FIG. 1 will be described. The beat detection unit 25 is executed in the process flow as shown in FIG.

  The beat detection unit 25 generates an average beat based on a change in the level of each scale sound for each predetermined time (hereinafter, this one predetermined time is referred to as one frame) output by the beat detection scale level detection unit 20. (Beat) interval (ie tempo) and beat position are detected. For this purpose, first, the beat detection unit 25 sums up the level increment values of each scale sound (the sum of the level increment values from the previous frame for all the scale sounds. When the level decreases from the previous frame Is added as 0) (step S100).

  In the configuration for calculating the level increment value of each scale tone, the beat detection scale level detector 20 detects the predetermined time (as described above) detected as shown in the middle of FIG. The increment value of the power spectrum of each scale sound (referred to as one frame) (the power spectrum shown in the vertical direction of C1 to A6 in the example of FIG. 5) is summed up for all the scale sounds. The sum of the incremental values of the levels shown in the lower part of FIG. 5, which shows the degree of change of the entire sound every predetermined time, is obtained.

  That is, in the configuration for calculating the level increment value of each scale note, the sum of the level increment values of each scale tone (the sum of the level increment values of the previous frame with all the scale sounds. The level from the previous frame is If it has decreased, it is added as 0).

That is, when the level of the i-th scale sound at the frame time t is L _i (t), the level increment value L _addi (t) of the i-th scale sound is as shown in the following equation 1, Using L _addi (t), the sum L (t) of the level increments of each scale tone at the frame time t can be calculated by the following equation (2). Here, T is the total number of scale sounds.

  The total L (t) value represents the degree of change in sound for each frame. This value suddenly increases at the beginning of sounding, and becomes larger as more sounds begin to sound at the same time. Since music often starts to sound at the beat position, there is a high possibility that the place where this value is large is the beat position.

  As an example, FIG. 5 shows a diagram of the sum of the waveform of a part of a certain song, the level of each scale note, and the level increment value of each scale note. The upper row is the waveform, the middle is the tone level of each scale in each frame (lower is lower, the upper is higher. In this figure, the range is C1 to A6), and the lower is each scale. Shows the sum of level increments. Since the level of each scale tone in this figure is output from the beat detection scale level detector 20, the frequency resolution is about 7.2 Hz, and the level of some scales below G # 2 is the level. However, in this case, since the purpose is to detect a beat, there is no problem that the level of a part of the lower tone cannot be measured.

  As seen in the lower part of the figure, the sum of the level increments of each scale sound has a peak periodically. This regular peak position is the beat position.

  As described above, the beat detection unit 25 uses the average beat (beat) interval (that is, tempo) based on the change in the level of each scale sound per predetermined time output by the beat detection scale level detection unit 20. For this purpose, the beat detector 25 first obtains the periodic peak interval, that is, the average beat interval for the purpose of obtaining the beat position. The average beat interval can be calculated from the autocorrelation of the total level increment value of each scale note (FIG. 4; step S102).

  When the total level increment value of each scale tone in a certain frame time t is L (t), this autocorrelation φ (τ) is calculated by the following equation (3).

ここで、Ｎは総フレーム数、τは時間遅れである。

Here, N is the total number of frames, and τ is a time delay.

  A conceptual diagram of autocorrelation calculation is shown in FIG. As shown in this figure, when the time delay τ is an integral multiple of the peak period of L (t), φ (τ) takes a large value. Therefore, if the maximum value of φ (τ) is obtained for a certain range of τ, the tempo of the music can be obtained.

  The range of τ for obtaining the autocorrelation may be changed according to the assumed tempo range of the song. For example, if the range of quarter note = 30 to 300 is calculated with a metronome symbol, the range for calculating the autocorrelation is 0.2 second to 2 seconds. The conversion formula from time (seconds) to frame is as shown in the following equation (4).

  Τ with the maximum autocorrelation φ (τ) in this range may be set as the beat interval, but τ when autocorrelation is maximum in all songs is not necessarily the beat interval, so the autocorrelation is the maximum value. It is preferable to obtain beat interval candidates from τ at the time (FIG. 4; step S104), and let the user determine the beat interval from these multiple candidates (FIG. 4; step S106).

When the beat interval is determined in this way (the determined beat interval is set to τ _max ), the head beat position is first determined.

A method for determining the first beat position will be described with reference to FIG. The upper part of FIG. 7 is a total L (t) of the level increment values of each scale tone at the frame time t, and the lower part M (t) is a function having a value at the determined beat interval τ _max . This is expressed by the following equation (5).

The cross correlation between L (t) and M (t) is calculated while shifting this function M (t) in the range of 0 to τ _max −1.

  The cross-correlation r (s) can be calculated by the following equation 6 from the characteristic of M (t).

  In this case, n may be determined appropriately according to the length of the first silent portion (n = 10 in the example of FIG. 7).

If r (s) is obtained in the range of s from 0 to τ _max −1, and s at which r (s) is maximized is obtained, this s frame is the first beat position.

  When the first beat position is determined, the subsequent beat positions are determined one by one (FIG. 4; step S108).

The method will be described with reference to FIG. Assume that the first beat is found at the position of the triangle in FIG. The second beat position is determined from a position where L (t) and M (t) are most correlated in the vicinity of the temporary beat position at a position separated by a beat interval τ _max from the first beat position. That is, when the leading beat position is b ₀ , the value of s is determined so that r (s) in the following expression is maximized. In this equation, s is a deviation from the temporary beat position, and is an integer in the range of Equation 7 below. F is a fluctuation parameter, and a value of about 0.1 is appropriate. However, a larger value may be used for a song with a large tempo fluctuation. n may be about 5.

  k is a coefficient that changes in accordance with the value of s, and has a normal distribution as shown in FIG. 9, for example.

If the value of s that maximizes r (s) is obtained, the second beat position b ₁ is calculated by the following equation (8).

  Thereafter, the third and subsequent beat positions can be obtained in the same manner.

  For songs with almost no change in tempo, the beat position can be obtained to the end of the song in this way, but the actual performance often fluctuates slightly or becomes partly slower.

  Therefore, the following method was considered so as to cope with these fluctuations in tempo.

That is, the function of M (t) in FIG. 8 is changed as shown in FIG.
1) is a conventional method, and when the intervals of each pulse are τ1, τ2, τ3, and τ4 as shown in the figure,
τ1 = τ2 = τ3 = τ4 = τ _max
It is.
In 2), τ1 to τ4 are uniformly increased or decreased.
τ1 = τ2 = τ3 = τ4 = τ max + s (-τ max · F ≦ s ≦ τ max · F) Thus, it corresponds to the case where sudden tempo changes.
3) rit. (Ritardando, gradually) or accele. (Accelerando, gradually faster), each pulse interval is
τ1 = τ _max
τ2 = τ _max + 1 · s
τ3 = τ _max + 2 · s (−τ _max · F ≦ s ≦ τ _max · F)
τ4 = τ _max + 4 · s
Calculated by
The coefficients 1, 2, and 4 are merely examples, and may be changed depending on the magnitude of tempo change.
4) is a rit. And accel. In this case, the position of the five pulses is changed where the current beat is to be obtained.

  By combining all of these, calculating the correlation between L (t) and M (t), and determining the beat position from the maximum of them, it is possible to determine the beat position even for a song whose tempo fluctuates. In the case of 2) and 3), the value of the coefficient k when calculating the correlation is also changed according to the value of s.

  Furthermore, although the five pulses are all the same in size at present, only the pulse at the position where the beat is calculated (the temporary beat position in FIG. 10) is increased, or the value is decreased as the distance from the position where the beat is determined is increased. Further, the sum of the level increment values of each scale tone at the position where the beat is sought may be emphasized [5) in FIG.

  When the position of each beat is determined as described above, the result is stored in the buffer 26, and the detected result is displayed via the display unit 80. You may be asked to do it.

  An example of a confirmation screen for the beat detection result is shown in FIG. The position of the triangle mark in the figure is the detected beat position.

  When the “play” button is pressed, the performance unit 70 performs D / A conversion on the current music sound signal and plays it from a speaker or the like. Since the current playback position is displayed with a playback position pointer such as a vertical line as shown in the figure, it is possible to confirm an error in the beat detection position while listening to the performance. Furthermore, if a sound such as a metronome is played at the beat position timing simultaneously with the reproduction of the original waveform of the detection, it is possible to check not only with the eyes but also with the sound, and it is possible to judge the false detection more easily. . As a method for reproducing the sound of the metronome, for example, a MIDI device can be considered.

The beat detection position is corrected by pressing the “correct beat position” button. When this button is pressed, a cross cursor appears on the screen. Click the correct beat position where the first beat detection is wrong. All beat positions after a position slightly before the clicked position (for example, half the position of _τmax ) are cleared, and the subsequent beat positions are detected again with the clicked position as the temporary beat position.

  Next, the time signature and measure detection by the measure detection unit 30 will be described.

  The bar detection unit 30 is configured by a CPU 1002 of a computer that similarly reads and executes a code name detection program and performs the following processing. As mentioned above, it calculates the average value of each scale note level for each beat, and sums the increments of the average level of each scale note for each beat for all the scale notes. A value indicating the degree of change in sound is obtained, and the time and bar line position are detected from the value indicating the degree of change in the overall sound for each beat.

  Since the beat position has been determined by the processing by the beat detection unit 25 so far, the measure detection unit 30 first determines the degree of change in sound for each beat. The degree of change in sound for each beat is calculated from the level of each scale sound for each frame output by the beat detection scale level detector 20.

When the number of frames of the j-th beat is b _j and the frames of the beats before and after the j-th beat are b _j−1 and b _{j + 1} , the degree of change in sound for each beat of the j-th beat is from the frame b _j−1. The average power of each scale sound in the frames up to b _j −1 and the average level of each scale sound in the frames from b _j to b _{j + 1} −1 are calculated. The degree of change in sound can be obtained and calculated by summing up all the scales.

That is, when the level of the i-th scale sound at the frame time t is L _i (t), the average level L _avg i (j) of the i-th scale sound level of the j-th beat is expressed by the following equation (9). Therefore, the sound change degree B _addi (j) for each beat of the i-th tone of the j-th beat is expressed by the following equation (10).

  Therefore, the sound change degree B (j) for each beat of the j-th beat is as shown in the following equation (11). Here, T is the total number of scale sounds.

  The bottom row in FIG. 12 shows the degree of change in sound for each beat. Further, the bar detection unit 30 obtains the time signature and the position of the first beat from the degree of change in sound for each beat.

  The time signature is obtained from the autocorrelation of the degree of sound change for each beat. In general, it is considered that the sound often changes in the first beat, so the time signature can be obtained from the autocorrelation of the sound change degree for each beat. For example, the autocorrelation φ (τ) of the sound change degree B (j) for each beat is determined in the range of 2 to 4 from the formula for obtaining the autocorrelation φ (τ) shown in the following equation (12). The delay τ that maximizes the autocorrelation φ (τ) is defined as the number of beats.

  N is the total number of beats, and φ (τ) is calculated in the range of τ = 2 to 4, and τ at which φ (τ) is the maximum is the number of beats.

Next, the first beat is obtained. This is the position where the sound change degree B (j) for each beat is the largest. That is, when phi (tau) is maximum tau and tau _max, the k of X (k) is maximum the following equation number 13 and _{k max, k max} th beat becomes the position of the first first beat Thereafter, the beat position obtained by adding τ _max is the first beat.

ｎ_ｍａｘは、τ_ｍａｘ・ｎ＋ｋ＜Ｎの条件で最大となるｎ

n _max is the _maximum n under the condition of τ _max · n + k <N

  As described above, when the bar detection unit 30 determines the time signature and the position of the first beat (bar line position), the result is stored in the buffer 31 and the detected result is displayed using the display unit 80. It is desirable to display the screen and let the user change it. In particular, music with odd time signatures cannot be handled by this method, so it is necessary to have the user specify the location of odd time signatures.

  With the above configuration, it is possible to detect the average tempo and accurate beat (beat) position of the entire song, as well as the time signature and the first beat position, from the acoustic signal of the performance of the tempo performed by a human. It becomes possible.

  Next, the configuration for detecting the code name will be described below.

  The chord detection scale level detection unit 40 is similarly read and executed by the chord name detection program. The chord detection tone level detection unit 40 is configured by the CPU 1002 and, as described above, from the acoustic signal input from the input unit 10, It has a function of performing an FFT operation using a parameter suitable for chord detection at a different predetermined time interval different from that at the time of detection, and obtaining the level of each scale sound for each predetermined time.

  The acoustic digital signal input from the input unit 10 is input to the beat detection scale sound level detection unit 20 and the chord detection scale sound level detection unit 40. These scale sound level detection units are each configured from the respective units shown in FIG. 3 and have the same configuration, so that the same components can be reused by changing only the parameters.

  The waveform preprocessing unit 21 used as the configuration of the chord detection scale sound level detection unit 40 has the same configuration as described above, and is suitable for future processing of the acoustic signal from the input unit 10 of the music acoustic signal. Downsample to the sampling frequency. (However, the sampling frequency after down-sampling, that is, the down-sampling rate may be changed for beat detection and chord detection, or may be the same in order to save time for down-sampling.)

  The down-sampling rate of the chord detection waveform pre-processing unit varies depending on the chord detection range. The chord detection sound range is a sound range used when the chord name determination unit 60 detects a chord. For example, if the chord detection sound range is C3 to A6 (C4 is the center), the basic frequency of A6 is about 1760 Hz (when A4 = 440 Hz), so the sampling frequency after downsampling is a Nyquist frequency of 1760 Hz or higher. It may be 3520 Hz or higher. From this, the downsampling rate may be about 1/12 when the original sampling frequency is 44.1 kHz (music CD). At this time, the sampling frequency after downsampling is 3675 Hz.

  In the downsampling process, data is skipped after passing through a low-pass filter that cuts off components above the Nyquist frequency (1837.5 Hz in this example), which is usually half the sampling frequency after downsampling (now In this example, 11 out of 12 waveform samples are discarded). This is for the same reason as described above.

  When the downsampling by the waveform preprocessing unit 21 is thus completed, the output signal of the waveform preprocessing unit is subjected to FFT (Fast Fourier Transform) by the FFT calculation unit 22 at predetermined time intervals.

  The FFT parameters (the number of FFT points and the shift amount of the FFT window) are different values at the time of beat detection and code detection. This is because if the number of FFT points is increased to increase the frequency resolution, the size of the FFT window increases, and one FFT is performed from a longer time, resulting in a decrease in time resolution. (In other words, it is better to increase the time resolution at the expense of frequency resolution when detecting beats). A method that does not deteriorate the time resolution even if the number of FFT points is increased by setting waveform data to only a part of the window and filling the rest with 0 without using a waveform with the same length as the window size. However, in the case of the present embodiment, a certain number of waveform samples are necessary in order to correctly detect the power on the bass side.

  In consideration of the above, in this embodiment, the number of FFT points is 512 at the time of beat detection, the window shift is 32 samples, 0 padding is not performed, the number of FFT points is 8192 at the time of code detection, and the window shift is 128 samples. Then, 1024 samples were used for the waveform sample in one FFT. When FFT calculation is performed with such a setting, the time resolution is about 8.7 ms and the frequency resolution is about 7.2 Hz when the beat is detected, and the time resolution is about 35 ms and the frequency resolution is about 0.4 Hz when the code is detected. Since the scale tone for which the level is to be obtained is in the range from C1 to A6, the frequency resolution of about 0.4 Hz at the time of detecting the chord is the difference between the basic frequency of C1 and C # 1 having the smallest frequency difference, about 1 .9 Hz is also supported. Considering that the length of a 32nd note is 25 ms in a song with a tempo of quarter note = 300, it can be seen that the time resolution of about 8.7 ms at the time of beat detection is a sufficient value.

  In this way, the FFT operation is performed at predetermined time intervals, the power is calculated from the square root of the sum of the squares of the real part and the imaginary part, and the result is sent to the level detector 23.

  The level detector 23 calculates the level of each scale tone from the power spectrum calculated by the FFT calculator 22. Since FFT only calculates the power of an integer multiple of the sampling frequency divided by the number of FFT points, in order to detect the level of each scale tone from this power spectrum, the beat detection scale tone The same processing as that of the level detection unit 20 is performed. That is, for all the sounds (C1 to A6) for which the scale sound is calculated, the maximum power in the power spectrum corresponding to frequencies in the range of 50 cents above and below the fundamental frequency of each sound (100 cents is a semitone) is obtained. Let the power of the spectrum it has be the scale level.

  When the levels are detected for all the scale sounds, this is stored in the buffer 41, and the waveform readout position is advanced by a predetermined time interval (32 samples at the time of beat detection and 128 samples at the time of chord detection in the previous example) The processing of the FFT calculation unit 22 and the level detection unit 23 is repeated until the end of the waveform.

  As described above, the level of each scale sound of the sound signal input to the music sound signal input unit 1 for each predetermined time is also stored in the chord detection buffer 41.

  The bass sound detecting unit 50 is similarly read and executed by the chord name detection program, and is constituted by the CPU 1002, and as described above, each scale sound detected by the chord detection scale sound level detecting unit 40. Among these levels, it has a function of detecting a bass sound from the level of the low-frequency tone in each measure detected by the measure detecting unit 30. That is, the bass sound is detected from the scale sound level of each frame output from the chord detection scale sound level detection unit 40.

  FIG. 13 shows the scale level of each frame output by the chord detection scale level detector 40 for the same part of the same song as in FIG. As shown in this figure, since the frequency resolution in the chord detection scale sound level detection unit 40 is about 0.4 Hz, the levels of all the scale sounds C1 to A6 are extracted.

  Since the bass sound may be different between the first half and the second half of the measure, the bass sound detection unit 50 detects the first half and the second half of each measure. When the first half and the second half are the same, this is confirmed as the bass of the measure, and the chord is also detected in the entire measure. When different bass sounds are detected in the first half and the second half, the chord is also detected separately in the first half and the second half. In some cases, the range to be divided may be further reduced to half (up to a quarter of the bar).

  In this embodiment, the bass sound determination unit 50 determines how to divide a bar and detect a chord. However, the method for dividing a bar is not limited to this, for example, by the applicant. As described in the prior application, Japanese Patent Application No. 2006-216361, the bars may be divided by the change of the sound in the chord detection range. Further, this measure division processing may be provided as an independent configuration instead of being performed by the bass sound detection unit 50.

  Further, as will be described later, when the section similarity determination unit 120 instructs the same section part to be re-detected so as to be divided into the same measure, the bass sound detection unit 50 is configured. . This is because the bass sound determination unit 50 originally determines how to divide a bar and detect chords. However, even if the chord detection unit 50 does not have such a function, when the above instruction is issued by the segment similarity determination unit 120, the same segment part is divided into the same measure in response to the instruction. You may provide as a measure division part (not shown) as an independent structure.

  The bass sound is obtained from the average intensity of the scale sound level in the bass detection range during the bass detection period. That is, this is the intensity of the bass sound.

When the level of the i-th note in the scale at frame time t and _L i (t), the average level of the i th scale notes of _{f e} from the frame _{f s L avgi (f s,} f e) is the following formula It can be calculated by Equation 14.

  This average level is calculated in a bass detection range, for example, a range from C2 to B3, and the bass sound detection unit 50 determines the scale sound having the highest average level as the bass sound. An appropriate threshold is set to prevent the bass sound from being erroneously detected in songs or silences that do not include sound in the bass detection range, and if the average level of the detected bass sound is below this threshold, Sound may not be detected. In addition, when the bass sound is important in later chord detection, it is checked whether the detected bass sound keeps a certain level or more continuously during the bass detection period, and only the more reliable ones are checked. You may make it detect as a bass sound. Further, instead of determining the scale tone having the highest average level in the bass detection range as the base tone, the average level of each pitch name is averaged for every 12 pitch names, The pitch name having the highest level may be determined as the bass pitch name, and the scale tone having the highest average level among the scale sounds in the bass detection range having the pitch name may be determined as the bass tone.

  When the bass sound is determined, the result may be stored in the buffer 51, and the bass detection result may be displayed on the display unit 80 so that the user can correct it if it is wrong. Further, since the bass range may be changed depending on the song, the user may be able to change the bass detection range.

  In FIG. 14, the example of a display of the bass detection result by the bass sound detection part 50 is shown.

  The chord name determination unit 60 is similarly read and executed by the chord name detection program, and is configured by the CPU 1002. As described above, the chord name determination unit 60 is based on the level of the bass sound and the scale sound detected by the bass sound detection unit 50. , Has the function of determining the chord name of each measure. In addition, as will be described later, when the section similarity determination unit 120 instructs the same section division to be re-detected so as to have the same measure division and the same code, the configuration of the code name determination unit 60 is: The code is re-detected.

  The chord detection process by the chord name determination unit 60 is similarly determined by calculating the average level of each tone in the chord detection period. In other words, this is the strength of the cord.

  In this embodiment, the code detection period and the base detection period are the same. The average level in the chord detection period, for example, the C3 to A6 scales in the chord detection period is calculated, and several pitch names are detected in order from the scale that has the largest value, and the pitch names of the bass sounds Extract code name candidates from.

  The code information storage unit 62 is configured to store the detected data, and includes the RAM 1006 and the hard disk drive 1020.

  In the chord name detection process, in the chord name detection process, the chord position stores the first frame number (chord 1, chord 2,. The average of the intervals of the 12 sounds in the bass sound range from C to B in the bass sound detection period is stored. Since the frame number can also be converted into a sample, it may be stored as a sample number. The bass sound stores the scale number of the bass sound detected during the bass sound detection period. The chord scale sound intensity stores a section average of the intensity of 12 sounds from C to B in the base detection period, and the chord constituent sound stores the chord constituent sound extracted in the base detection period. In the code name, the determined code name (or a corresponding number) may be stored.

  The code name determination unit 60 refers to the code information storage unit 62 to determine the code name. The chord name determination unit 60 searches for one chord name and its chord constituent sound from the chord name database storing the chord type (m, M7, etc.) and the pitch from the root tone of the chord constituent sound. The average intensity of the chord constituent sounds is calculated from the chord scale intensity in the chord information storage unit 62. The chord name having the highest chord constituent sound average intensity of all chords is determined as the chord name of the section. At this time, the chord root sound (five tone) and the fifth sound may be omitted in the musical instrument playing the chord, so that even if they are not included, they are extracted as chord name candidates. When a bass tone is detected, the pitch name of the bass tone is added to the chord name of this chord name candidate. In other words, if the chord root sound and the bass sound have the same pitch name, they can be left as they are. Alternatively, a plurality of chord constituent sound average intensities may be displayed on the display unit 80 to allow the user to select them.

  In the above method, when there are too many code name candidates to be extracted, limitation by bass sound may be performed. That is, when a bass sound is detected, chord name candidates whose root sound is not the same as the base sound are deleted.

  Furthermore, musical knowledge may be introduced into the calculation of the chord constituent sound average intensity. For example, the level of each musical note is averaged over all frames, and is averaged for every 12 pitch names to calculate the strength of each pitch name, and the key of the song is detected from the distribution of the strength. Then, the key diatonic chord is multiplied by a certain constant so that the average intensity of the chord constituent sound is increased, or the chord that includes the sound deviating from the sound on the key diatonic scale is included in the tone of the off sound. It is conceivable that the chord constituent sound average intensity is reduced according to the number. In addition, by storing a pattern of common chord progressions as a database and comparing it with the ones that are frequently used among chord candidates, a certain constant is applied so that the average intensity of chord constituent sounds increases. You may do it.

  In any case, when the code name is determined by the code name determination unit 60, the result (change information) is re-stored in the code information storage unit 62 via the buffer 61 and redisplayed on the display unit 80. Let

  The result of determining the code name is shown in FIG. This figure shows a part of the code information storage unit 62 in this embodiment.

  On the other hand, the acoustic signal input from the input unit 10 is stored in the acoustic signal storage unit 71 configured by the hard disk drive 1020. The chord name determined above is also output from the chord name information storage unit 62 and stored in the acoustic signal storage unit 71.

  In addition, when the code name is determined by the code name determination unit 60 and re-displayed on the display unit 80 via the buffer 61, the acoustic signal stored in the acoustic signal storage unit 71 according to a user instruction is The performance unit 70 can be made to perform.

  The first input unit 90 includes a keyboard 1012 and a mouse 1018. A chord name determination result is displayed on the display unit 80, and a user who listens to the acoustic signal on the performance unit 70 The first input unit 90 is used to receive a section break including a plurality of measures. As described above, a musical phrase is a single unit in a music format, which is a multiple of four bars, and for example, is often a unit of eight bars.

  While listening to the performance, the user clicks on the bar line at that point with the mouse 1018 when he thinks it is the break position of the passage. When clicked, a mark indicating a segment break is displayed at that location, and characters A, B, C,... Are displayed in order from the beginning of the song. The characters A, B, C,... Can be used as markers (marks) when the code is corrected later. If you click again on a section that has already been specified as a section break, it will function as a section break cancellation.

  On the other hand, it is also possible to adopt a configuration in which an input of a passage such as an A melody, a B melody, or a chorus can be received in addition to the division of the passage received by the first input unit 90. In the configuration of the above embodiment, as will be described later, after detecting a code, the same passage portion is detected by the passage similarity detection unit 100 based on the similarity of the detected code. Is only the section break position by the first input unit 90, and it is not necessary to be aware of whether it is A melody or B melody, as described above. Then, the same number of measures are compared with each other from the specified passage breaks, and if the chord progression is the same, the same passage is detected. However, if the user knows in advance that the passage is A melody, B melody, or chorus, the overall structure of the entire song will be clear from the beginning, and the detection accuracy will be further improved. As described above, it is also possible to adopt a configuration in which an input of a passage such as an A melody, a B melody, and a chorus can be received in addition to the division of the passage received by the first input unit 90.

  As described above, the passage similarity detection unit 100 is read and executed by the code name detection program, and is configured by the CPU 1002. As described above, the number of measures of each passage is calculated from the position of the division of the passage. The similarities are checked by comparing the character strings of the chord names detected from the ones with the same number of measures or the constituent sounds of the chord names, and the degree of similarity representing those similarities is a predetermined threshold or more. Each other has a function of assigning a unique ID.

  When all the passage breaks have been input by the first input unit 90, the similarity of the passages is detected by the passage similarity detection unit 100. FIG. 16 shows a processing flow for detecting the similarity of the passages by the passage similarity detection unit 100.

  As shown in the figure, the number of measures in each passage is counted from the position where the passage is separated (step S200), and it is checked whether there is a passage having the same number of measures (step S202).

  Here, if there is no passage with the same number of bars (step S202; N), the process is terminated.

  On the other hand, if there are passages with the same number of measures (step S202; Y), the similarity of the passages with the same number of measures is checked (step S204). A method for checking the similarity will be described later.

  As a result of the similarity check, a unique (unique) ID is assigned to a passage having a high similarity (step S206). At this time, the passages with the same ID may be displayed in the same color so as to be easily understood by the user.

  Further, it is checked whether there are other passages with the same number of bars (step S208), and if there are (step S208; Y), the process proceeds to step S204.

  On the other hand, if there are no other passages with the same number of measures (step S208; N), the remaining passages already have the markers (markers) A, B, C,. Leave it as it is, and end the process.

  Here, the method for checking the similarity of the above-described passages will be described below. Suppose that the first and second passages have the same number of bars as shown in FIG. If the third bar and the fourth bar are chord C and chord F, respectively, the chord constituent sounds are C = (de, mi, so) and F = ( F, la, do), only the constituent sounds do match and one of the three chords matches, so the matching rate is 1/3 for each of the third and fourth measures. In all other cases, the codes match, so the match rate is 1. Therefore, (1 + 1 + 1/3 + 1/3 + 1 + 1 + 1 + 1) / 8 (number of measures) = 20/24 = 5/6, and the value is 0.8333333. In this way, in addition to the method of comparing chord constituent sounds, it is also possible to compare simple character strings (in the case of character string comparison, code C and code F do not match at all).

  As a result of comparison, if the degree of similarity (which may be expressed in%) is equal to or greater than a certain threshold value, the same ID unique to both passages is assigned. At this time, if an ID has already been assigned to either one (that is, similarity with another passage has already been detected), the ID is assigned to a passage that has not yet been assigned an ID. If both have already been assigned IDs, determine which ID is to be used, and modify the IDs of all the passages having different IDs to the determined IDs. (That is, if two pairs of passages that have been detected as separate are judged to have the same pair of passages, they are all considered to be the same.)

  In addition, as the degree of matching in this passage similarity determination, the chord root (fundamental tone) may be weighted to calculate the degree of matching. In that case, it is calculated as shown in the following equation (15).

(Equation 15)
(Cord match degree when weighting the chord root) = (Number of constituent sounds of the same note name + (1 in the case of the same route; 0 otherwise) / (one having the larger number of chord constituent sounds + 1))

  The display unit 80 is configured by the display 1008 and has a function of displaying to the user the performance detection status of the acoustic signal, the segment separation status, and the similarity detection status including the segment with the unique ID. Yes. FIG. 18 is an explanatory diagram of a screen showing a state where the result of detecting the similarity of the passage of the song “Hometown” is displayed on the display unit 80. The user can check whether the similarity detection of the passage has been correctly performed by looking at the detection screen.

  Similar to the first input unit 90, the second input unit 110 includes a keyboard 1012, a mouse 1018, and the like, and the passage similarity detection result is displayed on the display unit 80, and the result is seen ( The second input unit 110 is used by a user who listens to the sound signal at the performance unit 70 in some cases, and has a function of accepting the determination of the similarity detection status.

  Similarly, the section similarity determination unit 120 is read and executed by the code name detection program, and is configured by the CPU 1002. As described above, the second input unit 110 determines the similarity detection status. When it is input that it is not confirmed, the threshold value is changed (if the threshold value set by default is a low value, the threshold value is usually changed to a higher value, but it is not limited to this). When the passage similarity detection unit 100 performs re-detection of similarity of each passage, and when the second input unit 110 also inputs the determination, the same passage portion is divided into the same measure division and The bass sound detecting unit 50 and the chord name determining unit 60 have a function of redetecting the same chord.

  The passages assigned to the same ID by the passage similarity detection unit 100 are displayed in the same color on the display unit 80 as described above. The user sees this and checks whether it is correctly judged (whether the same color part is the same passage) or not, and if it is wrong, the second input unit 110 checks with the passage similarity detection unit 100. The threshold used for is changed, and the passage similarity detection is performed again. The process is repeated until a correct determination is made. When the determination is correct, the user uses the second input unit 110 and the passage similarity determination unit 120 determines this.

  When the passage similarity is determined, the passage similarity determination section 120 instructs the code name determination section 60 to again measure and code the code so that the same section becomes the same measure and code. At this time, the chord name determination unit 60 uses all of the same passage parts to comprehensively determine bar division and chords. (For example, with respect to measure division, divide only when it is determined that all the same passages are divided, or when it is determined that more than half of the divisions are divided. Judgment is based on the sum of code candidates and likelihoods at code detection locations)

  Further, the correction unit 130 includes a keyboard 1012 and a mouse 1018 as in the case of the first input unit 90 and the second input unit 110, and the result determined by the passage similarity determination unit 120 is the display unit. The modification unit 130 is used by a user who has seen this result (and listened to the acoustic signal at the performance unit 70), and the passage similarity determination unit 120 causes the chord name determination unit 60, when the re-detection is instructed so that the same section portion has the same bar division and the same chord, the bar division and / or the chord correction can be performed.

  The same passage should have been detected as the same measure division and code by the above-mentioned passage similarity determination unit 120. However, when these are incorrect (for example, when the same code C is confirmed between the passages with the same ID is actually F), the correction unit 130 performs correction.

  At this time, as shown in FIG. 19, for example, when the fifth measure of the first passage is corrected from C to A, the fifth measure of the second passage with the same ID is also changed from C to A. Corrections are made. (If an ID is assigned to the measure of the measure, the measure with the same number of measures from the beginning of the measure of other measures with the same ID is corrected in the same manner.)

  According to the configuration of the present embodiment described above, after detecting a chord, the same passage part is detected from the chord progression similarity, so-called so-called music structure analysis is performed. In this case, the same passage portion basically has the same measure division and the same chord progression, and the detection accuracy of the chord name becomes extremely high. Moreover, even if there is a misrecognition in the code detection, the correction work by the user is possible. Therefore, by correcting only one of the parts with the same ID, the other part is automatically corrected. Therefore, the user does not need to repeat the same correction over and over, and the time and effort for correction can be greatly reduced.

  The code name detection device and the computer program of the present invention are not limited to the illustrated examples described above, and it is needless to say that various changes can be made without departing from the gist of the present invention.

  The code name detection device and the computer program of the present invention can perform beat editing by synchronizing the event in the video track with the time of the beat in the music track when creating a music promotion video, or by beat tracking. Audio editing processing that finds the position and cuts and pastes the waveform of the sound signal of music, controls elements such as lighting color, brightness, direction, special effects, etc. in synchronization with human performance, and automatically controls the clapping and cheers of the audience It can be used in various fields such as event control of a live stage to be controlled and computer graphics synchronized with music.

It is a circuit schematic diagram of a personal computer to which a preferred embodiment of the present invention is applied. 1 is an overall block diagram of a code name detection device according to the present invention. It is apparatus explanatory drawing of a scale sound level detection part 20 or 40. 4 is a flowchart showing a flow of processing in a beat detection unit 25. It is a graph which shows the sum total of the waveform of a part of a certain music, the level of each scale sound, and the level increment value of each scale sound. It is a conceptual diagram of autocorrelation calculation. It is explanatory drawing which shows the determination method of a head beat position. It is explanatory drawing which shows the method of determining the position of the beat after it after the first beat position determination. It is a graph which shows the distribution state of the coefficient k changed according to the value of s. It is explanatory drawing which shows the determination method of the position after the 2nd beat. It is a screen display figure which shows the example of the confirmation screen of a beat detection result. It is a screen display figure which shows the example of the confirmation screen of a bar detection result. It is a graph which shows the level of the scale sound of each flame | frame output by the chord detection scale sound level detection part 40 of the same part of a music. 5 is a graph showing a display example of a bass detection result by a bass sound detection unit 50. It is a screen display figure which shows the example of the confirmation screen of a code name detection result. 4 is a flowchart showing a passage similarity detection processing flow by a passage similarity detection unit 100; It is explanatory drawing which shows an example of a passage similarity. FIG. 10 is an explanatory diagram of a screen showing a state in which a section similarity detection result of a song “Hometown” is displayed on the display unit 80. It is explanatory drawing which shows the state in which the correction result is reflected in the same measure of a similar passage when correction of arbitrary measures etc. is made in the passage while similarity was decided.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Input part 20, 40 Scale sound level detection part 21 Waveform pre-processing part 22 FFT operation part 23 Level detection part 24, 26, 31, 41, 51, 61 Buffer 25 Beat detection part 30 Measure detection part 50 Bass sound detection part 60 Code name determination unit 62 Code information storage unit 70 Performance unit 71 Acoustic signal storage unit 80 Display unit 90 First input unit 100 Phrase similarity detection unit 110 Second input unit 120 Phrase similarity determination unit 130 Correction unit 1000 System bus 1002 CPU
1004 ROM
1006 RAM
1008 Display 1010 I / O interface 1012 Keyboard 1014 Sound system 1016 CD-ROM drive 1016a Program CD-ROM
1018 Mouse 1020 Hard disk drive

Claims (8)

In the chord name detection device for detecting beats, detecting bars, and detecting / determining chords from the acoustic signals for the input acoustic signals,
A performance means for performing the acoustic signal;
A first input means for receiving a break of a passage including a plurality of measures according to the performance;
The number of measures for each passage is determined from the positions of the above-mentioned passages, and the similarities are checked by comparing the character strings of the code names or the constituent sounds of the code names detected from those with the same number of measures. A passage similarity detection means for assigning a unique ID to the passages having a similarity equal to or greater than a predetermined threshold,
Display means for displaying to the user the performance status of the acoustic signal, the section separation status, and the similarity detection status including the passage with the unique ID assigned;
A second input means for accepting determination of the similarity detection status by the user;
When it is input that the determination of the similarity detection status is not fixed, the threshold value is changed, and the similarities of the sections are re-detected by the section similarity detecting means, and the determination is confirmed. A chord name detection apparatus comprising: a phrase similarity determination unit that, when input, re-detects the same section portion so as to have the same measure division and the same code.   A means for correcting and / or correcting a chord is provided when the same section portion is re-detected so as to have the same measure division and the same code by the measure similarity determining means. Item 2. The code name detection device according to Item 1.   3. The chord name detection apparatus according to claim 1, wherein an input of a melody such as an A melody, a B melody or a chorus is further received as a division of the melody received by the first input means. In the chord name detection device according to any one of claims 1 to 3, for a configuration that performs beat detection, measure detection, and chord detection / determination from an input acoustic signal.
An input means for inputting an acoustic signal;
First scale sound level detection means for performing FFT calculation using a parameter suitable for beat detection at predetermined time intervals from the input acoustic signal, and obtaining the level of each scale sound for each predetermined time;
The increment value of each scale sound level for each predetermined time is summed for all the scale sounds to obtain a total of level increment values indicating the degree of change in the overall sound for each predetermined time. Beat detection means for detecting the average beat interval and the position of each beat from the sum of the incremental values of the level indicating the degree of change in the overall sound for each,
The average value of the scale level for each beat is calculated, and the increment value of the average level of each scale sound for each beat is added for all the scale sounds to indicate the degree of change in the overall sound for each beat. A bar detecting means for obtaining a value and detecting a time signature and a bar line position from a value indicating a change degree of the whole sound for each beat;
From the input acoustic signal, an FFT operation is performed using a parameter suitable for chord detection at a predetermined time interval different from that at the time of the previous beat detection, and the level of each scale sound for each predetermined time is calculated. Second scale level detection means to be obtained;
Bass sound detection means for detecting a bass sound from the level of the low-frequency scale sound in each measure out of the detected scale levels,
Chord name determining means for determining the chord name of each measure from the detected bass sound and the level of each scale sound;
For every detected chord, from the chord position, the base tone scale intensity obtained from the scale sound level of the bass detection range during the base detection period, the base tone, and the scale sound level of the chord detection range during the chord detection period The chord name detection apparatus according to any one of claims 1 to 3, further comprising: a chord scale sound intensity, chord constituent sound, chord constituent sound number, and chord information storage means for storing chord names. By being read and executed by a computer, the computer is
For the input acoustic signal, from the acoustic signal, function as a configuration for detecting beats, detecting bars and detecting / determining chords,
A performance means for performing the acoustic signal;
A first input means for receiving a break of a passage including a plurality of measures according to the performance;
The number of measures for each passage is determined from the positions of the above-mentioned passages, and the similarities are checked by comparing the character strings of the code names or the constituent sounds of the code names detected from those with the same number of measures. A passage similarity detection means for assigning a unique ID to the passages having a similarity equal to or greater than a predetermined threshold,
Display means for displaying to the user the performance status of the acoustic signal, the section separation status, and the similarity detection status including the passage with the unique ID assigned;
A second input means for accepting determination of the similarity detection status by the user;
When it is input that the determination of the similarity detection status is not fixed, the threshold value is changed, and the similarities of the sections are re-detected by the section similarity detecting means, and the determination is confirmed. A computer program for detecting a chord name, comprising a function as a phrase similarity determining means for re-detecting the same section portion so as to have the same measure division and the same code when inputted. By being read and executed by a computer, the computer is
For the input acoustic signal, from the acoustic signal, function as a configuration for detecting beats, detecting bars and detecting / determining chords,
A performance means for performing the acoustic signal;
A first input means for receiving a break of a passage including a plurality of measures according to the performance;
The number of measures for each passage is determined from the positions of the above-mentioned passages, and the similarities are checked by comparing the character strings of the code names or the constituent sounds of the code names detected from those with the same number of measures. A passage similarity detection means for assigning a unique ID to the passages having a similarity equal to or greater than a predetermined threshold,
Display means for displaying to the user the performance status of the acoustic signal, the section separation status, and the similarity detection status including the passage with the unique ID assigned;
A second input means for accepting determination of the similarity detection status by the user;
When it is input that the determination of the similarity detection status is not fixed, the threshold value is changed, and the similarities of the sections are re-detected by the section similarity detecting means, and the determination is confirmed. A passage similarity determination unit that, when input, causes the same section portion to be re-detected to have the same measure division and the same code;
By means of the above-mentioned section similarity determination means, when the same section portion is re-detected so as to have the same measure division and the same code, a function as a correction means capable of dividing the bars and / or correcting the code is provided. A computer program for detecting code names.   7. The computer program for detecting a code name according to claim 5 or 6, wherein a passage such as an A melody, a B melody or a chorus is further received as a segment of the passage received by the first input means. By being loaded and executed on a computer,
An input means for inputting an acoustic signal;
First scale sound level detection means for performing FFT calculation using a parameter suitable for beat detection at predetermined time intervals from the input acoustic signal, and obtaining the level of each scale sound for each predetermined time;
The increment value of each scale sound level for each predetermined time is summed for all the scale sounds to obtain a total of level increment values indicating the degree of change in the overall sound for each predetermined time. Beat detection means for detecting the average beat interval and the position of each beat from the sum of the incremental values of the level indicating the degree of change in the overall sound for each,
The average value of the scale level for each beat is calculated, and the increment value of the average level of each scale sound for each beat is added for all the scale sounds to indicate the degree of change in the overall sound for each beat. A bar detecting means for obtaining a value and detecting a time signature and a bar line position from a value indicating a change degree of the whole sound for each beat;
From the input acoustic signal, an FFT operation is performed using a parameter suitable for chord detection at a predetermined time interval different from that at the time of the previous beat detection, and the level of each scale sound for each predetermined time is calculated. Second scale level detection means to be obtained;
Bass sound detection means for detecting a bass sound from the level of the low-frequency scale sound in each measure out of the detected scale levels,
Chord name determining means for determining the chord name of each measure from the detected bass sound and the level of each scale sound;
For every detected chord, from the chord position, the base tone scale intensity obtained from the scale sound level of the bass detection range during the base detection period, the base tone, and the scale sound level of the chord detection range during the chord detection period Functions as chord information storage means for storing the required chord scale sound intensity, chord constituent sound, chord constituent sound number, chord name, beat detection, measure detection and chord detection / determination from the input acoustic signal The computer program for code name detection according to any one of claims 5 to 7, wherein the computer is further provided with a configuration for performing the above.

Images