JP2604403B2 - Automatic music transcription method and device - Google Patents

Abstract

PURPOSE:To perform segmentation unaffected by acoustic signal fluctuation or sudden external sound, by performing the segmentation on the basis of the specific value defined by the length of a verse wherein the pitch data of an input acoustic signal continues in a certain intervale width. CONSTITUTION:The digital acoustic signal such as singing passing through an acoustic signal input apparatus 8 and an A/D converter 7 by a CPU 1 corresponding to a keyboard 4 is stored in the auxiliary memory device 6 of a working memory and subjected to process control processing by the CPU 1 corresponding to the program from a main memory device 3 to extract pitch data and a verse wherein the pitch data continues in a specified intervale width is calculated and the length of the sequence is defined to detect a continuous section wherein the length of the sequence is a predetermined value or more. The sampling point having the max. sequence length in this section is set as a representative point and the acoustic signal is divided at a sampling point where the difference between two adjacent representative pitch data is a predetermined value or more and the change of the pitch data is max. and segmentation not affected by the fluctuation of the acoustic signal or sudden external sound is performed.

Description

Description: BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an automatic music transcription method and apparatus for creating musical score data from acoustic signals such as singing voices, humming voices, and instrument sounds, and more particularly, to an audio signal from pitch information. Is related to a segmentation process for dividing into sections that can be regarded as the same pitch.

[Prior Art] In an automatic transcription system for converting an acoustic signal such as a singing voice, a humming voice, or a musical instrument sound into musical score data, a sound length, a pitch, a key, a time signature, and the like, which are basic information as a musical score from an acoustic signal. Detecting the tempo.

By the way, an acoustic signal is only a signal that continuously includes a repetitive waveform of a basic waveform, and the above-described information cannot be obtained immediately.

Therefore, in the conventional automatic transcription method, first, repetition information (hereinafter, referred to as pitch information) of a basic waveform representing a pitch of an acoustic signal and power information are extracted for each analysis cycle, and thereafter, the extracted pitch information is extracted. And the audio signal is divided into sections (segments) that can be regarded as having the same pitch from the power information (this processing is called segmentation).
Determine the pitch along the absolute pitch axis of the sound signal of each segment from the pitch information and / or power information, determine the tone of the sound signal based on the distribution information of the pitch information, and further determine the sound signal based on the segment. Each information was obtained in the order of determining the time signature and tempo.

Therefore, the interval, time signature, tempo, and the like are determined based on the segment (tone length), and thus the segmentation processing is particularly important in creating musical score data.

[Problems to be Solved by the Invention] By the way, an acoustic signal, particularly an acoustic signal uttered by a human, has an unstable pitch and a large fluctuation of the pitch.
Therefore, it has been difficult to satisfactorily perform the segmentation based on the pitch information. Also, even in the case of sound generated from musical instruments, the sound signal input device that converts the sound into an electric signal also captures surrounding noise, which enters pitch information, making segmentation based on pitch information difficult. There was a case.

As described above, the segmentation is an important element in creating score data, and the final segmentation is performed from the segmentation result based on the power information as necessary, but the segmentation based on the pitch information is performed in this manner. If the precision of the musical score data is low, the precision of the finally obtained musical score data also becomes extremely low.

The present invention has been made in consideration of the above points, and can perform a segmentation based on pitch information satisfactorily without being affected by fluctuations of acoustic signals or sudden external sounds. It is an object of the present invention to provide an automatic transcription method and apparatus that can further improve the accuracy of simple musical score data.

[Means for Solving the Problems] In order to solve the problems, according to a first aspect of the present invention, a process of extracting pitch information from an input audio signal, and converting an audio signal from the extracted pitch information to the same pitch. A segmentation process that divides the sound signal into musical score data, wherein the segmentation process counts, for each sampling point, the length of a continuous run of pitch information for a certain pitch width; Detecting a continuous section in which the counted run length is equal to or more than a predetermined value; extracting a sampling point having a maximum run length as a representative point for each detected section; When the pitch information difference has a difference equal to or greater than a predetermined value, a process of classifying the audio signal at the sampling point where the amount of change in pitch information between representative points is the largest It was made to consist of.

In the second aspect of the present invention, the apparatus further includes at least pitch extracting means for extracting pitch information from the input audio signal, and segmentation means for dividing the audio signal into sections that can be regarded as having the same pitch from the extracted pitch information. An automatic music transcription apparatus for converting an acoustic signal into musical score data, wherein the segmentation means counts the length of a continuous run of pitch information at a certain interval for each sampling point; a run length counting unit; A section detection unit that detects a continuous section whose length is equal to or more than a predetermined value; a representative point extraction unit that extracts a sampling point having a maximum run length as a representative point for each detected section; When the difference between the pitch information of the representative points is greater than or equal to a predetermined value, the audio signal is divided at the sampling point where the amount of change in the pitch information between the representative points is the largest. It constituted by an acoustic signal classification unit for.

[Effect] In both the first and second aspects of the present invention, when a section that can be regarded as the same pitch is divided from the pitch information, the pitch information having the pitch information is used so as not to be affected by the fluctuation of the acoustic signal or the sudden external sound. The classification is made using the characteristic value of the length of the run following the width.

That is, in the first aspect of the present invention, the run length is counted for each sampling point, and the sampling point having the maximum run length in the continuous section having the run length equal to or more than the predetermined value is detected. When the change of the pitch information between the detected adjacent sampling points is sufficiently large, the sampling point at which the change is the largest is determined as the segmentation point of the acoustic signal.

In the second aspect of the present invention, the length of a continuous run of pitch information at a certain pitch is counted for each sampling point by a run length counting section, and the run length counted is longer than a predetermined value. The section is detected by the section detection unit, and for each detected section, the sampling point having the maximum run length is extracted as a representative point by the representative point extraction unit, and the difference between the pitch information of two adjacent representative points is determined. When there is a difference equal to or more than a predetermined value, the acoustic signal is divided at the sampling point where the amount of change in pitch information between the representative points is maximum by the acoustic signal dividing unit.

Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings.

Automatic transcription system First, an automatic transcription system to which the present invention is applied will be described.

In FIG. 4, a central processing unit (CPU) 1 controls the whole of the apparatus, and performs a transcription process shown in FIG. 5 stored in a main storage device 3 connected via a bus 2. Execute the program. In addition to the CPU 1 and the main storage device 3, a keyboard 4 as an input device, a display device 5 as an output device, an auxiliary storage device 6 used as a working memory, and an analog / digital converter 7 are connected to the bus 2. I have.

To the analog / digital converter 7, for example, an audio signal input device 7 including a microphone is connected. The acoustic signal input device 8 captures an acoustic signal such as singing or humming uttered by a user or a musical tone generated from a musical instrument and converts the signal into an electric signal, and converts the electric signal into an analog / digital converter. 7 is output.

When a process is instructed by the keyboard input device 4, the CPU 1 starts the transcription process, executes a program stored in the main storage device 3, and converts the sound converted into a digital signal by the analog / digital converter 7. The signals are temporarily stored in the auxiliary storage device 6, and thereafter, these sound signals are converted into musical score data by executing the above-described program and output to the display device 5 as necessary.

Next, the music transcription process performed by the CPU 1 after capturing the audio signal will be described in detail with reference to the flowchart shown in the functional level of FIG.

First, the CPU 1 performs an autocorrelation analysis on the acoustic signal, extracts pitch information of the acoustic signal at each analysis cycle, and
The power information is extracted by multiplying and summing, and then post-processing such as noise removal and smoothing is performed (steps SP1 and SP
2). Thereafter, for the pitch information, the CPU 1 detects a distribution state around the absolute pitch axis, calculates a shift amount of the acoustic signal with respect to the absolute pitch axis, and shifts the obtained pitch information according to the shift amount. Is executed (step SP3). That is, the pitch information is corrected so that the difference between the pitch axis of the singer or the musical instrument that generated the acoustic signal and the absolute pitch axis becomes smaller.

Next, the CPU 1 obtains a continuous period of the pitch information in which the obtained pitch information is considered to indicate the same sound, executes a segmentation for cutting the sound signal into segments for each sound, and executes the obtained power. Perform segmentation based on information changes (step SP
4, SP5). Based on these two pieces of segment information obtained, the CPU 1 calculates a reference length corresponding to a time length of a quarter note, an eighth note, etc., and executes the segmentation again based on this reference length (step SP6). .

The CPU 1 identifies the pitch of the segment as a pitch on the absolute pitch axis that can determine that the pitch information in the segment is closest based on the pitch information of the segment thus segmented, and further identifies the identified continuous The segmentation is executed again based on whether or not the intervals of the segments are the same (steps SP7 and SP8).

Thereafter, the CPU 1 obtains the product sum of the frequency of occurrence of the pitch captured by the pitch information for each segment and a predetermined weighting coefficient determined according to the key, and, based on the maximum information of the product sum, for example, C major And the key of the music of the input audio signal is determined, such as a minor key or a minor key, and for the identified pitch in which the pitch difference between adjacent sounds in the determined key is small, the pitch is reviewed with respect to pitch information to confirm and correct the pitch ( Steps SP9, SP10). Next, the CPU 1 executes a review of the segmentation based on whether or not there is the same continuous segment from the finally determined pitch, and whether or not there is a power change between the continuous segments, Final segmentation is performed (step SP11).

When the pitch and duration (segment) are determined in this manner, the CPU 1 determines that the music starts from the first beat, the last sound of the phrase does not extend to the next bar, that there is a break in each bar, etc. Measures are extracted from the viewpoint, the beat is determined from the measure information and the segmentation information, and the tempo is determined from the determined beat information and the length of the measure (steps SP12 and SP13).

Then, the CPU 1 organizes the information on the determined pitch, pitch, key, beat, and tempo to finally create the musical score data (step SP14).

Segmentation Based on Pitch Information Next, a segmentation process (see step SP4) based on pitch information in such an automatic transcription system will be described in detail with reference to the flowcharts of FIGS.

FIG. 1 is a flowchart showing such processing at a functional level, and FIG. 2 is a flowchart showing FIG. 1 in more detail.

The CPU 1 calculates a run length from the obtained pitch information for all sampling points for each analysis cycle (step SP20). Here, the run length refers to a continuous period RUN in which the value of the pitch information in a narrow predetermined range R1 symmetrical about the pitch information of the point of interest P1 as shown in FIG.
The acoustic signal generated by a singer or the like is intended to generate a sound having a fixed pitch at predetermined intervals, and thus has fluctuations. The change in pitch information is considered to exist in a narrow range, and the run length RUN is a measure for capturing the same interval.

Next, the CPU 1 calculates a continuous section of sampling points whose run lengths are equal to or greater than a predetermined value (step SP21). This removes the influence of the change in pitch information. After that, the CPU 1 extracts the sampling point having the maximum run length as a representative point for each calculated section (step SP22).

Finally, when the difference (pitch difference) between the pitch information of two adjacent representative points has a difference equal to or greater than a predetermined value, the CPU 1 determines the amount of change in the pitch information between the representative points at each sampling point between them. , And the audio signal is segmented at the sampling point where the variation is the largest (step SP23).

Thus, the segmentation can be performed based on the pitch information without being affected by the fluctuation of the acoustic signal and without being affected by sudden external sound.

Next, such processing will be described in more detail with reference to FIG.

First, the CPU 1 calculates all sampling points t for each analysis cycle.
The run length run (t) is calculated for (t = 0 to N) (step SP30).

Next, after clearing the parameter t indicating the sampling of the processing target to 0, it is confirmed that the processing has not been completed for all the sampling points, and the run length (run (t)) of the sampling point t to be processed is determined. It is determined whether it is smaller than the threshold value θr (steps SP31 to SP33). as a result,
If it is determined that the run length run (t) is insufficient, the parameter t is incremented and the process returns to step SP32 (step SP34).

By repeating such processing, a sampling point whose run length (run (t)) is longer than the threshold value θr is eventually processed, and a negative result is obtained in step SP33. At this time, the CPU 1 stores the parameter t as the parameter s, marks it as a start point at which the run length run (t) has become equal to or larger than the threshold θr, and then confirms that the processing has not been completed for all sampling points. The run length (run (t)) of the sampling point t to be checked and processed is the threshold θr
It is determined whether or not it is smaller (steps SP35 to SP37). As a result, if it is determined that the run length run (t) is sufficient, the parameter t is incremented and the above-described step is performed.
The process returns to SP36 (step SP38).

By repeating such a process, a sampling point whose run length run (t) is shorter than the threshold value θr is eventually processed, and a positive result is obtained in step SP37. Thus, a continuous section in which the run length run (t) is longer than the threshold value θr, that is, a section from the mark point s to the immediately preceding processing sampling point t−1 is detected, and the CPU 1 determines the sampling point of this section. The point giving the maximum run length is marked as a representative point (step SP39). In addition,
When this process ends, the process returns to the step SP32 to detect the next continuous section in which the run length (run (t)) is equal to or larger than the threshold value θr.

In this way, when the processing of all the sampling points is completed and the detection of the continuous section in which the run length run (t) is equal to or larger than the threshold value θr and the marking on the representative point are completed, the CPU 1 resets the parameter t again. After clearing to 0, it is confirmed that the processing has not been completed for all sampling points, and it is determined whether or not a sampling point t to be processed is marked as a representative point (steps SP40 to SP42). If not, the process returns to step SP41 with the parameter t incremented (step SP43).

By repeating such a process, the marked sampling point becomes a processing target, the first representative point is found out, and the CPU 1 stores and marks this value t as the parameter s, and further marks the parameter t. Is incremented, and it is confirmed that the processing has not been completed for all sampling points, and it is determined whether or not the sampling point t to be processed is marked as a representative point (steps SP44 to SP47). If not, the parameter t
Is incremented, and the process returns to the above-mentioned step SP44 (step SP48).

By repeating such processing, the marked sampling points are eventually processed and the next representative point t
Is found. At this time, the CPU 1 determines whether or not the difference between the pitch information of these adjacent representative points s and t is smaller than the threshold value θp.
And proceed to the process of finding the next adjacent representative point.
On the other hand, if the difference is equal to or larger than the threshold value θp, the amount of change in pitch information between the representative points is determined for each of the sampling points st to t located therebetween, and the sampling point with the largest amount of change is marked with a segment (step SP49). ~ 51).

By repeating this processing, segments are sequentially marked between the representative points, and a positive result is obtained in step SP46, and the processing is ended.

Therefore, according to the above-described embodiment, since the segmentation is performed using the length of the run indicating the length in which the pitch information exists in a narrow range, even if the sound signal fluctuates, it is also sudden. Even if an external sound is included, the segmentation can be performed well.

Other Embodiments In the above-described embodiment, the case where the pitch information obtained by the autocorrelation analysis is subjected to the segmentation processing has been described. However, the method of extracting the pitch information is not limited to this.

In the above-described embodiment, the CPU 1 executes all the processing shown in FIG. 5 according to the program stored in the main storage device 3. However, a part or all of the processing is performed by a hardware configuration. May be executed. For example, as shown in FIG. 6 in which the same reference numerals are given to the corresponding parts in FIG. 4, the audio signal from the audio signal input device 8 is amplified through the amplifier circuit 10, and then the pre-filter is added.
The digital signal is supplied to an analog / digital converter 12 via an analog-to-digital converter 11 and converted into a digital signal. The acoustic signal converted to the digital signal is subjected to auto-correlation analysis by a signal processor 13 to extract pitch information. And extract power information
It may be provided to the software processing system by the CPU1. As described above, as the signal processor 13 used in the hardware configuration (10 to 13), a processor (for example, NEC Corporation) capable of real-time processing of a signal in a voice band and having an interface signal with the host CPU 1 is prepared. Co., Ltd. μPD7720) can be applied.

[Effects of the Invention] As described above, according to the present invention, the length of a run in which pitch information continues with a certain width is counted for each analysis cycle for each sampling point, and the length of the run is continuous for a predetermined value or more. Detect a section, for each detected section, extract the sampling point having the maximum run length as a representative point, when the difference between the pitch information of two adjacent representative points has a difference of a predetermined value or more, The amount of change in pitch information between representative points is obtained for each sampling point in between, and the audio signal is segmented at the sampling point where the amount of change is the largest, so even if there is fluctuation in the audio signal, Even if an external sound is included, the segmentation can be satisfactorily performed, and accurate final score data can be obtained.

[Brief description of the drawings]

1 and 2 are flowcharts showing a segmentation process based on pitch information according to one embodiment of the present invention, FIG. 3 is a schematic diagram for explaining the length of a run, and FIG. 4 is an application of the present invention FIG. 5 is a block diagram showing the structure of an automatic music transcription system, FIG.
FIG. 6 is a block diagram showing another configuration of the automatic transcription system. 1 ... CPU, 3 ... main storage device, 6 ... auxiliary storage device,
7 ... A / D converter, 8 ... Acoustic signal input device.

 ──────────────────────────────────────────────────続 き Continued on the front page (72) Inventor Masaki Fujimoto 5-7-15 Shiba, Minato-ku, Tokyo Inside NEC Technical Information System Development Co., Ltd. (72) Inventor Masanori Mizuno 5-7-1 Shiba, Minato-ku, Tokyo No. 15 Examiner, NEC Technical Information Systems Development Co., Ltd. Shigeo Arai

Claims (2)

(57) [Claims] 1. A method for extracting at least one pitch signal from an input audio signal and a segmentation process for classifying the audio signal into sections that can be regarded as having the same pitch based on the extracted pitch information. In the automatic music transcription method for converting data into data, the segmentation process counts the length of a run that continues at a certain pitch interval for each of the pitch information at each sampling point, and calculates a continuous section in which the counted run length is equal to or more than a predetermined value. , And for each detected section, the sampling point having the maximum run length is extracted as a representative point. When the difference between the pitch information of two adjacent representative points has a difference equal to or more than a predetermined value, the representative An automatic music transcription method comprising a process of classifying the acoustic signal at a sampling point where the amount of change in pitch information between points is maximum. 2. The apparatus according to claim 1, further comprising: pitch extracting means for extracting pitch information from the input sound signal; and segmentation means for dividing the sound signal into sections that can be regarded as having the same pitch based on the extracted pitch information. In the automatic music notation apparatus for converting music data into musical score data, the segmentation means includes a run length counting unit that counts, for each sampling point, a run length following the pitch information at a certain pitch width; A section detection unit that detects a continuous section whose length is equal to or greater than a predetermined value; a representative point extraction unit that extracts a sampling point having a maximum run length as a representative point for each detected section; When the difference between the pitch information of the representative points is greater than or equal to a predetermined value, the audio signal is divided at the sampling point where the amount of change in the pitch information between the representative points is the largest. Automatic transcription apparatus characterized by comprising an acoustic signal classification unit for.