JP2604407B2 - Automatic music transcription method and device - Google Patents

Abstract

PURPOSE:To obtain a good intervale as an identified one, by extracting a section identified with an erroneous intervale and correcting the identified intervale of said section so as to become the identified intervale of one of the front and rear sections reduced in intervale difference. CONSTITUTION:When the length of a segment to be processed is shorter than a threshold value, a CPU 1 judges whether the change tendency of the pitch data of said segment overshoots or undershoots. As the result of this judgement, when there is the possibility of an overshoot- or undershoot-segment, the difference between the front and rear segments is calculated and the segment smaller in difference is selected to judge whether the intervale difference with said segment is below the threshold value. When said segment is below the threshold value, it is judged whether the power change between segments is below the threshold value. When said change is below the threshold value as the result of the judgement, the CPU 1 corrects the intervale of said segment to that of a selected segment. By this method, a good intervale as an identified one can be obtained.

Description

Description: BACKGROUND OF THE INVENTION The present invention relates to an automatic music transcription method and apparatus for creating musical score data from acoustic signals such as singing voices, humming voices, and musical instrument sounds, and more particularly to an audio transcription method for a predetermined section. The present invention relates to a process for correcting an identified pitch, which corrects a pitch identified as a pitch as necessary.

[Prior Art] In an automatic transcription system for converting an acoustic signal such as a singing voice, a humming voice, or a musical instrument sound into musical score data, a sound length, a pitch, a key, a time signature, and the like, which are basic information as a musical score from an acoustic signal. Detecting the tempo.

By the way, an acoustic signal is only a signal that continuously includes a repetitive waveform of a basic waveform, and the above-described information cannot be obtained immediately.

Therefore, in the conventional automatic transcription method, first, repetition information (hereinafter, referred to as pitch information) of a basic waveform representing a pitch of an acoustic signal and power information are extracted for each analysis cycle, and thereafter, the extracted pitch information is extracted. And / or dividing the sound signal into segments (segments) that can be regarded as the same pitch based on the power information (this processing is called segmentation), and then identifying the pitch along the absolute pitch axis of the sound signal of each segment from the pitch information of the segment. Then, the tone of the sound signal is determined based on the distribution state of the pitch information, and further, the information is obtained in the order of determining the beat and the tempo of the sound signal based on the segment.

[Problems to be Solved by the Invention] By the way, even if an attempt is made to identify a certain segment of an acoustic signal as a pitch on an absolute pitch axis, the pitch of an acoustic signal, particularly an acoustic signal uttered by a human, is not stable.
Even if the sound is intended, the pitch fluctuates a lot.
For this reason, the pitch identification processing is very difficult.

In particular, when moving from one sound to the next, the pitch often fluctuates before and after the transition to the next sound cannot be made smoothly. For this reason, in the segmentation processing, there are many cases where the section is erroneously classified as a section of a different sound, and as a result, the section is sometimes identified as a different pitch also in the pitch identification processing.

The present invention has been made in consideration of the above points, and because of the fluctuation of the pitch generated at the transition of the next sound in the acoustic signal, the pitch of the segment identified as a pitch different from the pitch intended by the singer etc. The present invention aims to provide an automatic transcription method and apparatus which can correct the pitch information of the preceding and succeeding segments to obtain more accurate pitch information and further improve the accuracy of final score data. .

[Means for Solving the Problems] In order to solve the problems, in the first method of the present invention, a pitch information representing a repetition period of an input acoustic signal waveform, representing a pitch, and a power of the acoustic signal. A process of extracting information, a segmentation process of dividing an audio signal into sections that can be regarded as having the same pitch based on pitch information and / or power information, and an absolute pitch axis as a pitch of the section based on the pitch information for the divided section. At least a pitch identification process to identify the upper pitch, in an automatic music transcription method of converting the acoustic signal into musical score data,
A process of extracting a section shorter than a predetermined length from each of the divided sections, a process of further extracting a section in which a change in pitch information exhibits a peculiar predetermined tendency from the extracted section, and extracting the section based on the change tendency. Detecting the difference between the identified intervals of the section and the preceding and following sections and correcting the identified interval of the section to the identified interval of the section where the interval difference is small when at least one of the intervals is smaller than a predetermined value. This is provided as a post-processing of the identification processing.

Further, in the second method of the present invention, the process of correcting the identification interval of the section extracted based on the change tendency is performed when the interval difference between the preceding and following sections is small and the interval between the intervals is small. When the change in the power information is small, the identification pitch is corrected to the identification pitch in one of the preceding and following sections.

In the apparatus according to the third aspect of the present invention, pitch / power extracting means for extracting pitch information representing a pitch and power information of an audio signal, which is a repetition period of the input audio signal waveform, pitch information and / or power information Segmentation means for classifying an acoustic signal into sections that can be regarded as having the same pitch based on pitch information, and pitch identification means for identifying the divided sections as pitches on the absolute pitch axis as pitches of the section based on pitch information. An automatic transcription apparatus for converting an acoustic signal into musical score data in preparation for a short section extracting means for extracting a section shorter than a predetermined length from each section, and a predetermined section in which a change in pitch information is unique from the extracted section. Candidate section extracting means for further extracting a section exhibiting the tendency of the above, and an identification pitch between the section extracted based on the change tendency and the sections before and after the section The identification interval correcting means at least one of the pitch difference is detected is to modify the identification interval of the section identified musical interval of pitch difference is small section is smaller than a predetermined value the difference was provided after the interval identification means.

According to the fourth aspect of the present invention, when the interval difference between the preceding and following sections is small and the change in power information is small between the section and the section having the small pitch difference, the identification pitch correcting means identifies the preceding or succeeding section. Modified the identification pitch to the pitch.

[Operation] Although the first to fourth aspects of the present invention are intended for the same sound, they are divided into separate sections due to pitch fluctuation at the time of sound transition, and sections identified as different pitches. Is to correct the identification pitch of.

In the method according to the first aspect of the present invention, a section identified as an incorrect pitch based on a section length, a change tendency of pitch information of the section, and a pitch difference between the preceding and following sections is extracted, and one of the preceding and following ones having a small pitch difference is extracted. The identification pitch of the section is modified to the identification pitch of the section.

In the method according to the second aspect of the present invention, a section identified as an erroneous interval is extracted in consideration of the amount of change in power information between the preceding and following sections, and the identified interval of one of the preceding and following sections having a small pitch difference is extracted. Then, the identification pitch of the section was modified.

In the apparatus according to the third aspect of the present invention, a section having a short section length is extracted by a short section extracting means, and a section exhibiting a unique pitch information change tendency is extracted by a correction candidate section extracting means. With regard to the above, the identification pitch means confirms the pitch difference between the preceding and following sections, and when the pitch difference is small, the identification pitch of the preceding or following section having a small pitch difference is corrected to the identification pitch of that section.

In the device according to the fourth aspect of the present invention, the identified pitch means confirms a pitch difference and a power information change amount between the preceding and following sections of the section extracted by the correction candidate section extracting means, and determines a difference between the pitch difference and the power information. When both are small, the identification interval in one section before and after the interval difference is small is corrected.

Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings.

Automatic transcription system First, an automatic transcription system to which the present invention is applied will be described.

In FIG. 3, a central processing unit (CPU) 1 controls the whole of the apparatus, and performs a musical notation processing shown in FIG. 4 stored in a main storage device 3 connected via a bus 2. Execute the program. In addition to the CPU 1 and the main storage device 3, a keyboard 4 as an input device, a display device 5 as an output device, an auxiliary storage device 6 used as a working memory, and an analog / digital converter 7 are connected to the bus 2. I have.

An audio signal input device 8 including, for example, a microphone is connected to the analog / digital converter 7. The acoustic signal input device 8 captures an acoustic signal such as singing or humming uttered by a user or a musical tone generated from a musical instrument and converts the signal into an electric signal, and converts the electric signal into an analog / digital converter. 7 is output.

When a process is instructed by the keyboard input device 4, the CPU 1 starts the transcription process, executes a program stored in the main storage device 3, and converts the sound converted into a digital signal by the analog / digital converter 7. The signals are temporarily stored in the auxiliary storage device 6, and thereafter, these sound signals are converted into musical score data by executing the above-described program and output to the display device 5 as necessary.

Next, the transcription process performed by the CPU 1 after capturing the audio signal will be described in detail with reference to the flowchart shown in FIG.

First, the CPU 1 performs an autocorrelation analysis of the acoustic signal to extract pitch information of the acoustic signal at each analysis cycle, and also performs a sum-of-squares process to extract power information at each analysis cycle, and then performs noise removal and smoothing processing. And other post-processing (steps SP1, SP
2). Thereafter, for the pitch information, the CPU 1 calculates a shift amount of the pitch axis of the acoustic signal with respect to the absolute pitch axis based on the distribution state, and performs a tuning process of shifting the obtained pitch information according to the shift amount. Execute (step SP3). That is, the pitch information is corrected so that the difference between the pitch axis of the singer or the musical instrument that generated the acoustic signal and the absolute pitch axis becomes smaller.

Next, the CPU 1 obtains a continuous period of pitch information in which the obtained pitch information is considered to indicate the same pitch,
A segmentation is performed to divide the acoustic signal into segments for each sound, and a segmentation is performed based on the obtained change in the power information (step SP
4, SP5). Based on these two pieces of segment information obtained, the CPU 1 calculates a reference length corresponding to a time length of a quarter note, an eighth note, etc., and executes the segmentation again based on this reference length (step SP6). .

The CPU 1 identifies the pitch of the segment as a pitch on the absolute pitch axis that can determine that the pitch information is the closest based on the pitch information of the segment thus segmented, and further identifies the pitch of the identified continuous segment. Segmentation is again performed based on whether or not are the same (steps SP7 and SP8).

Thereafter, the CPU 1 obtains a sum of products of a frequency of appearance of a pitch obtained by summing pitch information about each segment around the pitch axis and a predetermined weighting coefficient determined according to a key, and obtains the maximum information of the sum of products. Based on this, for example, the key of the musical composition of the input audio signal is determined, such as C major or A minor, and for a predetermined pitch on a scale in the determined key, the pitch is reviewed for pitch information to confirm and correct the pitch. (Step
SP9, SP10). Next, the CPU 1 executes a review of the segmentation based on whether or not there is the same continuous segment from the finally determined pitch, and whether or not there is a power change between the continuous segments, Perform final segmentation (step SP
11).

Once the pitch and segment are determined in this way,
The CPU 1 extracts measures from the viewpoint that the music starts from the first beat, the last sound of the phrase does not extend to the next measure, and there is a break in each measure, and determines the time signature from the measure information and the segmentation information. The tempo is determined from the determined time signature information and the length of the bar (step SP1).
2, SP13).

Then, the CPU 1 organizes the information on the determined pitch, pitch, key, beat, and tempo to finally create the musical score data (step SP14).

Pitch Identifying Process Next, the above-described pitch identifying process of step SP7 in the automatic transcription method, particularly the review process of the once identified pitch, will be described in detail with reference to the flowchart of FIG.

First, for the segment obtained by the segmentation, the CPU 1 obtains, for example, the average value of the pitch information in the segment, and determines the interval of the segment to any interval that differs by a semitone on the absolute pitch axis closest to the average value. (Step SP20).

The pitch thus identified is reviewed as follows. Here, since the pitch is unstable at the time of transition of the sound, the review is performed on a segment which is classified as another segment and which is considered to have been identified independently of the preceding and following segments.

The CPU 1 first confirms that the processing has not been completed for the last segment and determines whether or not the length of the segment to be processed is shorter than a threshold. The process proceeds to the segment and returns to step SP20 (steps SP21 and SP22).

The reason for this is that if the sound is the same at the start or end of the sound at the time of the transition of the sound and is divided into other segments, the segment length is shortened. In this way, when it is detected that the segment has a short length, the CPU 1 matches the change tendency of the pitch information of the segment with the change tendency of the overshoot, and changes the pitch information of the segment. By matching the tendency with the change tendency of the undershoot, it is determined whether the change tendency of the pitch information of the segment is overshoot or undershoot (step
SP23, 24).

Here, when transitioning from a certain sound to the next sound, there may be a gradual transition from a slightly higher pitch to the pitch of the sound near the start of the next sound, and from a slightly lower pitch near the start of the next sound. There may be a gradual transition to the pitch of the sound, and in the vicinity of the end of a certain sound, it may gradually decrease from the pitch of that sound and transition to the next sound,
In the vicinity of the end of a sound, the sound may gradually rise from the pitch of the sound and shift to the next sound. Despite being such an identical sound, a portion higher than a predetermined pitch is an overshoot in a portion where the pitch changes while showing a gradually increasing or decreasing tendency due to the transition of the sound,
In addition, a portion lower than a predetermined pitch among portions where the pitch changes while showing a gradually increasing or decreasing tendency due to a transition of a sound is called an undershoot.

Such overshoot and undershoot portions may be classified as independent segments, and as described above, matching between the change tendency of the pitch information of the segment and a predetermined increasing or decreasing tendency is performed, and It is determined whether or not there is a possibility that any of the segments may exhibit an overshoot or undershoot.

If a negative result is obtained as a result of this determination, the processing target segment is set as the next segment, and the process returns to step SP21. On the other hand, when determining that there is a possibility of an overshoot or undershoot segment, the CPU 1 obtains a difference between the identified pitch of the preceding and following segments and the identified pitch of the segment, and marks the segment having the smaller difference. After that, it is determined whether or not the pitch difference from the marked segment is smaller than a threshold (steps SP25 and SP26).

If the segment is segmented into separate segments in spite of being the same sound, the pitch of that segment is not so different from the pitch of the preceding and following segments, and if the pitch difference between the preceding and following segments is large, In this case, it is considered that the segment is not an overshoot or undershoot segment. In this case, the CPU 1 shifts the segment to be processed to the next segment and proceeds to step SP21 described above.
Return to

On the other hand, if the pitch difference between the marked segment and the marked segment is small, the CPU 1 determines whether or not the change in the power information near the boundary between the segment and the marked segment is equal to or larger than a threshold (step SP27). When transitioning to a different sound, the power information often changes, and if the change in the power information is large, the segment is not considered to be an overshoot or undershoot segment. Move the segment to be processed to the next segment and go to step SP21
Return to

If a positive result is also obtained by the determination in step SP27, the segment is considered to be an overshoot or undershoot segment, so the CPU 1 corrects the pitch of the segment to the pitch of the marked segment. Is moved to the next segment and the process returns to step SP21 (step SP28).

Such a process is repeated to review the pitch for all the segments, and when the review process for the last segment is completed, a positive result is obtained in step SP21 and the processing program ends.

FIG. 2 shows an example in which the identified pitch has been corrected by the process, and the curve represents the pitch information PIT. In this example, the second segment S2 and the third segment are intended to be the same pitch. I have. Prior to the modification, the second segment S2 was identified as the pitch R2 which is a semitone lower than the identification pitch R3 of the segment S3, but the pitch R3C of the segment S3 has been corrected to the pitch R3 of the segment S3 by such processing.

Therefore, in the above embodiment, based on the segment length, the change tendency of the pitch information, the pitch difference between the preceding and following segments, and the power information difference between the preceding and following segments,
Detecting segments identified by mistake at different intervals and correcting the intervals, the identified intervals become more accurate, and the subsequent processing can be executed more accurately, increasing the accuracy of the score data. Can be.

Other Embodiments In the above-described embodiment, a segment identified at an incorrect pitch is extracted in consideration of a difference in power information between the preceding and following sections, but at least a change in segment length and pitch information is performed. What is necessary is just to extract based on a tendency and the pitch difference with the segment before and behind.

In addition, the method of detecting overshoot or undershoot based on the change tendency of pitch information is not limited to the method of detecting a simple increase tendency and a decrease tendency as described above,
It goes without saying that other methods such as comparison with a reference pattern can be applied.

Further, in the above-described embodiment, the CPU 1 executes all the processing shown in FIG. 4 according to the program stored in the main storage device 3. However, a part or all of the processing is performed by a hardware configuration. It is good to be executed in. For example, as shown in FIG. 5 in which the same reference numerals are given to the corresponding parts in FIG. 3, after the sound signal from the sound signal input device 8 is amplified through the amplifier circuit 10, the pre-filter is further added.
The digital signal is supplied to an analog / digital converter 12 via an analog-to-digital converter 11 and converted into a digital signal. The acoustic signal converted to the digital signal is subjected to autocorrelation analysis by a signal processor 13 to extract pitch information, and a square sum processing is performed. And extract power information
It may be provided to the software processing system by the CPU1. As the signal processor 13 used in such a hardware configuration (10 to 13), a processor (for example, a processor that can process a signal in a pitch band in real time and is provided with an interface signal with the CPU 1 of the host)
NEC Corporation μPD7720) can be applied.

[Configuration of the Invention] As described above, according to the present invention, a segment identified as an incorrect pitch is extracted based on at least the segment length, the change tendency of the pitch information, and the pitch difference between the preceding and following segments, and the Since the interval of the segment is corrected to the identified interval of one of the preceding and succeeding segments, a good identified interval can be obtained, and a highly accurate final score data can be obtained.

[Brief description of the drawings]

FIG. 1 is a flowchart showing a process for correcting an identified pitch according to an embodiment of the present invention, FIG. 2 is a schematic diagram showing an example of such a process for correcting the identified pitch, and FIG. 3 is an automatic transcription system to which the present invention is applied. FIG. 4 is a foot chart showing the automatic transcription process, and FIG. 5 is a block diagram showing another configuration of the automatic transcription system. 1 ... CPU, 3 ... main storage device, 6 ... auxiliary storage device,
7 ... A / D converter, 8 ... Acoustic signal input device.

 ──────────────────────────────────────────────────続 き Continued on the front page (72) Inventor Masaki Fujimoto 5-7-15 Shiba, Minato-ku, Tokyo Inside NEC Technical Information System Development Co., Ltd. (72) Inventor Masanori Mizuno 5-7-1 Shiba, Minato-ku, Tokyo No. 15 Examiner, NEC Technical Information Systems Development Co., Ltd. Shigeo Arai

Claims (4)

(57) [Claims] 1. A process for extracting pitch information representing a pitch and power information of the sound signal, which is a repetition period of an input sound signal waveform, and processing the sound signal based on the pitch information and / or the power information. At least includes a segmentation process of classifying the segment into sections that can be regarded as the same pitch, and a pitch identification process of identifying the segmented section as a pitch on the absolute pitch axis based on the pitch information as a pitch of the section. In the automatic music transcription method of converting the pitch into musical score data, a process of extracting a section shorter than a predetermined length from each of the divided sections, and a section in which the change of the pitch information exhibits a peculiar predetermined tendency from the extracted section. Further extracting, and detecting a difference in the identified pitch between the section extracted based on the change tendency and the section before and after the section, and performing at least one A process of correcting the identified interval in the section where the interval difference is small when the interval difference is smaller than a predetermined value, as a post-process of the interval identifying process. 2. A process for correcting an identification interval of said section extracted based on a change tendency, wherein a difference in pitch between the preceding and following sections is small, and a change in the power information between the section and the section having a small interval difference. 2. The automatic music transcription method according to claim 1, wherein the identification pitch is corrected to one of the preceding and succeeding identification pitches when is smaller. 3. A pitch / power extracting means for extracting a pitch information representing a pitch and a power information of the sound signal, which is a repetition period of an input sound signal waveform, and based on the pitch information and / or the power information. Segmentation means for dividing the acoustic signal into sections that can be regarded as having the same pitch, and pitch identification means for identifying the divided sections as pitches on the absolute pitch axis based on the pitch information. An automatic transcription apparatus for converting the acoustic signal into musical score data in preparation for the following: a short section extracting means for extracting a section shorter than a predetermined length from each of the divided sections; and a change in the pitch information from the extracted section. A modification candidate section extracting means for further extracting a section exhibiting a peculiar predetermined tendency; a section extracted based on the change tendency and a section before and after the section; An identification interval correction means for detecting the difference between the identification intervals and the interval, and when at least one of the interval differences is smaller than a predetermined value, correcting the identification interval of the interval to the identification interval of the interval having a small interval. An automatic music transcription device, which is provided at a subsequent stage. 4. An identification pitch correcting means which, when a pitch difference between the preceding and succeeding sections is small and a change in the power information is small between the sections having a small pitch difference, the identification pitch in one of the preceding and following sections. 4. The automatic transcription apparatus according to claim 3, wherein the identification pitch is corrected.