JP2713952B2 - Automatic music transcription method and device - Google Patents

Abstract

PURPOSE:To well perform score processing after accurate segmentation is performed, by performing segmentation for dividing the power data extracted from an acoustic signal into an effective section equal to or more than a threshold value and an ineffective section equal to or less than said threshold value. CONSTITUTION:The digital acoustic signal such as singing controlled by a CPU 1 corresponding to the indication from a keyboard 4 and passing through an acoustic signal input apparatus 8 and an A/D converter 7 is stored in the auxiliary memory device 6 of a working memory. The stored content thereof is subjected to process control processing by the CPU 1 corresponding to the program of a main memory device 3 to extract power data to perform segmentation for dividing the acoustic signal into effective and ineffective sections according to whether the extracted power data is a threshold value or more and the effective section of a predetermined length of less is also divided as the ineffective section. By this accurate segmentation unaffected by noise or intervale fluctuation, the score taking processing on and after corresponding to the segmentation is effectively performed and a highly accurate score is formed.

Description

Description: BACKGROUND OF THE INVENTION The present invention relates to an automatic music transcription method and apparatus for creating musical score data from acoustic signals such as singing voices, humming voices, and instrument sounds. The present invention relates to an automatic transcription method and apparatus in which sections that can be regarded as having the same pitch from power information are divided into segments for each note.

[Prior Art] In an automatic transcription system for converting an acoustic signal such as a singing voice, a humming voice, or a musical instrument sound into musical score data, a sound length, a pitch, a key, a time signature, and the like, which are basic information as a musical score from an acoustic signal. Detecting the tempo.

By the way, an acoustic signal is only a signal that continuously includes a repetitive waveform of a basic waveform, and the above-described information cannot be obtained immediately. For example, Japanese Patent Application Laid-Open No. 58-181090 discloses a "note creation device" which performs an autocorrelation analysis on an acoustic signal, calculates pitch information (repetition information of a basic waveform representing the pitch of the acoustic signal) from the analysis result, and outputs the same pitch. An apparatus for determining the length of a is disclosed. However, this method merely extracts the pitch and length on the scale of the input musical tone based on the pitch information extracted from the acoustic signal, and thus classifies the pitch information into sections (segments) regarded as having the same pitch. The accuracy of the segmentation process to be performed is low, the process of identifying the pitch on the absolute scale, the determination of the key of the audio signal based on the identified pitch information, and the determination of the beat or tempo of the audio signal based on the segment. There is a problem that the post-processing accuracy is low.

Therefore, in the conventional automatic transcription method, first, pitch information and power information are extracted for each analysis cycle, and then, based on at least the extracted power information, an audio signal is divided into segments that can be regarded as having the same pitch. From the pitch information, the pitch of the audio signal of each segment is identified as a pitch along the absolute pitch axis, the tone of the audio signal is determined based on the identified pitch information, and the beat and tempo of the audio signal are determined based on the segment. Each information was obtained in the order of deciding.

Therefore, the pitch, time signature, tempo, and the like are determined based on the segment (length), and the segmentation process is an important element particularly in creating score data.

[Problems to be Solved by the Invention] By the way, an acoustic signal has a feature that its power becomes large immediately after a change in sound, and as described above, segmentation is performed using power information by using this feature. ing.

However, an acoustic signal, especially an acoustic signal sung by a human, does not always change in power information in a specific pattern, and has fluctuations with respect to the change pattern. Some sounds are included. For this reason, even if a simple segmentation is performed while focusing on a change in the power information, it is not always possible to satisfactorily perform the division for each sound.

As described above, segmentation is an important element in creating score data, and if the accuracy of segmentation is low, the accuracy of score data that is finally obtained, such as pitch, will also be extremely low. It is desired to increase the accuracy of itself.

On the other hand, Japanese Patent Application Laid-Open No. 58-223198, "Syllable Input Method", discloses a syllable input method in which syllable segmentation for voice recognition is performed. This calculates the power of the input voice for each frame of a predetermined length,
A speech recognition method that extracts a silent section by determining the calculated power as a threshold value, extracts a silent section with a valley of voice power, detects a voice section by using a silent section and a syllable end, and outputs a kana character string. The purpose of the present invention is to detect a voice section from an input voice. However, this voice input method merely extracts information on the valley of voice power in order to detect a voice section even if there is no long pause between syllables. Despite the background that pitch information is originally unnecessary for kanakana character strings, the technology for segmenting syllables based on the valleys of audio power has been applied to the automatic transcription technology for acoustic signals that contain repeated waveforms of the basic waveform as they are. Apparently not applicable. That is, in the automatic transcription of singing sounds and the like, it is necessary to finally identify an acoustic signal for each note, and a technique for detecting a period in which the same interval continues before a syllable is indispensable. Thus, it was clear that automatic transcription could not be performed even if the segmentation was performed by focusing only on power.

The present invention has been made in consideration of the above points, and can perform segmentation based on pitch information and power information satisfactorily, and can improve the accuracy of final score data. It is intended to provide a method and apparatus.

[Means for Solving the Problems] In order to solve such problems, the present invention provides an automatic music transcription method for converting an acoustic signal into musical score data composed of a time series of musical notes. Extract the pitch information and power information of the sound signal for each
For the period in which the pitch information indicates the same interval, the power information is divided into an effective section of a predetermined value or more and an invalid section of less than the predetermined value, and an effective section of less than a predetermined length is regarded as an invalid section, The sound is divided into segments for each sound.

[Operation] According to the present invention, for a section that can be regarded as the same pitch based on the pitch information, the power information is subjected to threshold value discrimination and segmentation is performed, so that the acoustic signal is divided into segments for each sound, and noise components and power Automatic transcription including the rest period is performed without being affected by fluctuations of the sound, and effective sections shorter than a predetermined length are set as invalid sections, thereby eliminating too short valid sections due to noise and the like. In addition, the accuracy of automatic transcription can be improved.

Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings.

Automatic transcription system First, an automatic transcription system to which the present invention is applied will be described.

In FIG. 3, a central processing unit (CPU) 1 controls the whole of the apparatus, and performs a musical notation processing shown in FIG. 4 stored in a main storage device 3 connected via a bus 2. Execute the program. To the bus 2, in addition to the CPU 1 and the main storage device 3, a keyboard 4 as an input device, a display device 5 as an output device, an auxiliary storage device 6 used as a working memory, and an analog / digital converter 7 are connected. I have.

An audio signal input device 8 including, for example, a microphone is connected to the analog / digital converter 7.
The acoustic signal input device 8 captures an acoustic signal such as singing or humming uttered by a user or a musical tone generated from a musical instrument and converts the signal into an electric signal, and converts the electric signal into an analog / digital converter. 7 is output.

When a process is instructed by the keyboard input device 4, the CPU 1 starts the transcription process, executes a program stored in the main storage device 3, and converts the sound converted into a digital signal by the analog / digital converter 7. The signals are temporarily stored in the auxiliary storage device 6, and thereafter, these sound signals are converted into musical score data composed of time series of musical notes by executing the above-described program, and are output to the display device 5 as necessary. Has been made.

Next, the transcription process performed by the CPU 1 after capturing the audio signal will be described in detail with reference to the flowchart shown in FIG.

First, the CPU 1 performs an autocorrelation analysis of the acoustic signal to extract pitch information of the acoustic signal at each analysis cycle, and also performs a sum-of-squares process to extract power information at each analysis cycle, and then performs noise removal and smoothing processing. And other post-processing (steps SP1, SP
2). Thereafter, for the pitch information, the CPU 1 calculates a shift amount of the pitch axis of the acoustic signal with respect to the absolute pitch axis based on the distribution state, and performs a tuning process of shifting the obtained pitch information according to the shift amount. Execute (step SP3). That is, the pitch coasting information is corrected so that the difference between the pitch axis of the singer or musical instrument that generated the acoustic signal and the absolute pitch axis becomes smaller.

Next, the CPU 1 obtains a continuous period of pitch information in which the obtained pitch information is considered to indicate the same pitch,
A segmentation is performed to divide the acoustic signal into segments for each sound, and a segmentation is performed based on the obtained change in the power information (step SP
4, SP5). Based on these two pieces of segment information obtained, the CPU 1 calculates a reference length corresponding to a time length of a quarter note, an eighth note, etc., and executes the segmentation again based on this reference length (step SP6). .

The CPU 1 identifies the pitch of the segment as a pitch on the absolute pitch axis that can determine that the pitch information is the closest based on the pitch information of the segment thus segmented, and further identifies the pitch of the identified continuous segment. Segmentation is again performed based on whether or not are the same (steps SP7 and SP8).

Thereafter, the CPU 1 detects the appearance frequency of the pitch based on the distribution of the pitch information after the tuning process, obtains the product sum of the appearance frequency of the pitch and a predetermined weighting coefficient determined according to the key, and obtains this product. On the basis of the maximum information of the sum, for example, the key of the music of the input audio signal is determined such as C major or A minor, and for the pitch of the predetermined scale in the determined key, the pitch is described in more detail with respect to the pitch information. Review and confirm and correct the pitch (steps SP9 and SP10). Next, CUP1 performs a segmentation review based on whether or not there is an identical one in consecutive segments from the finally determined pitch, and whether or not there is a change in power between consecutive segments, Final segmentation is performed (step SP11).

When the pitch and segment are determined in this way, the CP
U1 extracts measures from the viewpoint that the music starts from the first beat, the last sound of the phrase does not extend to the next measure, there is a break in each measure, etc., and determines the time signature from this measure information and segmentation information Then, a tempo is determined from the determined time signature information and the bar length (step SP12,
SP13).

Then, the CPU 1 organizes the information on the determined pitch, pitch, key, beat, and tempo to finally create the musical score data (step SP14).

Segmentation Based on Power Information Next, a segmentation process based on power information in such an automatic transcription system (see step SP5) will be described in detail with reference to the flowchart of FIG.

The power information of the acoustic signal is obtained by squaring the acoustic signal at each sampling point in the analysis cycle,
The sum of the power values is used as power information in the analysis cycle.

CPU1 first clears the analysis point parameter t to 0,
It is confirmed that the data to be processed is not completed, and it is determined whether or not the power information pow (t) at the analysis point t is equal to or larger than the threshold value θ1 (steps SP20 to SP22). If the result is smaller than the threshold value θ1 and a negative result is obtained, the parameter t is incremented and the process returns to step SP21 (step SP23).

By repeating this processing, the CPU 1 eventually finds an analysis point that takes the power pow (t) above the threshold θ1 and obtains a positive result in step SP22. At this time,
After marking the start of the segment at this analysis point, the CPU 1 confirms that the data of the analysis point to be processed has not been completed, and the power information pow (t) at the analysis point t has a threshold θ.
It is determined whether it is smaller than 1 (steps SP24 to SP26).
If the result is not less than the threshold θ1 and a negative result is obtained, the parameter t
Is incremented, and the process returns to step SP25 (step SP27).

By repeating such processing, the CPU 1 eventually finds an analysis point that takes power information pow (t) smaller than the threshold θ1, and obtains a positive result in step SP26. At this time, after marking the analysis point with the end of the segment, the CPU 1 detects the length L of the segment from the above-described mark information of the start of the segment and the mark information of the end of the segment. Judge whether it is small (step SP
28-30). This determination step is a step for preventing a too short segment from being regarded as a valid segment, and the threshold value θ2 is determined from the relationship with musical notes. CP
If U1 obtains a positive result in this step SP30, after removing the mark of the start and end of the segment, the parameter t is incremented and the process returns to the above-described step SP21. When the result is obtained, the parameter t is immediately incremented without removing the mark, and the process returns to step SP21 (step SP31,
32).

By repeating such processing, processing of all power information is eventually completed, and step SP22 or
A positive result is obtained in SP25, and the gram is ended.

FIG. 2 shows an example of a time change of the power information and a segmentation result corresponding to the time change.
In the case of this example, the segments S1, S2,... SM are obtained by executing the processing of FIG. In addition, from time t1
In the period of t2, the power information exceeds the threshold θ1,
Since this period is short and its length is equal to or less than the threshold θ2,
It is not extracted as a segment.

Therefore, according to the above-described embodiment, a period in which the power information is equal to or larger than the threshold is detected, and when the length is sufficiently long, the segment is extracted as a segment. Therefore, erroneous segmentation due to noise components or fluctuations may be performed. No high-precision segmentation can be performed. In addition, since the power information period equal to or greater than the threshold value is used as a segment, an unstable period of a low power interval is not used for the pitch identification process, and the pitch identification process can be performed well.

Other Embodiments In the above-described embodiment, the power information is obtained by using the sum of squares of the audio signal. However, other parameters may be used. For example, the square root of the sum of squares may be used.

Further, in the above-described embodiment, the CPU 1 executes all the processing shown in FIG. 4 according to the program stored in the main storage device 3. However, a part or all of the processing is performed by a hardware configuration. May be executed. For example, as shown in FIG. 5 in which the same reference numerals are given to the corresponding parts in FIG. 3, after the sound signal from the sound signal input device 8 is amplified through the amplifier circuit 10, the pre-filter is further added.
The digital signal is supplied to an analog / digital converter 12 via an analog-to-digital converter 11 and converted into a digital signal. The acoustic signal converted to the digital signal is subjected to autocorrelation analysis by a signal processor 13 to extract pitch information, and a square sum processing is performed. And extract power information
It may be provided to the software processing system by the CPU1. As the signal processor 13 used in such a hardware configuration (10 to 13), a processor capable of processing a signal in a voice band in real time and providing an interface signal with a host CPU 1 (for example, NEC Corporation) Co., Ltd. μpD7720) can be applied.

[Effects of the Invention] As described above, according to the present invention, the period in which the pitch information indicates the same pitch is divided into an effective section in which the power information is equal to or more than a predetermined value and an invalid section in which the power information is less than the predetermined value. In addition, an effective section shorter than a predetermined length is determined to be an invalid section and is divided into segments for each note. By segmenting a power information period exceeding a predetermined value as a segment, an unstable period of a low-pitch pitch is obtained. Periods are excluded from the process of pitch identification processing, and high-precision pitch identification processing is possible.Since the period in which the same pitch is specified based on pitch information obtained from autocorrelation analysis is specified, only power information is used. It is possible to precisely analyze the note arrangement at the same interval, which is impossible only by discriminating the threshold value and dividing it into syllables. By converting power information that exceeds a predetermined value into segments when the length is sufficiently long, erroneous segmentation due to noise components and power fluctuations can be performed. , Segmentation with extremely high accuracy can be performed, and the following processes such as the pitch identification process using the segmentation result can be performed satisfactorily.

[Brief description of the drawings]

FIG. 1 is a flowchart showing a segmentation process based on power information according to an embodiment of the present invention.
FIG. 3 is a characteristic curve diagram showing a temporal change of power information together with a segmentation result, FIG. 3 is a block diagram showing a configuration of an automatic transcription system to which the present invention is applied, and FIG. 4 is a diagram shown in FIG. FIG. 5 is a flowchart showing an automatic transcription process, and FIG. 5 is a block diagram showing another configuration of the automatic transcription system. DESCRIPTION OF SYMBOLS 1 ... CPU 3 ... Main storage device 6 ... Auxiliary storage device 7 ... Analog / digital converter 8 ... Audio signal input device

 ──────────────────────────────────────────────────続 き Continued on the front page (72) Inventor Masaki Fujimoto 5-7-15 Shiba, Minato-ku, Tokyo Inside NEC Technical Information System Development Co., Ltd. (72) Inventor Masanori Mizuno 5-7-1 Shiba, Minato-ku, Tokyo No. 15 Examiner, NEC Technical Information Systems Development Co., Ltd. Shigeo Arai (56) References JP-A-58-181090 (JP, A) JP-A-58-223198 (JP, A)

Claims (2)

(57) [Claims] 1. An automatic music transcription method for converting an acoustic signal into musical score data comprising a time series of musical notes, wherein said acoustic signal is subjected to an autocorrelation analysis, and pitch information and power information of the acoustic signal are extracted for each analysis cycle. For a period in which the pitch information indicates the same pitch, the power information is divided into an effective section of a predetermined value or more and an invalid section of less than the predetermined value, and an effective section of less than a predetermined length is regarded as an invalid section,
An automatic music transcription method characterized by dividing each sound into segments. 2. An automatic music transcription apparatus for converting an acoustic signal into musical score data comprising a time series of musical notes, wherein said acoustic signal is subjected to autocorrelation analysis, and information extraction for extracting pitch information and power information of the acoustic signal at each analysis cycle. Means, for a period in which the pitch information indicates the same interval, the power information is divided into an effective section of a predetermined value or more and an invalid section of less than the predetermined value, and an invalid section of less than a predetermined length is invalid. An automatic music transcription apparatus comprising: a segmentation unit that divides the sound into segments for each sound.