JP2604414B2 - Automatic music transcription method and device - Google Patents

Description

Description: BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an automatic music transcription method and apparatus for creating musical score data from audio signals such as singing voices, humming voices, and instrument sounds, and more particularly, to a predetermined section of an audio signal. The present invention relates to a pitch identification method for identifying a pitch on an absolute pitch axis as a pitch of the pitch.

[Prior Art] In an automatic transcription system for converting an acoustic signal such as a singing voice, a humming voice, or a musical instrument sound into musical score data, a sound length, a pitch, a key, a time signature, and the like, which are basic information as a musical score from an acoustic signal. Detecting the tempo.

By the way, an acoustic signal is only a signal that continuously includes a repetitive waveform of a basic waveform, and the above-described information cannot be obtained immediately.

Therefore, in the conventional automatic transcription method, first, repetition information (hereinafter, referred to as pitch information) of a basic waveform representing a pitch of an acoustic signal and power information are extracted for each analysis cycle, and thereafter, the extracted pitch information is extracted. And / or dividing the audio signal into sections (segments) that can be regarded as the same pitch based on the power information (this processing is called segmentation), and then, based on the pitch information of the segment, the pitch along the absolute pitch axis as the pitch of the audio signal of each segment. , The tone of the acoustic signal is determined based on the distribution information about the pitch axis of the pitch information, and the time signature and the tempo of the acoustic signal are determined based on the segments.

[Problems to be Solved by the Invention] By the way, even if an attempt is made to identify a certain segment of an acoustic signal as a pitch on an absolute pitch axis, the pitch of an acoustic signal, particularly an acoustic signal uttered by a human, is not stable and the same. Even if the pitch is intended, there is much fluctuation in the pitch. For this reason, the pitch identification processing is very difficult.

Since the pitch is a fundamental element of the musical score data together with the pitch, it is necessary to identify it accurately, and if it cannot be accurately identified, the accuracy of the musical score data will be low.

The present invention has been made in consideration of the above points, and proposes a new pitch identification method capable of accurately identifying pitches, and automatic transcription that can further improve the accuracy of final score data. It is intended to provide a method and apparatus.

[Means for Solving the Problems] In order to solve the problems, in the first aspect of the present invention, pitch information representing a pitch and a power information of an acoustic signal, which is a repetition period of an input acoustic signal waveform, is extracted. And a segmentation process of dividing the acoustic signal into sections that can be regarded as having the same pitch based on the pitch information and / or the power information, and a pitch identification process of determining a pitch on the absolute pitch axis of the acoustic signal for the divided section. In an automatic transcription method for converting an acoustic signal into musical score data, the pitch identification processing is performed for all analysis points in each of the divided sections, and pitch information within a predetermined range centered on the pitch information of the analysis points. A process of counting the length of a continuous period that takes a value, and a process of extracting the longest period among the counted periods for each of the divided sections. And a process of identifying a pitch of the segment as a pitch on the absolute pitch axis based on the extracted pitch information of the analysis point giving the longest period.

Further, in the second aspect of the present invention, pitch / power extraction means for extracting pitch information representing a pitch and power information of the audio signal, which is a repetition period of the input audio signal waveform, comprises pitch information and / or power information. Segmentation means for classifying an audio signal into sections which can be regarded as having the same pitch on the basis of the pitch, and pitch identification means for determining a pitch on the absolute pitch axis of the audio signal for the divided section. In an automatic transcription apparatus for converting data into data, a pitch identification means is provided for all analysis points in each segmented section for a length of a continuous period in which a pitch information value within a predetermined range centered on the pitch information of the analysis point. A length detection unit for counting the run length, a longest period extraction unit for extracting the longest period of the counted periods for each of the divided sections, It was constructed by the pitch determiner for identifying the pitch of the segment to the pitch on the absolute musical interval axis by the pitch information of the analysis points giving period.

[Operation] In the first aspect of the present invention, although the acoustic signal has fluctuations, attention is paid to fluctuations in a narrow range around the intended pitch, and pitch information is obtained for each analysis point in the divided section. The length of a continuous period connected to the analysis point having pitch information that falls within a predetermined upper and lower value around the center is measured, and the pitch information of the analysis point that gives the longest period in the counted continuous period is close. The pitch of the section is identified as the pitch on the absolute pitch axis.

Further, according to a second aspect of the present invention, based on the similar attention, for the analysis points in each of the divided sections, a continuous period in which a change in pitch information of the analysis points falls within a predetermined value is counted. The maximum length of the counted period is extracted by the longest period extracting unit, and the pitch determining unit identifies the pitch of the segment based on the pitch frequency of the analysis point giving the longest period. did.

Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings.

Automatic transcription system First, an automatic transcription system to which the present invention is applied will be described.

In FIG. 4, a central processing unit (CPU) 1 controls the whole of the apparatus, and performs a transcription process shown in FIG. 5 stored in a main storage device 3 connected via a bus 2. Execute the program. In addition to the CPU 1 and the main storage device 3, a keyboard 4 as an input device, a display device 5 as an output device, an auxiliary storage device 6 used as a working memory, and an analog / digital converter 7 are connected to the bus 2. I have.

An audio signal input device 8 including, for example, a microphone is connected to the analog / digital converter 7. The acoustic signal input device 8 captures acoustic signals such as singing and humming uttered by a user and musical sounds generated from musical instruments and converts the signals into electric signals, and converts the electronic signals into analog / digital signals. Output to the container 7.

When a process is instructed by the keyboard input device 4, the CPU 1 starts the transcription process, executes a program stored in the main storage device 3, and converts the sound converted into a digital signal by the analog / digital converter 7. The signals are temporarily stored in the auxiliary storage device 6, and thereafter, these sound signals are converted into musical score data by executing the above-described program and output to the display device 5 as necessary.

Next, the music transcription process performed by the CPU 1 after capturing the audio signal will be described in detail with reference to the flowchart shown in the functional level of FIG.

First, the CPU 1 performs an autocorrelation analysis of the acoustic signal to extract pitch information of the acoustic signal at each analysis cycle, and also performs a sum-of-squares process to extract power information at each analysis cycle, and then performs noise removal and smoothing processing. And other post-processing (steps SP1, SP
2). Thereafter, for the pitch information, the CPU 1 calculates a shift amount of the pitch axis of the acoustic signal with respect to the absolute pitch axis based on the distribution state, and performs a tuning process of shifting the obtained pitch information according to the shift amount. Execute (step SP3). That is, the pitch information is corrected so that the difference between the pitch axis and the absolute pitch axis of the singer or musical instrument that has generated the acoustic signal is reduced.

Next, the CPU 1 obtains a continuous period of pitch information in which the obtained pitch information is considered to indicate the same pitch,
A segmentation is performed to divide the acoustic signal into segments for each sound, and a segmentation is performed based on the obtained change in the power information (step SP
4, SP5). Based on these two pieces of segment information obtained, the CPU 1 calculates a reference length corresponding to a time length of a quarter note, an eighth note, etc., and executes the segmentation again based on this reference length (step SP6). .

The CPU 1 identifies the pitch of the segment as a pitch on the absolute pitch axis that can determine that the pitch information is the closest based on the pitch information of the segment thus segmented, and further identifies the pitch of the identified continuous segment. Segmentation is again performed based on whether or not are the same (steps SP7 and SP8).

Thereafter, the CPU 1 obtains a product sum of the frequency of appearance of the pitch obtained by summing the pitch information after tuning and a predetermined weighting coefficient determined according to the key, and based on the maximum information of the product sum, for example, Determine the key of the musical composition of the input audio signal, such as C major or A minor, and review and correct the pitch of the predetermined pitch on the scale in the determined key with respect to the pitch information (steps SP9 and SP10). ). Next, the CPU 1 executes a review of the segmentation based on whether or not there is the same continuous segment from the finally determined pitch, and whether or not there is a power change between the continuous segments, Final segmentation is performed (step SP11).

Once the pitch and segment are determined in this way,
The CPU 1 extracts measures from the viewpoint that the music starts from the first beat, the last sound of the phrase does not extend to the next measure, and there is a break in each measure, and determines the time signature from the measure information and the segmentation information. The tempo is determined from the determined time signature information and the length of the bar (step SP1).
2, SP13).

Then, the CPU 1 organizes the information on the determined pitch, pitch, key, beat, and tempo to finally create the musical score data (step SP14).

Next, the pitch identification processing (see step SP7 in FIG. 5) in such an automatic transcription system will be described in detail with reference to the flowchart in FIG.

The CPU 1 first extracts the first segment from the segment obtained by the segmentation process (step SP6 in FIG. 5), and counts the run length run (t) for each analysis point in the segment (steps SP20 and SP21). .

Here, the length of the run will be described with reference to FIG.
FIG. 2 shows a temporal change of the pitch information. The horizontal axis represents the analysis point t, and the vertical axis represents the pitch information. As an example, the length of the run at the analysis point tp will be described.

As shown in FIG. 2, the range of the analysis point which takes a value between the pitch information ho and h2 vertically shifted by a minute width Δh from the pitch information h1 of the analysis point tp is from the analysis point to to the analysis point ts. And a period L from the analysis point to to the analysis point ts
Is the length of the run of the analysis point tp.

When the run length run (t) is counted for all the analysis points in the segment, the CPU 1 determines the run length run (t).
Extract the analysis point whose (t) is the longest (step SP2)
2). Thereafter, the CPU 1 extracts the pitch information at the analysis point giving the longest run length run (t), and identifies the pitch of the segment as the pitch on the absolute pitch axis closest to this pitch information (step SP23). Note that the pitch of each segment of the acoustic signal is identified as one of the pitches that differs by a semitone on the absolute pitch axis.

Next, the CPU 1 determines whether or not the segment whose pitch has been identified by performing such processing is the last segment (step SP24). As a result, when the process is completed,
If the processing program is terminated and the processing is not terminated, the next segment is set as a processing target and the above-described step 21 is performed.
Return to (Step SP25).

By repeating the processing loop including steps SP21 to SP25, pitch identification is performed on all the segments based on the pitch information of the analysis point that gives the length of the longest run in the segment.

Here, the use of the run length run (t) for the pitch identification processing is performed when the singer or the like intends to use the same pitch, although the acoustic signal has fluctuations. In fact, it has been confirmed that the correlation between the pitch information of the analysis point giving the longest run length and the intended pitch is extremely high.

FIG. 3 shows an example of pitch identification of an input acoustic signal by such processing.

In FIG. 3, the pitch information distribution with respect to the analysis cycle is indicated by a dotted curve 15. The vertical lines 18a, 18b, 18c and 18d are
The segmentation by segmentation is shown, and the solid line 17 in the horizontal direction shows the pitch information of the analysis point that gives the length of the longest run in the segment. A dotted line 16 indicates a pitch identified by the pitch information. As is apparent from FIG. 3, the pitch information giving the longest run length has a small deviation from the pitch on the absolute pitch axis, and the pitch can be identified well.

Therefore, according to the above-described embodiment, the length of the run is determined for the analysis point of each segment, and the pitch information of the analysis point at which the length of the run is maximized is extracted. A continuous part with a small change is extracted, that is, a part with a small pitch change.The pitch of each segment is identified by the part with a small pitch change. Can be.

Other Embodiments Note that the pitch information used for the pitch identification processing may be represented by Hz in frequency units, or represented by cent units often used in the music field. It may be something.

In the above-described embodiment, the CPU 1 executes all the processing shown in FIG. 5 according to the program stored in the main storage device 3. However, a part or all of the processing is performed by a hardware configuration. May be executed. For example, as shown in FIG. 6 in which the same reference numerals are given to the corresponding parts in FIG. 4, the audio signal from the audio signal input device 8 is amplified through the amplifier circuit 10, and then the pre-filter is added.
The digital signal is supplied to an analog / digital converter 12 via an analog-to-digital converter 11 and converted into a digital signal. The acoustic signal converted to the digital signal is subjected to autocorrelation analysis by a signal processor 13 to extract pitch information, and a square sum processing is performed. And extract power information
It may be provided to the software processing system by the CPU1. As the signal processor 13 used in such a hardware configuration (10 to 13), a processor capable of processing a signal in a voice band in real time and providing an interface signal with a host CPU 1 (for example, NEC Corporation) Co., Ltd. μPD7720) can be applied.

[Effects of the Invention] As described above, according to the present invention, the pitch identification process is performed from the pitch information of the analysis point that gives the longest run length. The accuracy of the musical score data can be further improved.

[Brief description of the drawings]

FIG. 1 is a flowchart showing a pitch identification process according to one embodiment of the present invention, FIG. 2 is an explanatory diagram of a run length applied to the present invention, and FIG. 3 is a schematic diagram showing an example of the pitch identification process. FIG. 4 is a block diagram showing the configuration of an automatic transcription system to which the present invention is applied, FIG. 5 is a flowchart showing the automatic transcription process, and FIG. 6 is a block diagram showing another configuration of the automatic transcription system. It is. 1 ... CPU, 3 ... main storage device, 6 ... auxiliary storage device,
7 ... A / D converter, 8 ... Acoustic signal input device.

 ──────────────────────────────────────────────────続 き Continued on the front page (72) Inventor Masanori Mizuno Examiner, NEC Technical Information System Development Co., Ltd. Shigeo Arai, 5-7-15 Shiba, Minato-ku, Tokyo

Claims (2)

(57) [Claims] 1. A process for extracting pitch information representing a pitch and power information of the sound signal, which is a repetition period of an input sound signal waveform, and processing the sound signal based on the pitch information and / or the power information. Automatically segmenting the sound signal into musical score data, including at least a segmentation process of segmenting the sound signal into the same interval and a pitch identification process of determining a pitch on the absolute pitch axis of the acoustic signal for the segmented segment. In the music notation method, the pitch identification processing determines a length of a continuous period in which a pitch information value within a predetermined range around the pitch information of the analysis point is obtained for all the analysis points in each of the divided sections. Counting, extracting the longest period of the counted periods for each of the divided sections, and providing the extracted maximum period. Automatic transcription method by the pitch information of the analysis points, characterized in that it consists of a process of identifying the pitch of the segment to the pitch on the absolute musical interval axis that. 2. A pitch / power extracting means for extracting pitch information representing a pitch and power information of the audio signal, which is a repetition period of an input audio signal waveform, and based on the pitch information and / or the power information. Segmentation means for classifying the acoustic signal into sections that can be regarded as having the same pitch, and pitch identification means for determining a pitch on the absolute pitch axis of the acoustic signal for the divided section. In the automatic music transcription apparatus for converting into musical score data, the pitch identifying means may be configured to continuously obtain pitch information values within a predetermined range around the pitch information of the analysis points for all the analysis points in each of the divided sections. Length detection unit for counting the length of the divided period, and the longest period for extracting the longest period among the counted periods for each of the divided sections. Extractor and, extracted automatic music transcription apparatus characterized by being configured by the pitch determiner by the pitch information of the analysis points which gives the maximum period to identify the pitch of the segment to the pitch on the absolute musical interval axis.