JP2614631B2 - Automatic music transcription method and device - Google Patents

Abstract

PURPOSE:To easily and well determine an intervale, by determining the intervale of a section on the basis of the average value of section pitch data with respect to a section of an acoustic signal divided by segmentation processing. CONSTITUTION:The digital acoustic signal passing through an acoustic signal input apparatus 8 and an A/D converter 7 by a CPU 1 corresponding to the order from a keyboard 4 is stored in the auxiliary memory device 6 of a working memory and the program from a main memory device 3 is executed. The pitch data of the acoustic signal is extracted to perform segmentation processing for dividing the acoustic signal into the section of the same intervale. Further, the average value of the pitch data of each section is calculated and the intervale on the absolute intervale axis nearest to the average value is determined as the intervale of each section and an intervale is easily and well determined to enhance the accuracy of the score automatically taken from the acoustic signal.

Description

Description: BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an automatic music transcription method and apparatus for creating musical score data from audio signals such as singing voices, humming voices, and instrument sounds, and more particularly, to a predetermined section of an audio signal. This relates to a pitch identification process for identifying a pitch on the absolute pitch axis as a pitch of.

[Prior Art] In an automatic transcription system for converting an acoustic signal such as a singing voice, a humming voice, or a musical instrument sound into musical score data, a sound length, a pitch, a key, a time signature, and the like, which are basic information as a musical score from an acoustic signal. Detecting the tempo.

By the way, an acoustic signal is only a signal that continuously includes a repetitive waveform of a basic waveform, and the above-described information cannot be obtained immediately.

Therefore, in the conventional automatic transcription method, first, repetition information (hereinafter, referred to as pitch information) of a basic waveform representing a pitch of an acoustic signal and power information are extracted for each analysis cycle, and thereafter, the extracted pitch information and Alternatively, the audio signal is divided into sections (segments) that can be regarded as having the same pitch from the power information (this processing is called segmentation),
Identify the pitch along the absolute pitch axis of the audio signal of each segment from the pitch information of the segment, determine the tone of the audio signal based on the distribution information around the pitch axis of the pitch information, and further determine the audio signal based on the segment. Each information was obtained in the order of determining the time signature and tempo.

[Problems to be Solved by the Invention] By the way, even if an attempt is made to identify a certain segment of an acoustic signal as a pitch on an absolute pitch axis, the pitch of an acoustic signal, particularly an acoustic signal uttered by a human, is not stable and the same. Even if the pitch is intended, there is much fluctuation in the pitch. For this reason, the pitch identification processing is very difficult.

Since the pitch is a fundamental element of the musical score data together with the pitch, it is necessary to identify it accurately. If the musical score data cannot be identified accurately, the accuracy of the musical score data is reduced.

The present invention has been made in consideration of the above points, and proposes a new pitch identification method capable of accurately identifying pitches, and automatic transcription that can further improve the accuracy of final score data. It is intended to provide a method and apparatus.

[Means for Solving the Problems] In order to solve the problems, in the first aspect of the present invention, pitch information representing a pitch and a power information of an acoustic signal, which is a repetition period of an input acoustic signal waveform, is extracted. And a segmentation process of dividing the acoustic signal into sections that can be regarded as having the same pitch based on the pitch information and / or the power information, and a pitch identification process of determining a pitch on the absolute pitch axis of the acoustic signal for the divided section. In an automatic transcription method for converting an acoustic signal into musical score data, the pitch identification processing calculates an average value of pitch information for each of the divided sections, and an absolute pitch in which the calculated average value is the closest. And determining the pitch of each section on the on-axis pitch.

Further, in the second aspect of the present invention, pitch / power extraction means for extracting pitch information representing a pitch and power information of the audio signal, which is a repetition period of the input audio signal waveform, comprises pitch information and / or power information. Segmentation means for classifying an audio signal into sections that can be regarded as having the same pitch based on the sound signal, and pitch identification means for determining a pitch on the absolute pitch axis of the audio signal for the divided section. In an automatic transcription apparatus that converts data into data, a pitch identification unit is provided with an average value calculation unit that calculates an average value of pitch information for each section divided by the segmentation unit, and a calculated average value on an absolute pitch axis closest to the pitch information. A pitch determination unit for determining a pitch in the section is provided.

[Operation] In the first aspect of the present invention, when the pitch of each section is identified as the pitch on the absolute pitch axis, even if the sound signal fluctuates, the sound signal fluctuates around the intended pitch. Focusing on a point at which the average pitch corresponds very well to the average value of the pitch information, the average value of each section is calculated, and the average value is identified as a pitch on the absolute pitch axis that is close.

Further, the second present invention similarly calculates an average value of the pitch information of each segmented section by the average value calculating unit based on the fact that the average value of the pitch information is close to the intended pitch of the audio signal, The interval in the section is identified as the interval on the absolute interval axis whose average value calculated by the interval determining unit is close.

Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings.

Automatic transcription system First, an automatic transcription system to which the present invention is applied will be described.

In FIG. 3, a central processing unit (CPU) 1 controls the whole of the apparatus, and performs a musical notation processing shown in FIG. 4 stored in a main storage device 3 connected via a bus 2. Execute the program. In addition to the CPU 1 and the main storage device 3, a keyboard 4 as an input device, a display device 5 as an output device, an auxiliary storage device 6 used as a working memory, and an analog / digital converter 7 are connected to the bus 2. I have.

An audio signal input device 8 including, for example, a microphone is connected to the analog / digital converter 7. The acoustic signal input device 8 captures an acoustic signal such as singing or humming uttered by a user or a musical tone generated from a musical instrument and converts the signal into an electric signal, and converts the electric signal into an analog / digital converter. 7 is output.

When a process is instructed by the keyboard input device 4, the CPU 1 starts the transcription process, executes a program stored in the main storage device 3, and converts the sound converted into a digital signal by the analog / digital converter 7. The signals are temporarily stored in the auxiliary storage device 6, and thereafter, these sound signals are converted into musical score data by executing the above-described program and output to the display device 5 as necessary.

Next, the transcription process performed by the CPU 1 after capturing the audio signal will be described in detail with reference to the flowchart shown in FIG.

First, the CPU 1 performs an autocorrelation analysis of the acoustic signal to extract pitch information of the acoustic signal at each analysis cycle, and also performs a sum-of-squares process to extract power information at each analysis cycle, and then performs noise removal and smoothing processing. And other post-processing (steps SP1, SP
2). Thereafter, for the pitch information, the CPU 1 calculates a shift amount of the pitch axis of the acoustic signal with respect to the absolute pitch axis based on the distribution state, and performs a tuning process of shifting the obtained pitch information according to the shift amount. Execute (step SP3). That is, the pitch information is corrected so that the difference between the pitch axis and the absolute pitch axis of the singer or musical instrument that has generated the acoustic signal is reduced.

Next, the CPU 1 obtains a continuous period of pitch information in which the obtained pitch information is considered to indicate the same pitch,
A segmentation is performed to divide the acoustic signal into segments for each sound, and a segmentation is performed based on the obtained change in the power information (step SP
4, SP5). Based on these two pieces of segment information obtained, the CPU 1 calculates a reference length corresponding to the time length of a quarter note or eighth note, and executes the segmentation again based on this reference length (step SP6).

The CPU 1 identifies the pitch of the segment as a pitch on the absolute pitch axis that can determine that the pitch information is the closest based on the pitch information of the segment thus segmented, and further identifies the pitch of the identified continuous segment. Segmentation is again performed based on whether or not are the same (steps SP7 and SP8).

Thereafter, the CPU 1 obtains a product sum of the frequency of appearance of the pitch obtained by summing the pitch information after tuning and a predetermined weighting coefficient determined according to the key, and based on the maximum information of the product sum, for example, Determine the key of the musical composition of the input audio signal, such as C major or A minor, and review and correct the pitch of the predetermined pitch on the scale in the determined key with respect to the pitch information (steps SP9 and SP10). ). Next, the CPU 1 executes a review of the segmentation based on whether or not there is the same continuous segment from the finally determined pitch, and whether or not there is a power change between the continuous segments, Final segmentation is performed (step SP11).

Once the pitch and segment are determined in this way,
The CPU 1 extracts measures from the viewpoint that the music starts from the first beat, the last sound of the phrase does not extend to the next measure, and there is a break in each measure, and determines the time signature from the measure information and the segmentation information. The tempo is determined from the determined time signature information and the length of the bar (step SP1).
2, SP13).

Then, the CPU 1 organizes the information on the determined pitch, pitch, key, beat, and tempo to finally create the musical score data (step SP14).

Next, the pitch identification process (see step SP7) in such an automatic transcription system will be described in detail with reference to the flowchart of FIG.

CPU1 first extracts the first segment from the segments obtained by the segmentation, and then
An average value of all pitch information in the segment is calculated (steps SP20 and SP21).

Thereafter, the CPU 1 identifies a pitch on the absolute pitch axis having the closest calculated average value as a pitch of the segment (step SP22). Note that the pitch of each segment of the acoustic signal is identified as one of the pitches that differs by a semitone on the absolute pitch axis. The CPU 1 determines whether or not the segment whose pitch has been identified by performing the processing is the last segment (step SP23). As a result, when the process is completed,
If the processing program is terminated and the processing is not terminated, the next segment is set as a processing target and the above-described step 21 is performed.
Return to step SP24.

By repeating the processing loop consisting of steps SP21 to SP24, pitch identification is executed for all the segments based on the average value information of the pitch information in the segments.

Here, the average value is used for the pitch identification processing. Although the sound signal has fluctuation, it is considered that the sound signal fluctuates around a pitch intended by a singer or the like. This is because it is thought that it corresponds.

FIG. 2 shows an example of pitch identification by such processing, where a dotted curve P1 shows pitch information of an acoustic signal,
A vertical solid line VR indicates a segment break. The average value of each segment in this example is indicated by a horizontal solid line HR, and the identified pitch is indicated by a horizontal dotted line HP. As is apparent from FIG. 2, the average value has a small deviation with respect to the pitch on the absolute pitch axis and can be identified well.

Therefore, according to the above-described embodiment, the average value of the pitch information is calculated for each segment, and the interval of the segment is identified as the interval on the absolute pitch axis having the closest average value, so that the interval is determined with high accuracy. be able to. Since the acoustic signal is tuned prior to pitch identification, according to this method, the average value takes a value close to the pitch on the absolute pitch axis, and the identification is very easy.

Other Embodiments Note that the pitch information used for the pitch identification processing may be expressed in Hz in frequency units, or in segment units often used in the music field. It may be something.

In the above-described embodiment, the CPU 1 executes all the processing shown in FIG. 4 according to the program stored in the main storage device 3. However, a part or all of the processing is performed by a hardware configuration. May be executed. For example, as shown in FIG. 5 in which the same reference numerals are given to the corresponding parts in FIG. 3, after the sound signal from the sound signal input device 8 is amplified through the amplifier circuit 10, the pre-
The digital signal is supplied to an analog / digital converter 12 via an analog-to-digital converter 11 and converted into a digital signal. The acoustic signal converted to the digital signal is subjected to autocorrelation analysis by a signal processor 13 to extract pitch information, and a square sum processing is performed. And extract power information
It may be provided to the software processing system by the CPU1. As the signal processor 13 used in such a hardware configuration (10 to 13), a processor capable of processing a signal in a voice band in real time and providing an interface signal with a host CPU 1 (for example, NEC Corporation) Co., Ltd. μPD7720) can be applied.

[Effects of the Invention] As described above, according to the present invention, the pitch of each segment is identified based on the average value of the pitch information of the segment, so that the pitch can be determined well, and the accuracy of the score data can be determined. Can be further increased.

[Brief description of the drawings]

FIG. 1 is a flowchart showing a pitch identification process according to an embodiment of the present invention, FIG. 2 is a schematic diagram showing an example of the pitch identification process, and FIG. 3 shows a configuration of an automatic transcription system to which the present invention is applied. FIG. 4 is a flowchart showing the automatic transcription process, and FIG. 5 is a block diagram showing another configuration of the automatic transcription system. 1 ... CPU, 3 ... main storage device, 6 ... auxiliary storage device,
7 ... A / D converter, 8 ... Acoustic signal input device.

 ──────────────────────────────────────────────────続 き Continued on the front page (72) Inventor Masaki Fujimoto 5-7-15 Shiba, Minato-ku, Tokyo NEC Technical Information Systems Development Co., Ltd. (72) Inventor Masanori Mizuno 5-7-15 Shiba, Minato-ku, Tokyo No. NEC Technical Information System Development Co., Ltd.Examiner Shigeo Arai

Claims (2)

(57) [Claims] 1. A process for extracting pitch information representing a pitch and power information of the sound signal, which is a repetition period of an input sound signal waveform, and processing the sound signal based on the pitch information and / or the power information. Automatically segmenting the sound signal into musical score data, including at least a segmentation process for segmenting the sound signal into segments that can be regarded as the same pitch, and a pitch identification process for determining a pitch on the absolute pitch axis of the acoustic signal for the segmented segment. In the music transcription method, the pitch identification processing includes: calculating an average value of pitch information for each of the divided sections; and calculating a pitch of each of the sections to a pitch on an absolute pitch axis closest to the calculated average value. An automatic transcription method, comprising: determining. 2. A pitch / power extracting means for extracting pitch information representing a pitch and power information of the audio signal, which is a repetition period of an input audio signal waveform, and based on the pitch information and / or the power information. Segmentation means for classifying the acoustic signal into sections that can be regarded as having the same pitch, and pitch identification means for determining a pitch on the absolute pitch axis of the acoustic signal for the divided section. In an automatic transcription apparatus for converting musical score data, an average value calculating unit that calculates an average value of pitch information for each of the sections divided by the segmentation unit, wherein the calculated average value is the closest to the absolute value. An automatic music transcription apparatus, comprising: a pitch determining unit that determines a pitch in each of the above sections on a pitch on a pitch axis.