JP2604406B2 - Automatic music transcription method and device - Google Patents

Abstract

PURPOSE:To well determine an interval, by calculating the central value of the pitch data of the section of an acoustic signal divided by segmentation processing and determining the intervale of said section on the basis of the calculated value. CONSTITUTION:A CPU 1 stores the digital acoustic signal such as singing passing through an acoustic signal input apparatus 8 and an A/D converter 7 in the auxiliary memory device 6 of a working memory corresponding to the order from a keyboard 4 and executes the program from a main memory device 3. The pitch data of the acoustic signal is extracted to perform segmentation processing for dividing the acoustic signal into the section of the same interval. Further, the central value of the pitch data of each section is extracted and the intervale on the absolute intervale axis nearest to the central value as the intervale of said section and, even when the acoustic signal is fluctuated, a good intervale is identified to enhance the accuracy of the score taken from the acoustic signal.

Description

Description: BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an automatic music transcription method and apparatus for creating musical score data from audio signals such as singing voices, humming voices, and instrument sounds, and more particularly, to a predetermined section of an audio signal. This relates to a pitch identification process for identifying a pitch on the absolute pitch axis as a pitch of.

[Background Art] In an automatic transcription system that converts acoustic signals such as singing voices, humming voices, and musical instrument voices into musical score data, basic information as musical scores from the acoustic signals is pitch, pitch, key,
Detecting time signature and tempo.

By the way, an acoustic signal is only a signal that continuously includes a repetitive waveform of a basic waveform, and the above-described information cannot be obtained immediately.

Therefore, in the conventional automatic transcription method, first, repetition information (hereinafter, referred to as pitch information) of a basic waveform representing a pitch of an acoustic signal and power information are extracted for each analysis cycle, and thereafter, the extracted pitch information is extracted. And / or dividing the audio signal into sections (segments) that can be regarded as the same pitch based on the power information (this processing is called segmentation), and then, based on the pitch information of the segment, the pitch along the absolute pitch axis as the pitch of the audio signal of each segment. Each information is obtained in the order of determining the tone of the sound signal based on the distribution information around the pitch axis of the pitch information, and further determining the beat and tempo of the sound signal based on the segment.

[Problems to be Solved by the Invention] By the way, even if an attempt is made to identify a certain segment of an acoustic signal as a pitch on an absolute pitch axis, the pitch of an acoustic signal, particularly an acoustic signal uttered by a human, is not stable and the same. Even if the pitch is intended, there is much fluctuation in the pitch. For this reason, the pitch identification processing is very difficult.

Since the pitch is a fundamental element of the musical score data together with the pitch, it is necessary to identify it accurately. If the musical score data cannot be identified accurately, the accuracy of the musical score data is reduced.

The present invention has been made in consideration of the above points, and proposes a new pitch identification method capable of accurately identifying pitches, and automatic transcription that can further improve the accuracy of final score data. It is intended to provide a method and apparatus.

[Means for Solving the Problems] In order to solve the problems, in the first aspect of the present invention, pitch information representing a pitch and a power information of an acoustic signal, which is a repetition period of an input acoustic signal waveform, is extracted. And a segmentation process of dividing the acoustic signal into sections that can be regarded as having the same pitch based on the pitch information and / or the power information, and a pitch identification process of determining a pitch on the absolute pitch axis of the acoustic signal for the divided section. In an automatic transcription method for converting an acoustic signal into musical score data, the pitch identification process includes a process of extracting a median of pitch information for each of the divided sections, and an absolute pitch in which the extracted median is closest. And determining the pitch of each section on the on-axis pitch.

Further, in the second aspect of the present invention, pitch / power extraction means for extracting pitch information representing a pitch and power information of the audio signal, which is a repetition period of the input audio signal waveform, comprises pitch information and / or power information. Segmentation means for classifying an audio signal into sections that can be regarded as having the same pitch based on the sound signal, and pitch identification means for determining a pitch on the absolute pitch axis of the audio signal for the divided section. In an automatic transcription apparatus for converting data into data, a pitch identification means, a median value extraction unit for extracting a median value of pitch information for each section segmented by the segmentation means, and an extracted median value on an absolute pitch axis closest to the pitch information. A pitch determination unit for determining a pitch in the section is provided.

[Operation] In the first aspect of the present invention, when the pitch of each section is identified as the pitch on the absolute pitch axis, even if the sound signal fluctuates, the sound signal fluctuates around the intended pitch. Focusing on a point at which the median corresponding to the median of the pitch information corresponds very well, the median of each section is extracted, and the median is identified as a median on the absolute musical pitch axis that is close.

Further, the second present invention similarly extracts the median value of the pitch information of each segmented section by the median value extraction unit based on the fact that the median value of the pitch information is close to the intended pitch of the audio signal, The pitch determination section identifies the pitch in that section as the pitch on the absolute pitch whose extracted median is close.

Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings.

Automatic transcription system First, an automatic transcription system to which the present invention is applied will be described.

In FIG. 3, a central processing unit (CPU) 1 controls the whole of the apparatus, and performs a musical notation processing shown in FIG. 4 stored in a main storage device 3 connected via a bus 2. Execute the program. In addition to the CPU 1 and the main storage device 3, a keyboard 4 as an input device, a display device 5 as an output device, an auxiliary storage device 6 used as a working memory, and an analog / digital converter 7 are connected to the bus 2. I have.

An audio signal input device 8 including, for example, a microphone is connected to the analog / digital converter. The acoustic signal input device 8 captures an acoustic signal such as singing or humming generated by a user or a musical tone generated from a musical instrument and converts the signal into an electric signal, and converts the electric signal into an analog / digital converter. 7 is output.

When a process is instructed by the keyboard input device 4, the CPU 1 starts the transcription process, executes a program stored in the main storage device 3, and converts the sound converted into a digital signal by the analog / digital converter 7. The signals are temporarily stored in the auxiliary storage device 6, and thereafter, these sound signals are converted into musical score data by executing the above-described program and output to the display device 5 as necessary.

Next, the transcription process performed after the CPU 1 captures the audio signal will be described in detail with reference to the flowchart shown in FIG.

First, the CPU 1 performs an autocorrelation analysis of the acoustic signal to extract pitch information of the acoustic signal at each analysis cycle, and also performs a sum-of-squares process to extract power information at each analysis cycle, and then performs noise removal and smoothing processing. And other post-processing (steps SP1, SP
2). Thereafter, for the pitch information, the CPU 1 calculates a shift amount of the pitch axis of the acoustic signal with respect to the absolute pitch axis based on the distribution state, and executes a tuning process of shifting the obtained pitch information according to the shift amount. (Step SP3). That is, the pitch information is corrected so that the difference between the pitch axis and the absolute pitch axis of the singer or musical instrument that has generated the acoustic signal is reduced.

Next, the CPU 1 obtains a continuous period of pitch information in which the obtained pitch information is considered to indicate the same pitch,
A segmentation is performed to divide the acoustic signal into segments for each sound, and a segmentation is performed based on the obtained change in the power information (step SP
4, SP5). Based on these two pieces of segment information obtained, the CPU 1 calculates a reference length corresponding to a time length of a quarter note, an eighth note, etc., and executes the segmentation again based on this reference length (step SP6). .

The CPU 1 identifies the pitch of the segment as a pitch on the absolute pitch axis that can determine that the pitch information is the closest based on the pitch information of the segment thus segmented, and further identifies the pitch of the identified continuous segment. Segmentation is again performed based on whether or not are the same (steps SP7 and SP8).

Thereafter, the CPU 1 obtains a product sum of a frequency of occurrence of a pitch for each segment obtained by summing the pitch information after tuning and a predetermined weighting coefficient determined according to the key, and based on the maximum information of the product sum. For example, the key of the music of the input audio signal is determined, such as C major or A minor, and the predetermined pitch on the scale in the determined key is reviewed based on the pitch information to confirm and correct the pitch. (Steps SP9 and SP10). Next, the CPU 1 executes a review of the segmentation based on whether or not there is the same continuous segment from the finally determined pitch, and whether or not there is a power change between the continuous segments, Final segmentation is performed (step SP11).

Once the pitch and segment are determined in this way,
The CPU 1 extracts measures from the viewpoint that the music starts from the first beat, the last sound of the phrase does not extend to the next measure, and there is a break in each measure, and determines the time signature from the measure information and the segmentation information. The tempo is determined from the determined time signature information and the length of the bar (step SP1).
2, SP13).

Then, the CPU 1 organizes the information on the determined pitch, pitch, key, beat, and tempo to finally create the musical score data (step SP14).

Next, the pitch identification process (see step SP7) in the automatic transcription system will be described in detail with reference to the flowchart of FIG.

CPU1 first extracts the first segment from the segments obtained by the segmentation, and then
The median of all pitch information in the segment is extracted (steps SP20 and SP21). Here, the median value means that when the pitch information of the segment is arranged in order of size,
When the number of data is odd, it is the value of the pitch information at the center, and when the number of data is even, it is the average value of the two pieces of pitch information at the center.

Thereafter, the CPU 1 identifies a pitch on the absolute pitch axis having the closest extracted median as a pitch of the segment (step SP22). Note that the pitch of each segment of the acoustic signal is identified as one of the pitches that differs by a semitone on the absolute pitch axis. The CPU 1 determines whether or not the segment whose pitch has been identified by performing the processing is the last segment (step SP23). As a result, when the process is completed,
If the processing program is terminated and the processing is not terminated, the next segment is set as a processing target and the above-described step 21 is performed.
Return to step SP24.

By repeating such a processing loop consisting of steps SP21 to SP24, pitch identification is executed for all the segments based on the median information of the pitch information in the segments.

Here, the median value is used for the pitch identification processing, although the sound signal has fluctuation, it is considered that the singer fluctuates around the pitch intended, and the median value is set to the intended pitch. This is because it is considered to correspond.

FIG. 2 shows an example of pitch identification by such processing. A dotted curve PIT indicates pitch information of an acoustic signal, and a vertical solid line VP indicates a gap between segments. The median of each segment in this example is the horizontal solid line
HP is indicated, and the identified pitch is indicated by a dotted horizontal line.
Indicated by HP. As is apparent from FIG. 2, the median has a small deviation from the pitch on the absolute pitch axis, and can be identified well. In addition, the pitch information before and after the segment break is unstable (for example, the curved portions C1 and C1).
The pitch can be identified without being affected by C2).

Therefore, according to the above-described embodiment, the median of the pitch information of each segment is extracted and the pitch of the segment is identified as the pitch on the absolute pitch axis whose median is closest, so that the pitch is determined with high accuracy. be able to. Since the acoustic signal is tuned prior to the pitch identification, according to this method, the median value takes a value close to the pitch on the absolute pitch axis, and the identification becomes very easy.

Other Embodiments Note that the pitch information used for the pitch identification processing may be represented by Hz in frequency units, or represented by cent units often used in the music field. It may be something.

In the above-described embodiment, the CPU 1 executes all the processing shown in FIG. 4 according to the program stored in the main storage device 3. However, a part or all of the processing is performed by a hardware configuration. May be executed. For example, as shown in FIG. 5 in which the same reference numerals are given to the corresponding parts in FIG. 3, after the sound signal from the sound signal input device 8 is amplified through the amplifier circuit 10, the pre-filter is further added.
The digital signal is supplied to an analog / digital converter 12 via an analog-to-digital converter 11 and converted into a digital signal. The acoustic signal converted to the digital signal is subjected to autocorrelation analysis by a signal processor 13 to extract pitch information, and a square sum processing is performed. And extract power information
It may be provided to the software processing system by the CPU1. As a signal processor 13 used in such a hardware configuration (10 to 13), a processor capable of processing a signal in a voice band in real time and providing an interface signal with a host CPU 1 (for example, Japan) ΜPD7720 manufactured by Denki Co., Ltd.) can be applied.

[Effects of the Invention] As described above, according to the present invention, the pitch of each segment is identified based on the median value of the pitch information of the segment, so that the pitch can be determined well, and the accuracy of the score data can be determined. Can be further increased.

[Brief description of the drawings]

FIG. 1 is a flowchart showing a pitch identification process according to an embodiment of the present invention, FIG. 2 is a schematic diagram showing an example of the pitch identification process, and FIG. 3 shows a configuration of an automatic transcription system to which the present invention is applied. FIG. 4 is a flowchart showing the automatic transcription process, and FIG. 5 is a block diagram showing another configuration of the automatic transcription system. 1 ... CPU, 3 ... main storage device, 6 ... auxiliary storage device,
7 ... A / D converter, 8 ... Acoustic signal input device.

 ──────────────────────────────────────────────────続 き Continued on the front page (72) Inventor Masaki Fujimoto 5-7-15 Shiba, Minato-ku, Tokyo Inside NEC Technical Information System Development Co., Ltd. (72) Inventor Masanori Mizuno 5-7-1 Shiba, Minato-ku, Tokyo No. 15 Examiner, NEC Technical Information Systems Development Co., Ltd. Shigeo Arai

Claims (2)

(57) [Claims] 1. A process for extracting pitch information representing a pitch and power information of the sound signal, which is a repetition period of an input sound signal waveform, and processing the sound signal based on the pitch information and / or the power information. Automatically segmenting the sound signal into musical score data, including at least a segmentation process for segmenting the sound signal into segments that can be regarded as the same pitch, and a pitch identification process for determining a pitch on the absolute pitch axis of the acoustic signal for the segmented segment. In the music transcription method, the pitch identification processing includes a process of extracting a median of pitch information of each of the divided sections, and determining a pitch of the section to a pitch on an absolute pitch axis closest to the extracted median. Automatic transcription method, comprising: 2. A pitch / power extracting means for extracting pitch information representing a pitch and power information of the audio signal, which is a repetition period of an input audio signal waveform, and based on the pitch information and / or the power information. Segmentation means for classifying the acoustic signal into sections that can be regarded as having the same pitch, and pitch identification means for determining a pitch on the absolute pitch axis of the acoustic signal for the divided section. In an automatic transcription apparatus for converting musical score data, a median extracting unit that extracts a median of pitch information for each of the sections divided by the segmentation means, An automatic transcription apparatus comprising: a pitch on a pitch axis; and a pitch determination unit that determines a pitch in the section.