JPH01219627A - Automatic score taking method and apparatus - Google Patents

Abstract

PURPOSE:To enhance the accuracy of segmentation, by performing segmentation processing for classifying an input acoustic signal into a section regarded as the same sound and finely dividing a section having a length equal to or more than the reference length extracted from the length of the section. CONSTITUTION:The digital acoustic signal such as singing passing through an acoustic signal input apparatus 8 and an A/D converter 8 by a CPU 1 corresponding to the order from a keyboard 4 is stored in the auxiliary memory device 6 of a working memory and the program from a main memory device 3 is executed. The pitch data of the acoustic signal is extracted to perform segmentation processing for dividing the acoustic signal into a section regarding the acoustic signal as the same sound. Subsequently, a reference length is extracted on the basis of the length of the section and the section of the reference length or more is subjected to finely dividing segmentation processing based on the reference length to exclude a section containing two or more sounds caused by the fluctuation of power data and the accuracy of segmentation is enhanced to perform highly accurate score taking processing.

Description

[Detailed Description of the Invention] [Industrial Application Field] The present invention relates to an automatic musical notation method and apparatus for creating musical score data from acoustic signals such as singing voices, humming voices, musical instrument sounds, etc. This relates to segmentation processing that divides into segments that can be considered.

[Prior Art] In an automatic score transcription system that converts acoustic signals such as singing voices, humming voices, and musical instrument sounds into musical score data, the basic information of musical scores such as length, pitch, key, meter, and so on are extracted from the acoustic signals. It has to detect the tempo.

By the way, the acoustic signal is only a signal that continuously includes a repeating waveform of the basic waveform, and the above-mentioned information cannot be obtained immediately.

Therefore, in the conventional automatic score transcription method, first, the repetition information (hereinafter referred to as pitch information) of the basic waveform of the acoustic signal is
and power information are extracted for each analysis cycle, and then the acoustic signal is divided into sections (segments) that can be considered to have the same pitch based on at least the extracted power information (segmentation). The sequence of identifying the pitch of the acoustic signal of a segment as a pitch along the absolute pitch axis, determining the key of the acoustic signal based on the distribution of pitch information, and further determining the time signature and tempo of the acoustic signal based on the segment. I was getting all the information.

Therefore, pitch, time signature, tempo, etc. are determined by segments (note lengths).
The segmentation result is determined based on
This is especially important when creating musical score data.

[Problems to be Solved by the Invention] However, since the power information and/or pitch information, which is information used to perform segmentation, has fluctuations, two or more notes may be combined into one segment.

As mentioned above, segmentation is an important element in creating musical score data, and if the accuracy of segmentation is low, the identified pitches are likely to be incorrect, and the final score data The accuracy will also be significantly lower, so when final segmentation is performed from both the segmentation results based on pitch information and the segmentation results based on power information, or when the final segmentation is performed from power information, the accuracy of the segmentation process itself will also be lowered. It is hoped that this will improve.

The present invention has been made in consideration of the above points, and aims to provide an automatic score transcription method and device that can improve the accuracy of segmentation and further improve the accuracy of final musical score data. It is something.

[Means for solving the problem] In order to solve the problem, in the first invention,
It is the repetition period of the input acoustic signal waveform, and includes processing to extract pitch information representing the pitch and power information of the acoustic signal, and dividing the acoustic signal into sections that can be considered to have the same pitch based on the pitch information and/or power information. a first segmentation process, an interval identification process that identifies the interval on the absolute pitch axis as the interval of the segmented interval based on the pitch information, and an interval identification process that identifies the interval on the absolute interval axis based on the identified interval and power information, and the same note is divided into multiple intervals based on the identified interval and power information. In an automatic music score method for converting an acoustic signal into musical score data, the method includes at least a second segmentation process that detects cases where the continuous sections are divided into one section and connects the continuous sections to form one section. A standard length extraction process that extracts a standard length corresponding to the time length of a predetermined note based on the length of each divided section, and a standard length extraction process that extracts a standard length corresponding to the time length of a predetermined note based on the length of each divided section, and extracts a standard length that is longer than a predetermined length among the divided sections based on the extracted standard length. An interval subdivision process for subdividing something is provided as a preprocessing for the pitch identification process.

Further, in the second aspect of the present invention, there is provided a means for extracting pitch information and power information of the acoustic signal, which is the repetition period of the input acoustic signal waveform and represents the pitch, and an acoustic signal based on the pitch information and/or the power information. a first segmentation means that divides the signal into intervals that can be considered to be the same pitch; a pitch identification means that identifies the interval on the absolute pitch axis as the pitch of the divided interval based on pitch information; and the identified pitch and power. A second segmentation means detects when the same sound is divided into a plurality of sections based on the information and connects the continuous sections to form one section, and converts the acoustic signal into musical score data. In the automatic score transcription apparatus, a reference length extraction means extracts a reference length corresponding to the time length of a predetermined note based on the length of each section divided by the first segmentation means; A section subdivision means for subdividing the sections having a predetermined length or more among the sections divided by the pitch identification means is provided at a stage preceding the pitch identification means.

[Operation] In the first aspect of the present invention, even if the first segmentation process divides the sound into sections that are considered to be the same sound, the pitch information and/or power information that is the source of segmentation has fluctuations. Considering that more than one note may be considered as one section, a standard length corresponding to the time length of a predetermined note is extracted based on the length of each section, and based on that standard length, a period longer than the predetermined length is Sections with length are subdivided. As a result, sections containing two or more tones are excluded. Note that the second segmentation process connects the sections divided into two or more sections even though one note is intended.

Also, in the second invention, focusing on the same point, the reference length is extracted by the reference length extraction means, and the section having a predetermined length or more is subdivided based on the reference length by the subdivision means. I made it.

Embodiment] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings.

First, an automatic score transcription method to which the present invention is applied will be explained.

In FIG. 3, a central processing unit (CPU) 1 controls the entire device, and performs the music transcription process shown in FIG. 4 stored in a main storage device 3 connected via a bus 2. It executes the program. Connected to the bus 2 are a keyboard 4 as an input device, a display device 5 as an output device, an auxiliary storage device 6 used as a working memory, and an analog/digital converter 7, in addition to the CPUI and main storage device 3. There is.

Connected to the analog/digital converter 7 is an acoustic signal input device 8 consisting of, for example, a microphone. This acoustic signal input device 8 captures acoustic signals such as singing or humming vocalized by a user, or musical sounds generated from musical instruments, and converts them into electrical signals, and converts the electrical signals into electrical signals. 7.

When the processing is instructed by the keyboard input device 4, the CPU starts the music transcription processing, executes the program stored in the main storage device 3, and converts the sound converted into a digital signal by the analog/digital converter 7. The signals are temporarily stored in the auxiliary storage device 6, and then the above-mentioned program is executed to convert these acoustic signals into musical score data, which is output to the display device 5 as necessary.

Next, the score transcription process performed by the CPU after capturing the audio signal will be described in detail according to the flowchart shown at the functional level in FIG. 4.

First, the CPU performs autocorrelation analysis on the acoustic signal to extract pitch information of the acoustic signal for each analysis period, performs sum-of-squares processing to extract power information for each analysis period, and then performs noise removal and smoothing processing. Execute post-processing such as (step SPI
, 5P2). After that, the CPU calculates the amount of deviation of the pitch axis of the acoustic signal from the absolute pitch axis based on the distribution status of the pitch information, and executes tuning processing to shift the obtained pitch information according to the amount of deviation. (Step 5P3). That is, the pitch information is corrected so that the difference between the pitch axis of the singer or musical instrument that generated the acoustic signal and the absolute pitch axis becomes small.

Next, the CPU obtains continuous periods of pitch information in which the obtained pitch information is considered to indicate the same pitch, performs segmentation to cut the acoustic signal into segments for each note, and also calculates the obtained power. Segmentation is performed based on changes in information (step SP4.5P5). Based on the obtained segment information, the CPU calculates a reference length corresponding to the time length of a quarter note, an eighth note, etc., and executes segmentation in more detail based on this reference length (step 5
P6).

Based on the pitch information of the segments segmented in this way, the CPUI identifies the pitch of the segment to the pitch on the absolute pitch axis that can be determined to be closest to the pitch information, and further identifies the pitch of the identified continuous segment. The segmentation is reviewed based on whether or not they are the same (step SP7.5P8).

After that, the CPU calculates the product sum of the appearance frequency of the pitch obtained by summing up the pitch information after the tuning process and the predetermined weighting coefficient determined according to the key, and calculates the product sum based on the maximum information of this product sum. For example, the key of the music of the input acoustic signal is determined, such as C major or A minor, and the pitch information is reviewed for a predetermined interval of the scale in the determined key to confirm and correct the interval (step SP9 .5P
10). Then, the CPUI performs a segmentation review based on whether consecutive segments from the finally determined pitch are the same and whether there is a change in power between consecutive segments;
If necessary, connect the segments for final segmentation (step 5PII).

Once the pitch and segment are determined in this way, C
PUI extracts measures from the viewpoints of whether a song starts from the first beat, the last note of a phrase does not span the next measure, or there is a break between each measure, and determines the time signature from this measure information and segmentation information. Then, the tempo is determined from this determined time signature information and measure length (step 5).
P12.5P13).

The CPU then organizes the determined pitch, length, key, time signature, and tempo information and finally creates musical score data (Step 5P14>).

Developing Cementation 1 Next, the segmentation review process (see step SP6) in such an automatic music transcription method will be described in detail using the flowchart in FIG.

Note that such a review process may result in segments being incorrectly
If the segment is segmented to include more than one note, there is a strong possibility that the identified pitch will be incorrect, reducing the accuracy of the score data. Therefore, before the pitch identification process, subdivide the segment in advance and It is provided to execute pitch identification and improve the accuracy of pitch identification processing.

In this case, it is conceivable to divide one note into two or more segments, but this does not pose a problem because segments considered to be one note are connected based on the identified pitch and power information in the segmentation process in Step 5 PII described above. .

In the segmentation review process, the CPU first checks that the segment to be processed is not the last segment, and then matches the length of the segment with the overall segmentation result (
Step 5P20.21).

Here, matching refers to the sum of the absolute values of the differences between an integral fraction or multiple of the length of the segment and the length of another segment, and This means finding the number of times the length of an integer multiple does not match the length of another segment (1,000 mismatches). In this embodiment, the other segments to be matched are both the segment obtained based on the pitch information and the segment obtained based on the power information.

For example, the previous segmentation process (step SP
4.5), when the first segment S1 is divided into 10 segments as shown in FIG. =21
J is obtained, and the number of mismatches is 7.

When the CPUI obtains the number of mismatches and its degree (total difference information) for the segment to be processed in this way, the CPU stores it in the auxiliary storage device 6 and then sets the segment to be processed as the next segment to step 5P20 described above.
Return to step 5P22.

By repeating the processing loop consisting of Steps 5P20 to 22, information on the number of mismatches and its degree can be obtained for all segments, and eventually Step 5P20-22 is repeated.
A positive result is obtained at P20. At this time, the CPU
I determines a reference length based on the segment length with the smallest mismatch count and degree information stored in the auxiliary storage device 6 (step 5P).
24). Here, the reference length refers to a time length corresponding to a quarter note (or eighth note).

In the case of the example shown in FIG. 2, "60" is extracted as the segment length with the smallest number of mismatches and its degree, and r120J, which is twice this length "60", is selected as the reference length. In reality, the length that the time length for a quarter note can take is a value within a predetermined range, and from this point of view, r120J is extracted as the reference length instead of "60".

After extracting the reference length, the CPUI subdivides the segment approximately longer than the reference length after inoculating half of the reference length, and ends the segmentation review process (step 5P25).

In the example of FIG. 2, the fifth segment S5 is subdivided into "61" and "60", and the sixth segment S6 is divided into "63".
and “62”, and the ninth segment S9 is divided into “60” and “62”.
” and “59”, and the tenth segment SIO is subdivided into “58”, “58”, “58”, and “57”.

Therefore, according to the above-described embodiment, even when two or more tones are segmented as one segment, it is possible to subdivide them, and it is possible to accurately perform processes such as pitch identification processing and pitch correction processing. Can be done.

In this case, since post-processing is provided to connect segments considered to be the same sound, one sound will not remain erroneously divided into two or more segments.

In addition, in the above-described embodiment, the reference length is extracted based on the number of mismatches and the degree thereof, but the reference length may be extracted based on the frequency of occurrence of segment lengths.

Further, in the above-described embodiment, the time length corresponding to a quarter note is used as the reference length, but the time length corresponding to an eighth note may be used as the reference length. In this case, instead of subdividing by half the standard length, subdivision is performed by the standard length itself.

Furthermore, although the above-described embodiments have been applied to a method having segmentation based on both pitch information and power information, the present invention is applicable to at least an automatic music transcription method having segmentation processing based on power information. can do.

Furthermore, in the above-described embodiment, all the processes shown in FIG. It may also be executed in the configuration. For example, as shown in FIG. 5, in which parts corresponding to those in FIG. The signal processing processor 13 performs autocorrelation analysis on the acoustic signal converted to the digital signal to extract pitch information, and performs sum-of-squares processing to obtain power. The information may be extracted and provided to a software processing system by the CPU 1. Hardware configuration like this (
As the signal processing processor 13 used in 10 to 13), a processor that can process audio band signals in real time and has an interface signal with the host CPU (for example, NEC Corporation's μP
D 7720) may be applied.

[Effects of the Invention] As described above, according to the present invention, a reference length corresponding to a quarter note or an eighth note is extracted and an already obtained segment is subdivided based on this reference length. , the accuracy of post-processing such as pitch identification processing can be further improved, and the accuracy of final musical score data can be improved.

[Brief explanation of the drawing]

FIG. 1 is a flowchart showing a segmentation review process according to an embodiment of the present invention, FIG. 2 is a route map for explaining the review process, and FIG. 3 is a diagram showing the configuration of an automatic music transcription system to which the present invention is applied. FIG. 4 is a flowchart showing the automatic music transcription processing procedure, and FIG. 5 is a block diagram showing another configuration of the automatic music transcription method. 1... CPU, 3... Main storage device, 6... Auxiliary storage device, 7... Analog/digital converter, 8...
Acoustic signal input device.

Claims (2)

[Claims] (1) Process of extracting pitch information representing the pitch and power information of the acoustic signal, which is the repetition period of the input acoustic signal waveform, and matching the acoustic signal based on the pitch information and/or the power information. A first segmentation process that divides into intervals that can be regarded as intervals; an interval identification process that identifies intervals on the absolute pitch axis as intervals in this divided interval based on the pitch information; and the identified interval and the power information. converting the acoustic signal into musical score data, including at least a second segmentation process that detects when the same sound is divided into a plurality of sections based on the above, and connects the continuous sections to form one section; In the automatic music transcription method, a standard length extraction process that extracts a standard length corresponding to the time length of a predetermined note based on the length of each section divided by the first segmentation process; An automatic musical notation method characterized in that a section subdivision process for subdividing sections having a predetermined length or more out of the sections divided by the interval is provided as a preprocessing of the pitch identification process. (2) means for extracting pitch information representing the pitch and power information of the acoustic signal, which is a repetition period of the input acoustic signal waveform; and a means for extracting the pitch information indicating the pitch and the power information of the acoustic signal; a first segmentation means for dividing into intervals that can be regarded as intervals; an interval identification means for identifying intervals on the absolute interval axis as intervals in the divided intervals based on the pitch information; and information on the identified intervals and the power. and a second segmentation means that detects when the same sound is divided into a plurality of sections based on the above-mentioned section and connects the continuous sections to form one section, and converts the acoustic signal into musical score data. In the automatic score transcription apparatus for converting, a reference length extraction means extracts a reference length corresponding to the time length of a predetermined note based on the length of each section divided by the first segmentation means; An automatic score transcription apparatus characterized in that a section subdivision means for subdividing sections having a predetermined length or more among the sections divided based on a reference length is provided at a stage upstream of the pitch identification means.