JP2604401B2 - Automatic music transcription method and device - Google Patents

Description

Description: BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an automatic music notation method and apparatus for creating musical instrument data from acoustic signals such as singing voices, humming voices, and musical instrument sounds. This relates to a tuning process for matching an absolute pitch axis.

[Prior Art] An acoustic signal such as a singing voice, a humming voice, or a musical instrument sound has a repeating waveform of a basic waveform. In an automatic music transcription device that converts such an acoustic signal into musical score data, first, a repetition frequency of a basic waveform in the acoustic signal (hereinafter, referred to as “repetition frequency”)
Is called a pitch frequency, a cycle corresponding to the pitch frequency is called a pitch cycle, and a concept of combining them is called a pitch) is extracted for each analysis cycle, and then, based on this pitch frequency,
A section (segment) considered to be the same pitch, a pitch in the segment, and the like are determined.

[Problems to be Solved by the Invention] By the way, the pitch of an acoustic signal, particularly a singing voice or a humming pitch uttered by a person, is different from the absolute pitch. Therefore, even if an attempt is made to identify the pitch on the absolute pitch axis based on the pitch information obtained from the acoustic signal, it may not be possible to properly identify the pitch due to the deviation. The greater the deviation, the more inaccurate the pitch identified on the pitch axis, and the lower the accuracy of the finally created musical score data.

The present invention has been made in consideration of the above points, and detects a deviation between a pitch axis and an absolute pitch axis of an acoustic signal, corrects pitch information according to the deviation amount, and improves the following processing. It is an object of the present invention to provide an automatic transcription method and apparatus which allows a user to create musical score data.

[Means for Solving the Problems] In order to solve the problems, according to a first aspect of the present invention, an analog / digital conversion means for converting an input audio signal into a digital signal, and a predetermined processing procedure are stored. Storage means, comprising a control means for executing the processing procedure stored in the storage means, to extract the pitch information and power information from the acoustic signal converted to a digital signal,
Based on the extracted pitch information and / or power information, the audio signal is divided into sections that can be regarded as having the same pitch, the pitch along the absolute pitch axis of the audio signal in each section is determined from the pitch information, and the pitch of the audio signal is determined based on the pitch information. Determine the key, determine the time signature and tempo of the audio signal based on the section information, and in the automatic transcription method that converts the audio signal into musical score data, the extracted pitch information is aggregated to determine the pitch information around the absolute pitch axis. Detects the distribution, detects the deviation between the pitch axis of the acoustic signal and the absolute pitch axis based on the detected pitch distribution information, and calculates the difference between the pitch information extracted according to the detected deviation and the absolute pitch axis. Moved and corrected to minimize.

Further, in the second aspect of the present invention, analog / digital conversion means for converting an input audio signal into a digital signal, and pitch / power extraction for extracting pitch information and power information from the audio signal converted into a digital signal Means, segmentation means for dividing the audio signal into sections that can be regarded as having the same pitch from the extracted pitch information and / or power information, and pitch identification means for determining a pitch along the absolute pitch axis of the audio signal in each section from the pitch information And a key determining means for determining the key of the audio signal based on the pitch information;
A time signature / tempo determination means for determining a time signature and a tempo of the audio signal based on the section information; and an automatic transcription apparatus for converting the audio signal into musical score data. Pitch distribution detecting means for detecting a distribution of pitch information, pitch deviation detecting means for detecting a deviation between a pitch axis and an absolute pitch axis of an acoustic signal based on the detected pitch distribution information, and Pitch information correcting means for moving and correcting the extracted pitch information so that the difference from the absolute pitch axis is minimized.

[Operation] In the first aspect of the present invention, in order to correct the difference between the pitch axis and the absolute pitch axis of the sound signal and proceed to the process of determining a section and determining a pitch, first, a pitch obtained from the sound signal is used. The distribution around the absolute pitch axis of the information is detected to process the pitch axis information of the audio signal so as to be easily captured, and the distribution is statistically processed to detect the deviation of the pitch of the audio signal from the absolute pitch axis. A process for moving and correcting the pitch information extracted according to the deviation so as to minimize the difference from the absolute pitch axis is provided to improve the accuracy of the finally obtained score data.

Further, in the second aspect of the present invention, the pitch information extracted by the pitch power extracting means is totalized by the pitch distribution detecting means to detect the distribution of pitch information around the absolute pitch axis, and the detected pitch distribution information is added to the detected pitch distribution information. The pitch deviation of the acoustic signal and the absolute pitch axis are detected by the pitch deviation detecting means based on the pitch information extracted by the pitch information correcting means in accordance with the detected deviation. By correcting the movement so as to match the pitch axis of the acoustic signal with the absolute pitch axis, the accuracy of the finally obtained musical score data is improved.

Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings.

Automatic transcription system First, an automatic transcription system to which the present invention is applied will be described.

In FIG. 3, a central processing unit (CPU) 1 controls the whole of the apparatus, and performs a musical notation processing shown in FIG. 4 stored in a main storage device 3 connected via a bus 2. Execute the program. In addition to the CPU 1 and the main storage device 3, a keyboard 4 as an input device, a display device 5 as an output device, an auxiliary storage device 6 used as a working memory, and an analog / digital converter 7 are connected to the bus 2. I have.

An audio signal input device 8 including, for example, a microphone is connected to the analog / digital converter 7. The acoustic signal input device 8 captures an acoustic signal such as singing or humming uttered by a user or a musical tone generated from a musical instrument and converts the signal into an electric signal, and converts the electric signal into an analog / digital converter. 7 is output.

When a process is instructed by the keyboard input device 4, the CPU 1 starts the transcription process, executes a program stored in the main storage device 3, and converts the sound converted into a digital signal by the analog / digital converter 7. The signals are temporarily stored in the auxiliary storage device 6, and thereafter, these sound signals are converted into musical score data by executing the above-described program and output to the display device 5 as necessary.

Next, the transcription process performed by the CPU 1 after capturing the audio signal will be described in detail with reference to the flowchart shown in FIG.

First, the CPU 1 performs an autocorrelation analysis of the audio signal, extracts pitch information of the audio signal in each analysis cycle, and extracts power information by performing a square sum process, and then performs post-processing such as noise removal and smoothing. (Steps SP1 and SP2). afterwards,
For the pitch information, the CPU 1 calculates a shift amount of the acoustic signal with respect to the absolute pitch axis, and executes a tuning process of shifting the obtained pitch information according to the shift amount (step SP3). That is, the pitch information is corrected so that the difference between the pitch axis of the singer or the musical instrument that generated the acoustic signal and the absolute pitch axis becomes smaller.

Next, the CPU 1 obtains a continuous period of pitch information in which the obtained pitch information is considered to indicate the same pitch,
A process of dividing the acoustic signal into segments for each sound (hereinafter, referred to as segmentation) is performed, and a segmentation is performed based on the obtained change in power information (steps SP4 and SP5). On the basis of these two pieces of segment information obtained, the CPU 1 calculates a reference length corresponding to a time length of a quarter note, an eighth note, and the like, and performs more detailed segmentation based on the reference length (step SP6).

Based on the pitch information of the segment thus segmented, the CPU 1 identifies the pitch of the segment as a pitch on the absolute pitch axis that can be determined that the pitch information is the closest, and further, determines the pitch of the identified continuous segment. The segmentation is performed again based on whether or not the pitches are the same (steps SP7 and SP8).

Thereafter, the CPU 1 obtains, for example, the C major and the C major based on the maximum information of the product sum obtained by calculating the product sum of the pitch appearance frequency obtained by totaling the pitch information and a predetermined weighting coefficient determined according to the key. The key of the musical composition of the input sound signal is determined, such as a minor key, and the pitch of the predetermined pitch on the scale in the determined key is reviewed with respect to the pitch information to check and correct the pitch (steps SP9 and SP10). Next, the CPU 1 executes a review of the segmentation based on whether or not there is the same continuous segment from the finally determined pitch, and whether or not there is a power change between the continuous segments, Final segmentation is performed (step SP11).

When the pitch and duration (segment) are determined in this manner, the CPU 1 determines that the music starts from the first beat, the last sound of the phrase does not extend to the next bar, that there is a break in each bar, etc. Measures are extracted from the viewpoint, the beat is determined from the measure information and the segmentation information, and the tempo is determined from the determined beat information and the length of the measure (steps SP12 and SP13).

Then, the CPU 1 organizes the information on the determined pitch, pitch, key, beat, and tempo to finally create the musical score data (step SP14).

Tuning Next, the tuning process in the automatic transcription system that performs transcription by performing such processing (see step SP3)
Will be described in detail with reference to the detailed flowchart of FIG.

The CPU 1 first expresses input pitch information expressed in Hz which is a unit of frequency, in cents which is a unit of scale (the frequency ratio of a certain interval to a reference interval is expressed by a logarithm having a base of 2, 1200 Is converted to pitch data represented by (multiplication of...) (Step SP20). Note that a difference of 100 cents corresponds to a pitch difference of a semitone.

Thereafter, the CPU 1 creates a histogram as shown in FIG. 2 by summing up each pitch data in which the last two digits of the cent value are the same (step SP21). That is, the cent value is 0, 10
Data of 0, 200,... Are counted as the same data, data of cent value of 1, 101, 201,... Are counted as the same data, and data of cent value of 2, 102, 202,. Tally, and the tally is 99, 199, 299,
… For the data of the group of. Thus,
As shown in FIG. 2, a histogram is obtained for pitch information having a total width of 100 cents which differs by 1 cent.

Note that the different pitch information, which is 100 cents different as the same, is different by an integral multiple of a semitone, and since the sound signal uses a semitone and a whole tone as a reference for the pitch difference, the obtained histogram is uniform. Instead of exhibiting a distribution, the frequency has a peak near the cent value corresponding to the pitch axis of the singer who uttered the acoustic signal or the musical instrument that generated the acoustic signal.

Next, the CPU 1 clears the parameters i and j to 0 and sets the parameter MIN to a sufficiently large value A (step SP22). After that, the CPU 1 calculates a statistical variance VAR centering on i-cent using the obtained histogram information (step SP23). Thereafter, it is determined whether or not the calculated variance VAR is larger than the parameter MIN. If it is smaller, the parameter MIN is updated to the variance VAR, the parameter i is updated to the value i, and the process proceeds to step SP26. If it is larger, the process immediately proceeds to step SP26 without performing the update operation (steps SP24 to SP26). Thereafter, the CPU 1 determines whether or not the parameter i is 99, and if different, increments the parameter i and returns to step SP23 described above (step SP27).

In this way, cent information (j) with the smallest variance is obtained from the obtained total information of pitch information. Here, since the variance around the cent information is the smallest, the cent group (j, 100 + j, 2
00 + j,...). In other words, it can be understood that it represents the pitch axis of the singer or musical instrument.

Therefore, the CPU 1 shifts the pitch axis of the acoustic signal to the absolute pitch axis by shifting by the cent information. First, CPU1
Determine whether parameter j is less than 50 cents,
That is, it is determined which of the higher pitch and the lower pitch is closer to the absolute pitch axis. If the pitch is closer to the higher pitch side, all pitch information is shifted to the higher pitch side by the obtained cent j and corrected to the lower pitch side. In this case, all pitch information is corrected by shifting it to the lower tone side by the obtained cent j (step SP28).
~ 30).

Thus, the pitch axis of the acoustic signal is substantially aligned with the absolute pitch axis, and such pitch information is used for the subsequent processing.

Therefore, according to the above-described embodiment, the obtained pitch information is not directly applied to the segmentation, the pitch identification processing, and the like, but the pitch information for each semitone is totaled coaxially, and the variance is used as a parameter based on the total information. The deviation from the absolute pitch axis is detected, and the pitch axis of the audio signal is corrected by that deviation for use in subsequent processing. You can get things.

In the above-described embodiment, the tuning processing is performed on the pitch information obtained by the autocorrelation analysis. However, the method of extracting the pitch information is not limited to this.

In the above-described embodiment, the pitch axis of the acoustic signal is obtained by using the variance. However, the pitch axis may be detected by using another statistical method.

Further, in the above-described embodiment, the pitch information statistically processed in the tuning process is shown in cent units, but it goes without saying that the unit system is not limited to this.

Further, in the above-described embodiment, the CPU 1 executes all the processes shown in FIG. 4 according to the program stored in the main storage device 3. However, some or all of the processes are executed by hardware. The configuration may be executed. For example, as shown in FIG. 5 in which the same reference numerals are given to the corresponding parts in FIG. 3, the sound signal from the sound signal input device 8 is amplified via the amplifier circuit 10 and then further passed through the pre-filter 11. The digital signal is converted to a digital signal by an analog / digital converter 12, and the acoustic signal converted to the digital signal is subjected to autocorrelation analysis by a signal processor 13 to extract pitch information, and is processed by sum-of-squares processing to obtain power. Information may be extracted and provided to the software processing system by the CPU 1. As the signal processor 13 used in such a hardware configuration (10 to 13), a processor (for example, a processor that can process a signal in a voice band in real time and is provided with an interface signal with a host CPU 1)
NEC Corporation μPD7720) can be applied.

[Effects of the Invention] As described above, according to the present invention, the pitch information is corrected by the difference between the pitch axis of the acoustic signal and the absolute pitch axis, and the corrected pitch information is used for the subsequent processing. Is more accurate as the musical score data.

[Brief description of the drawings]

FIG. 1 is a flowchart showing a tuning process according to an embodiment of the present invention, FIG. 2 is a histogram showing a distribution state of pitch information, FIG. 3 is a block diagram showing a configuration of an automatic transcription system to which the present invention is applied, FIG. 4 is a flowchart showing a processing procedure of the automatic transcription system, and FIG. 5 is a block diagram showing another configuration of the automatic transcription system. 1 ... CPU, 3 ... main storage device, 6 ... auxiliary storage device,
7 ... A / D converter, 8 ... Acoustic signal input device.

 ──────────────────────────────────────────────────続 き Continuing on the front page (72) Inventor Masaki Fujimoto 5-7-15 Shiba, Minato-ku, Tokyo Inside NEC Technical Information System Development Co., Ltd. (72) Inventor Masanori Mizuno 5-7-1 Shiba, Minato-ku, Tokyo No. 15 Examiner, NEC Technical Information Systems Development Co., Ltd. Shigeo Arai

Claims (2)

(57) [Claims] An analog / digital converter for converting an input audio signal into a digital signal, a storage for storing a predetermined processing procedure, and a control for executing the processing procedure stored in the storage. Means for extracting pitch information and power information from the audio signal converted to a digital signal, dividing the audio signal from the extracted pitch information and / or power information into sections that can be regarded as having the same pitch, Determine the pitch along the absolute pitch axis of the sound signal of each section from the information, determine the key of the sound signal based on the pitch information, and determine the beat and tempo of the sound signal based on the section information. In the automatic music transcription method for determining and converting the acoustic signal into musical score data, the extracted pitch information is summed up and the distribution of pitch information around the absolute pitch axis is detected. Out, detecting a deviation between the pitch axis and the absolute pitch axis of the acoustic signal based on the detected pitch distribution information, and extracting the pitch information extracted according to the detected deviation from the absolute pitch axis with a minimum difference. An automatic transcription method, wherein the movement is corrected so that 2. An analog / digital converter for converting an input audio signal into a digital signal, a pitch / power extractor for extracting pitch information and power information from the audio signal converted into a digital signal, and extraction. Segmentation means for dividing the acoustic signal into sections that can be regarded as having the same pitch based on the obtained pitch information and / or power information, and pitch identification means for determining a pitch along the absolute pitch axis of the acoustic signal in each section from the pitch information And key determination means for determining the key of the audio signal based on the pitch information, and time signature and tempo determination means for determining the time signature and tempo of the audio signal based on the section information,
An automatic music transcription device for converting the acoustic signal into musical score data; a pitch distribution detecting means for counting the extracted pitch information and detecting a distribution of pitch information around an absolute pitch axis; Pitch shift detecting means for detecting a shift between the pitch axis and the absolute pitch axis of the acoustic signal, and moving the pitch information extracted according to the detected shift so that the difference between the absolute pitch axis is minimized. An automatic music transcription device comprising a pitch information correction means for correcting.