JP2604409B2 - Automatic music transcription method and device - Google Patents

Description

Description: BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an automatic music transcription method and apparatus for creating musical score data from acoustic signals such as singing voices, humming voices, and instrumental sounds. This is related to a key determination process to be determined.

[Prior Art] In an automatic transcription system for converting an acoustic signal such as a singing voice, a humming voice, or a musical instrument sound into musical score data, a sound length, a pitch, a key, a time signature, and the like, which are basic information as a musical score from an acoustic signal. Detecting the tempo.

By the way, an acoustic signal is only a signal that continuously includes a repetitive waveform of a basic waveform, and the above-described information cannot be obtained immediately.

Therefore, in the conventional automatic transcription method, first, repetition information (hereinafter, referred to as pitch information) of a basic waveform representing a pitch of an acoustic signal and power information are extracted for each analysis cycle, and thereafter, the extracted pitch information is extracted. And / or dividing the audio signal into sections (segments) that can be regarded as the same pitch based on the power information (this processing is called segmentation), and then, based on the pitch information of the segment, the pitch along the absolute pitch axis as the pitch of the audio signal of each segment. Is identified, the key of the sound signal is determined from the above-described pitch information, and further, the time signature and the tempo of the sound signal are determined based on the segment.

[Problems to be Solved by the Invention] By the way, even if an attempt is made to identify a certain segment of an acoustic signal as a pitch on an absolute pitch axis, the pitch of an acoustic signal, particularly an acoustic signal uttered by a human, is not stable. Even if a sound is intended, the pitch fluctuates frequently. For this reason, the pitch identification processing is very difficult.

Musical score data is recorded information for ensuring the reproducibility of acoustic signals, and from the viewpoint of reproducibility, pitch is a fundamental element of musical score data together with pitch, and it is necessary to identify it accurately. If it cannot be identified correctly, the accuracy of the score data is deemed to be low.

By the way, the key of the sound signal is not only an element of the musical score data but also has a certain relationship with the musical interval, and provides an important clue for determining the musical interval. Therefore, it is desirable to determine the key and review the identified pitch in order to improve the accuracy of the identified pitch, and it is desirable to determine the key of the acoustic signal well.

The present invention has been made in consideration of the above points, and provides an automatic music transcription method and apparatus that can accurately determine the tone of an acoustic signal and can further improve the accuracy of final score data. What you want to do.

[Means for Solving the Problems] In order to solve the problems, in the first aspect of the present invention, pitch information representing a pitch and a power information of an acoustic signal, which is a repetition period of an input acoustic signal waveform, is extracted. And a segmentation process of dividing the acoustic signal into sections that can be regarded as having the same pitch based on the pitch information and / or the power information, and a pitch identification process of determining a pitch on the absolute pitch axis of the acoustic signal for the divided section. And at least a key determination process of determining a key of the sound signal based on the pitch information, wherein the key determination process converts the pitch information to each of the absolute pitch axes on the absolute pitch axis. A process of extracting the frequency of occurrence of the pitch of the sound signal by dividing and summing up the pitch, and a predetermined weight for each scale in the key Calculating the sum of the products of the pitch coefficient and the scale occurrence frequency of the pitch of the extracted audio signal for all the keys; and determining the key having the largest sum of the products of the calculated sum of the products as the key of the audio signal. Processing.

Further, in the second aspect of the present invention, pitch / power extraction means for extracting pitch information representing a pitch and power information of the audio signal, which is a repetition period of the input audio signal waveform, comprises pitch information and / or the power Segmentation means for dividing the sound signal into sections that can be regarded as having the same pitch based on the information; pitch identification means for determining the pitch on the absolute pitch axis of the sound signal for the divided section; In an automatic transcription apparatus which partially converts a sound signal into musical score data by providing a key determining means for determining a key to have, the key determining means sorts pitch information into respective pitches on an absolute pitch axis and totals the pitch information. A scale frequency extraction unit for extracting a scale occurrence frequency of a pitch included in a signal: a weighting coefficient predetermined for each scale in a key and an extracted sound A sum-of-products calculation unit that calculates the sum of products of the pitches of the pitches and the scale occurrence frequency for all tones; It consisted of:

[Operation] In the first aspect of the present invention, a weighting coefficient for each scale is created in advance for each key, focusing on the fact that important scales differ depending on the key, and the scale generation of the pitch of the input acoustic signal is performed. The frequency is extracted based on the pitch information, the product sum of the extracted frequency and the scale weighting coefficient is obtained for each key, and the key having the maximum product sum is determined as the key of the sound signal.

In the second aspect of the present invention, a weighting coefficient for each scale is created in advance for each key based on the same attention, and the scale generation frequency of the pitch of the input acoustic signal is determined based on the pitch information. Extracted by the frequency extraction unit,
The product sum of the extracted frequency and this weighting coefficient is obtained for each key by the product-sum calculation unit, and the key that maximizes the product sum is determined as the key of the sound signal by the key determination unit.

Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings.

Automatic transcription system First, an automatic transcription system to which the present invention is applied will be described.

In FIG. 3, a central processing unit (CPU) 1 controls the whole of the apparatus, and performs a transcription process shown in FIG. 4 stored in a main storage device 3 connected via a bus 2. Execute the program. In addition to the CPU 1 and the main storage device 3, a keyboard 4 as an input device, a display device 5 as an output device, an auxiliary storage device 6 used as a working memory, and an analog / digital converter 7 are connected to the bus 2. I have.

The analog / digital converter 7 is connected to an acoustic signal input device 8 composed of, for example, a microfan. The acoustic signal input device 8 captures an acoustic signal such as singing or humming uttered by a user or a musical tone generated from a musical instrument and converts the signal into an electric signal, and converts the electric signal into an analog / digital converter. 7 is output.

When a process is instructed by the keyboard input device 4, the CPU 1 starts the transcription process, executes a program stored in the main storage device 3, and converts the sound converted into a digital signal by the analog / digital converter 7. The signals are temporarily stored in the auxiliary storage device 6, and thereafter, these sound signals are converted into musical score data by executing the above-described program and output to the display device 5 as necessary.

Next, the transcription process performed by the CPU 1 after capturing the audio signal will be described in detail with reference to the flowchart shown in FIG.

First, the CPU 1 performs an autocorrelation analysis of the acoustic signal to extract pitch information of the acoustic signal at each analysis cycle, and also performs a sum-of-squares process to extract power information at each analysis cycle, and then performs noise removal and smoothing processing. And other post-processing (steps SP1, SP
2). Thereafter, for pitch information, the CPU 1 calculates a shift amount of a pitch axis of the acoustic signal with respect to the absolute pitch axis based on the distribution state of the pitch information based on the distribution information of the pitch information, and calculates the obtained pitch information as the shift amount. (Step SP3). That is, the pitch information is corrected so that the difference between the pitch axis and the absolute pitch axis of the singer or musical instrument that has generated the acoustic signal is reduced.

Next, the CPU 1 obtains a continuous period of pitch information in which the obtained pitch information is considered to indicate the same pitch,
A segmentation is performed to divide the acoustic signal into segments for each sound, and a segmentation is performed based on the obtained change in the power information (step SP
4, SP5). Based on these two pieces of segment information obtained, the CPU 1 calculates a reference length corresponding to a time length of a quarter note, an eighth note, etc., and executes the segmentation again based on this reference length (step SP6). .

Based on the pitch information of the segment thus segmented, the CPU 1 identifies the pitch of the segment as a pitch on the absolute pitch axis that can be determined that the pitch information is the closest, and further, determines the pitch of the identified continuous segment. The segmentation is performed again based on whether or not the pitches are the same (steps SP7 and SP8).

After that, the CPU 1 determines, based on the pitch information distribution information,
For example, the key of the music of the input sound signal is determined such as C major or A minor, and the predetermined pitch on the scale in the determined key is reviewed in more detail based on the pitch information to confirm the pitch, Modify (steps SP9, SP1
0). Next, the CPU 1 executes a review of the segmentation based on whether or not there is the same continuous segment from the finally determined pitch, and whether or not there is a power change between the continuous segments, Final segmentation is performed (step SP11).

Once the pitch and segment are determined in this way,
The CPU 1 extracts measures from the viewpoint that the music starts from the first beat, the last sound of the phrase does not extend to the next measure, and there is a break in each measure, and determines the time signature from the measure information and the segmentation information. The tempo is determined from the determined time signature information and the length of the bar (step SP1).
2, SP13).

Then, the CPU 1 organizes the information on the determined pitch, pitch, key, beat, and tempo to finally create the musical score data (step SP14).

Processing for Determining the Key of an Acoustic Signal Next, the processing for determining the key (step SP9) of an audio signal in such an automatic transcription system will be described in detail with reference to the flowchart of FIG.

The CPU 1 creates a scale histogram from all pieces of pitch information tuned by the above-described tuning process (step SP20). Here, the musical scale histogram is “ha (do)”, “sharp ha: strange ni (do #: leb)”,
Histograms of the scales on the 12 absolute pitch axes of "ni (re)", ..., "i (la)", "sharp a: strange ro (la #: shi b)", and "ro (shi)" When the pitch information is not on the absolute pitch axis, the pitch information is distributed to the two scales on the absolute pitch axis that are close to each other in proportion to the distance between them, and totaled. Therefore, intervals that differ by one octave are counted as the same scale.

Next, the CPU 1 performs twelve majors of "C major", "D major major", "D major", ..., "B major major", "B major",
Fig. 2 Determined by all the tones in a total of 24 tones, including 12 minors of "A minor", "B minor", "B minor", ..., "G minor", "G minor A" Then, the product sum of the weighting coefficient for each scale and the total value of the above-described scale histogram is obtained (step SP21).

FIG. 2 shows the weighting coefficient of “C major” in the first column CO
In L1, the weighting coefficient of "A minor" is in the second column COL2, and in the third column COL3 is the weighting coefficient of "in D major"
In the fourth column, COL4 is used as an example. Similarly, for other keys, a weighting coefficient of "202021020201" is used for the major key from "do" and for a minor key, the weighting coefficient is " 202201022010 ".

Here, the weighting coefficient gives a weight other than “0” to the scale that can be expressed without the accidentals (#, b) in the key, and combines the seven scales of the major and minor scales and the pentatone scale, That is, when the main tones are matched in major and minor, "2" is used for the scale in which the pitch difference from the main tone matches, and "1" is used for the scale in which the pitch difference does not match. Such weighting coefficients correspond to the importance of each scale in the key.

When the sum of products is obtained for all 24 tones in this way, the CPU 1 determines the tone with the largest sum of products as the tone of the audio signal, and ends the tone determination process (step SP22). ).

Therefore, according to the above-described embodiment, a scale histogram is created, the frequency of occurrence of each pitch as a scale is captured, and the product of the frequency and a weighting coefficient as an importance parameter of the scale determined according to the key is obtained. Since the key of the sum of the products is determined as the key of the sound signal, the key can be determined accurately, the pitch identified by this key can be reviewed, and the accuracy of the score data can be improved. It can be further improved.

Other Embodiments Note that the pitch information used in the key determination process may be expressed in Hz in frequency units, or in cent units often used in the music field. It may be something.

Further, the weighting coefficients are not limited to those of the above-described embodiment, and for example, a larger weight may be given to the tonic.

Further, in the above-described embodiment, the CPU 1 executes all the processing shown in FIG. 4 according to the program stored in the main storage device 3. However, a part or all of the processing is performed by a hardware configuration. May be executed. For example, as shown in FIG. 5 in which the same reference numerals are given to the corresponding parts in FIG. 3, after the sound signal from the sound signal input device 8 is amplified through the amplifier circuit 10, the pre-filter is further added.
The digital signal is supplied to an analog / digital converter 12 via an analog-to-digital converter 11 and converted into a digital signal. The acoustic signal converted to the digital signal is subjected to autocorrelation analysis by a signal processor 13 to extract pitch information, and a square sum processing is performed. And extract power information
It may be provided to the software processing system by the CPU1. As the signal processor 13 used in such a hardware configuration (10 to 13), a processor (for example, a processor that can process a signal in an audio band in real time and is provided with an interface signal with the host CPU 1)
NEC Corporation μPD7720) can be applied.

[Effects of the Invention] As described above, according to the present invention, the product sum of the frequency of occurrence of the pitch of an acoustic signal as a scale and the weighting coefficient indicating the importance of each scale created in advance for each key is determined. Since the calculated key of the sum of products is set as the key of the sound signal, the key can be determined well, and the accuracy of the final score data can be further improved.

[Brief description of the drawings]

FIG. 1 is a flowchart showing a key determination process according to an embodiment of the present invention, FIG. 2 is a table showing an example of a weighting coefficient of each scale determined according to each key, and FIG. Is a block diagram showing a configuration of an automatic transcription system to which is applied, FIG. 4 is a flowchart showing an automatic transcription process procedure, and FIG. 5 is a block diagram showing another configuration of the automatic transcription system. 1 ... CPU, 3 ... main storage device, 6 ... auxiliary storage device,
7 ... A / D converter, 8 ... Acoustic signal input device.

 ──────────────────────────────────────────────────続 き Continued on the front page (72) Inventor Masaki Fujimoto 5-7-15 Shiba, Minato-ku, Tokyo Inside NEC Technical Information System Development Co., Ltd. (72) Inventor Masanori Mizuno 5-7-1 Shiba, Minato-ku, Tokyo No. 15 Examiner, NEC Technical Information Systems Development Co., Ltd. Shigeo Arai

Claims (2)

(57) [Claims] 1. A process for extracting pitch information representing a pitch and power information of the sound signal, which is a repetition period of an input sound signal waveform, and processing the sound signal based on the pitch information and / or the power information. Is divided into sections that can be regarded as the same pitch, a pitch identification process that determines a pitch on the absolute pitch axis of the acoustic signal for the divided section, and a key that the acoustic signal has based on the pitch information. An automatic music transcription method for converting the audio signal into musical score data, wherein the key determination process allocates the pitch information to each pitch on an absolute pitch axis and totals the audio signal. Extracting the scale occurrence frequency of the pitches of the key, and extracting the predetermined weighting coefficient and the extracted weighting coefficient for each scale in the key. Processing of calculating the sum of products of the scale occurrence frequency of the reverberation signal for all tones, and determining the key having the largest sum of products of the calculated sum of products as the key of the acoustic signal. Characteristic automatic transcription method. 2. A pitch / power extracting means for extracting pitch information representing a pitch and power information of the audio signal, which is a repetition period of an input audio signal waveform, and based on the pitch information and / or the power information. Segmenting means for dividing the acoustic signal into sections that can be regarded as having the same pitch, pitch identifying means for determining a pitch on the absolute pitch axis of the acoustic signal for the divided section, and the acoustic signal based on the pitch information. An automatic transcription apparatus for converting the acoustic signal into musical score data, which is partially provided with a key determining means for determining the key of the musical note data, wherein the key determining means distributes the pitch information to each pitch on an absolute pitch axis. A scale frequency extraction unit for totalizing and extracting a scale occurrence frequency of a pitch included in the sound signal, and a predetermined weight for each scale in the key Sum-of-products calculation unit for calculating the sum of products of the scale generation frequency of the extracted sound signal and the scale occurrence frequency of the extracted sound signal, and the key having the largest sum of products of the calculated sum of products is calculated as the sound signal of the sound signal. An automatic transcription apparatus comprising: a key determination unit that determines a key.