JP2604411B2 - Automatic music transcription method and device - Google Patents

Abstract

PURPOSE:To prevent that segmentation is performed by error by a noise component, by dividing a section wherein power data is larger than a threshold value into small sections and finely dividing the large section at the rising change point of power data. CONSTITUTION:A CPU 1 divides a section wherein the power data of an acoustic signal at an analytical point is below a threshold value and a section wherein said power data is the threshold value or more. Next, the CPU 1 applies start and finish marks to the first and last of the section equal to or more than the threshold value and subsequently calculates the change function of power in the section wherein power data is the threshold value or more and then extracts the rising changing point of power on the basis of the calculated change function to finely divide the section equal to or more than the threshold value. By this method, it can be prevented that segmentation is performed erroneously by a noise component and processing on and after such as intervale identification processing can be well executed.

Description

Description: BACKGROUND OF THE INVENTION The present invention relates to an automatic music transcription method and apparatus for creating musical score data from acoustic signals such as singing voices, humming voices, and instrument sounds, and more particularly, to a method and apparatus based on power information. The present invention relates to a segmentation process for dividing an audio signal into sections that can be regarded as having the same pitch.

[Prior Art] In an automatic transcription system for converting an acoustic signal such as a singing voice, a humming voice, or a musical instrument sound into musical score data, a sound length, a pitch, a key, a time signature, and the like, which are basic information as a musical score from an acoustic signal. Detecting the tempo.

By the way, an acoustic signal is only a signal that continuously includes a repetitive waveform of a basic waveform, and the above-described information cannot be obtained immediately.

Therefore, in the conventional automatic transcription method, first, repetition information (hereinafter, referred to as pitch information) of a basic waveform representing a pitch of an acoustic signal and power information are extracted for each analysis period, and then at least the extracted power is extracted. Based on the information, the audio signal is divided into sections (segments) that can be regarded as having the same pitch (this processing is called segmentation). Then, from the pitch information of the segment, the pitch of the audio signal of each segment is identified as a pitch along the absolute pitch axis. Each information is obtained in the order of determining the tone of the acoustic signal based on the distribution of the pitch information around the pitch axis, and further determining the beat and tempo of the acoustic signal based on the segment.

Therefore, the interval, time signature, tempo, and the like are determined based on the segment (tone length), and thus the segmentation processing is particularly important in creating musical score data.

[Problems to be Solved by the Invention] As described above, the segmentation is an important element in creating the score data, and if the accuracy of the segmentation is low, the accuracy of the finally obtained score data also becomes extremely low. The accuracy of the segmentation process itself from the power information should be improved both when performing the final segmentation from both the segmentation result based on the pitch information and the segmentation result based on the power information, or when performing the final segmentation based on the power information. Is desired.

The present invention has been made in view of the above points, and an object of the present invention is to provide an automatic transcription method and apparatus that can perform segmentation based on power information satisfactorily and improve the accuracy of musical score data. Is what you do.

[Means for Solving the Problem] In order to solve the problem, according to the first aspect of the present invention, a process of extracting power information from an input audio signal is the same as a process of extracting an audio signal based on the extracted power information. At least including a segmentation process for segmenting into intervals that can be regarded as intervals, in an automatic transcription method for converting an acoustic signal into musical score data, the segmentation process includes:
A process of dividing power information into a valid section having a predetermined value or more and an invalid section having a power value of not more than a predetermined value; processing of extracting a rising change point of power information for the valid section; The processing is performed by a series of processing including the subdivision processing.

Further, in the second aspect of the present invention, the power extraction means for extracting power information from the input audio signal and the segmentation means for dividing the audio signal into sections which can be regarded as having the same pitch based on the extracted power information are provided. And a division processing unit for dividing the segmentation means into an effective section in which power information is equal to or greater than a predetermined value and an invalid section in which power information is equal to or less than a predetermined value. A change point extracting unit that extracts a rising change point of information and a subdivision unit that subdivides the above-described effective section with the extracted rising change point.

[Operation] In the first aspect of the present invention, power information is divided into an effective section having a power value equal to or greater than a predetermined value and an invalid section having a power value equal to or less than the predetermined value so as to reduce the influence of power fluctuation and noise of the audio signal. In addition, since an effective section including two or more sounds may occur, the power rises at the time of sound transition, and the effective rising section is subdivided by extracting a power rising change point.

Also, in the second aspect of the present invention, similarly, the power information is divided into an effective section having a predetermined value or more and an invalid section having a predetermined value or less by the division processing section so as not to be affected by power fluctuation or noise. . In addition, the rising point of the power within the effective section is extracted by the change point extracting section, and the effective section is subdivided by the subdivision section at the rising change point so that a section including two or more sounds does not occur.

Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings.

Automatic transcription system First, an automatic transcription system to which the present invention is applied will be described.

In FIG. 4, a central processing unit (CPU) 1 controls the whole of the apparatus, and performs a transcription process shown in FIG. 5 stored in a main storage device 3 connected via a bus 2. Execute the program. In addition to the CPU 1 and the main storage device 3, a keyboard 4 as an input device, a display device 5 as an output device, an auxiliary storage device 6 used as a working memory, and an analog / digital converter 7 are connected to the bus 2. I have.

An audio signal input device 8 including, for example, a microphone is connected to the analog / digital converter 7. The acoustic signal input device 8 captures an acoustic signal such as singing or humming uttered by a user or a musical tone generated from a musical instrument and converts the signal into an electric signal, and converts the electric signal into an analog / digital converter. 7 is output.

When a process is instructed by the keyboard input device 4, the CPU 1 starts the transcription process, executes a program stored in the main storage device 3, and converts the sound converted into a digital signal by the analog / digital converter 7. The signals are temporarily stored in the auxiliary storage device 6, and thereafter, these sound signals are converted into musical score data by executing the above-described program and output to the display device 5 as necessary.

Next, the music transcription process performed by the CPU 1 after capturing the audio signal will be described in detail with reference to the flowchart shown in the functional level of FIG.

First, the CPU 1 performs an autocorrelation analysis of the acoustic signal to extract pitch information of the acoustic signal at each analysis cycle, and also performs a sum-of-squares process to extract power information at each analysis cycle, and then performs noise removal and smoothing processing. And other post-processing (steps SP1, SP
2). Thereafter, for the pitch information, the CPU 1 calculates a shift amount of the pitch axis of the acoustic signal with respect to the absolute pitch axis based on the distribution around the pitch axis, and converts the obtained pitch information into a shift amount of the pitch axis. A tuning process for shifting according to the position is executed (step SP3). That is, the pitch information is corrected so that the difference between the pitch axis and the absolute pitch axis of the singer or musical instrument that has generated the acoustic signal is reduced.

Next, the CPU 1 obtains a continuous period of pitch information in which the obtained pitch information is considered to indicate the same pitch,
A segmentation is performed to divide the acoustic signal into segments for each sound, and a segmentation is performed based on the obtained change in the power information (step SP
4, SP5). On the basis of these two pieces of segment information obtained, the CPU 1 calculates a reference length corresponding to a time length of a quarter note, an eighth note, and the like, and performs more detailed segmentation based on the reference length (step SP6).

The CPU 1 identifies the pitch of the segment as a pitch on the absolute pitch axis that can determine that the pitch information is the closest based on the pitch information of the segment thus segmented, and further identifies the pitch of the identified continuous segment. Segmentation is again performed based on whether or not are the same (steps SP7 and SP8).

Thereafter, the CPU 1 obtains a product sum of the frequency of occurrence of the pitch obtained by summing the pitch information after tuning around the pitch axis and a predetermined weighting coefficient determined according to the key, and based on the maximum information of the product sum. Then, for example, the key of the musical composition of the input sound signal is determined, such as C major or A minor, and for a predetermined pitch of the scale in the determined key, the pitch is reviewed with respect to pitch information to confirm and correct the pitch (step SP9, S
P10). Next, the CPU 1 determines whether or not there is the same one for consecutive segments from the finally determined pitch,
The segmentation is reviewed based on whether or not there is a change in power between consecutive segments, and final segmentation is performed (step SP11).

Once the pitch and segment are determined in this way,
The CPU 1 extracts measures from the viewpoint that the music starts from the first beat, the last sound of the phrase does not extend to the next measure, and there is a break in each measure, and determines the time signature from the measure information and the segmentation information. The tempo is determined from the determined time signature information and the length of the bar (step SP1).
2, SP13).

Then, the CPU 1 organizes the information on the determined pitch, pitch, key, beat, and tempo to finally create the musical score data (step SP14).

Segmentation Based on Power Information Next, a segmentation process (step SP5 in FIG. 5) based on the power information of the audio signal in such an automatic transcription system will be described in detail with reference to the flowcharts of FIGS. FIG. 1 is a flowchart showing such processing at a functional level, and FIG. 2 is a flowchart showing FIG. 1 in more detail.

As power information of the acoustic signal, the acoustic signal is squared at each sampling point in the analysis cycle, and
The sum of the power values is used as power information in the analysis cycle.

The CPU 1 compares the power information of each analysis point with a threshold, divides the audio signal into a section larger than the threshold and a section smaller than the threshold, sets a section larger than the threshold as a segment of an effective section, sets a smaller section as a segment of an invalid section, Is marked at the beginning of the segment and the end of the segment is marked at the end (steps SP15 and SP16). The reason for this is that the pitch of the acoustic signal is often unstable in a range where the power information is small, and the pitch identification is often not performed properly, and also to detect a rest period. .

Next, the CPU 1 calculates a change function of the power information in the divided effective segments, extracts a rising change point of the power information from the change function, and subdivides the effective segment into the extracted rising change points ( Step SP17, 1
8). The reason for this is that the sound may shift to the next sound while maintaining a certain level of power. In this case also, it is considered that the power is increased at the start of the next sound.

Next, such processing will be described in more detail with reference to the flowchart of FIG.

CPU1 first clears the analysis point parameter t to 0,
Confirm that the analysis point data to be processed is not completed, and power information of the acoustic signal at that analysis point
It is determined whether (t) is smaller than the threshold value θp (steps SP19 to SP21).

When the power information power (t) is smaller than the threshold value θp, the CPU 1 increments the analysis point parameter t and returns to step SP20 again to determine the power information of the next analysis point.

On the other hand, the CPU 1 determines in step SP21 that the power information power
If the value of (t) is equal to or larger than the threshold value θp, the analysis point is marked as a segment start point, and the process proceeds to the next step SP24 and subsequent steps (step SP23).

The CPU 1 confirms that the processing has not been completed for all the analysis points, and again sets the value of the power information power (t) to the threshold θ.
It is determined whether it is smaller than p, and if it is not smaller than the threshold value θp, the analysis cycle parameter t is incremented and the process returns to step SP24 (steps SP24 to SP26). On the other hand, power information power
When the value of (t) becomes smaller than the threshold value θp, a mark is added to the analysis point as a segment end point, and the process returns to step SP20 (step SP27).

CPU1 executes the above processing in step SP20 or step
In SP24, the processing is performed until it is detected that processing has been completed for all analysis points, and power information powe for all analysis points is
By comparing r (t) with the threshold value θp, the audio signal is divided into an effective segment equal to or more than the threshold value θp and an invalid segment equal to or less than the threshold value θp, and the process proceeds to step SP28 and thereafter.

After this, the CPU 1 sets the analysis point parameter t to 0.
Clear and start the following processing from the first analysis point. CP
After confirming that the analysis point data to be processed has not been completed, U1 determines whether or not the analysis point has been marked with the start of the segment (steps SP29 and SP30). If it is not the analysis point at the start of the segment, the CPU 1 increments the analysis point parameter t and returns to step SP29.

On the other hand, if the analysis point at the start of the segment is detected,
That is, when the start point of the effective segment is found, it is confirmed again that there is no analysis point data to be processed, and it is further determined whether or not the analysis point is at the end of the segment (steps SP32 and SP33). If it is not the analysis point at the end of the segment, it is therefore the analysis point of the active segment.
A change function d (t) of the power information power (t) (hereinafter, referred to as a rising extraction function, which is used for the extraction of the rising edge of the power information in the following processing) is obtained according to the equation (1) (step SP34).

d (t) = {power (t + k) -power (t)} / {power (t + k) + power (t)} (1) where k is a natural number indicating an appropriate time for capturing a change in power. .

Thereafter, the CPU 1 determines whether or not the value of the rising extraction function d (t) obtained is smaller than the threshold value θd. If it is smaller, the analysis point parameter t is incremented and the process returns to step SP32 (steps SP35 and SP36). On the other hand, the rising extraction function d
If (t) is equal to or larger than the threshold value θd, the analysis point is marked as a new segment start (step SP3).
7). Thus, the effective segment is subdivided.

Thereafter, after confirming that the processing has not been completed for all the analysis point data, the CPU 1 determines whether or not the analysis point being processed is marked with a segment end mark.
If so, the process returns to step SP29 described above and returns to the process of detecting the start analysis point of the next valid segment (steps SP38 and SP39).

On the other hand, if the analysis point is not the analysis point at the end of the segment, the rising extraction function d (t) is obtained from the power information power (t) by Expression (1), and the rising extraction function d (t) is determined by the threshold θ
It is determined whether it is smaller than d (steps SP40, 41).
When it becomes smaller, the process returns to step SP32 and proceeds to the process of extracting the rising change point of the power information. On the other hand, step SP41
If the rising point extraction function d (t) of the analysis point continues to be equal to or greater than the threshold value θd, the analysis point parameter t is incremented to determine whether the rising point extraction function d (t) has become smaller than the threshold value θd for the next analysis point. Step SP to judge
Return to 38.

By repeating the above processing, steps SP29 and SP
When it is detected at 32 or SP38 that processing has been completed for all analysis points, the program is terminated.

FIG. 3 shows an example of the segmentation by such processing. In this example, the effective segments S1 to S8 and the invalid segments S11 to S8 are repeated based on the power information power (t) by repeating the processing up to step SP27.
It is divided into 18. Further, by repeating the processing after step SP28, the effective segment S4 is subdivided into S41 and S42 according to the rising transition point of the power based on the rising extraction function d (t).

Therefore, according to the above-described embodiment, the audio signal is divided into an effective segment whose power information is equal to or larger than the threshold value and an invalid segment whose power information is equal to or smaller than the threshold value, and the effective segment is subdivided by the rising transition point of the power information. ,
It is possible to execute highly accurate segmentation without performing erroneous segmentation due to noise or fluctuation of power.

Further, since a section having power information equal to or greater than the threshold value is set as a valid segment, it is possible to avoid using an unstable period during which the sound power is low for an interval such as a pitch identification process.

Further, since the rising change point of the power is extracted and subdivided, the segmentation can be favorably performed even when shifting to the next sound while the power is maintained at a predetermined level or more.

Other Embodiments In the above-described embodiment, the power information is obtained by using the sum of squares of the audio signal. However, other parameters may be used. For example, the square root of the sum of squares may be used. In addition, the function for extracting the rising change point is obtained as in equation (1), but other parameters may be used. For example, the change point may be extracted by a function using only the numerator of equation (1). You may do it.

In the above-described embodiment, the CPU 1 executes all the processing shown in FIG. 5 according to the program stored in the main storage device 3. However, a part or all of the processing is performed by a hardware configuration. May be executed. For example, as shown in FIG. 6 in which the same reference numerals are given to the corresponding parts in FIG. 4, the audio signal from the audio signal input device 8 is amplified through the amplifier circuit 10, and then the pre-filter is added.
The digital signal is supplied to an analog / digital converter 12 via an analog-to-digital converter 11 and converted into a digital signal. The acoustic signal converted to the digital signal is subjected to autocorrelation analysis by a signal processor 13 to extract pitch information, and a square sum processing is performed. And extract power information
It may be provided to the software processing system by the CPU1. As the signal processor 13 used in such a hardware configuration (10 to 13), a processor capable of processing a signal in a voice band in real time and providing an interface signal with a host CPU 1 (for example, NEC Corporation) Co., Ltd. μPD7720) can be applied.

[Effects of the Invention] As described above, according to the present invention, the power information is divided into sections larger and smaller than the threshold, and the larger sections are subdivided at the rising change points of the power information. It is possible to reduce erroneous segmentation due to components and the like, and it is possible to satisfactorily execute the subsequent processing such as the pitch identification processing, thereby improving the transcription accuracy of musical score data.

[Brief description of the drawings]

FIG. 1 is a schematic flowchart showing a segmentation process based on power information according to one embodiment of the present invention, FIG. 2 is a flowchart showing the segmentation process in more detail, and FIG. 3 is a characteristic curve showing an example of the segmentation by the process. FIG. 4 is a block diagram showing a configuration of an automatic transcription system to which the present invention is applied, FIG. 5 is a flowchart showing a free transcription process, and FIG. 6 is a block diagram showing another configuration of the automatic transcription system. is there. 1 ... CPU, 3 ... main storage device, 6 ... auxiliary storage device,
7 ... A / D converter, 8 ... Acoustic signal input device.

 ──────────────────────────────────────────────────続 き Continued on the front page (72) Inventor Masanori Mizuno Examiner, NEC Technical Information System Development Co., Ltd. Shigeo Arai, 5-7-15 Shiba, Minato-ku, Tokyo

Claims (2)

(57) [Claims] An information processing apparatus comprising: at least a process of extracting power information of an input audio signal; and a segmentation process of dividing the audio signal into sections that can be regarded as having the same pitch based on the extracted power information. In the automatic music transcription method for converting into musical score data, the segmentation processing is divided into an effective section in which the power information is equal to or more than a predetermined value and an invalid section in which the power information is equal to or less than the predetermined value. An automatic music transcription method, comprising: a process of extracting points; and a process of subdividing the valid section based on the extracted rising transition points. 2. A power extraction unit for extracting power information from an input audio signal, and a segmentation unit for dividing the audio signal into sections that can be regarded as having the same pitch based on the extracted power information. In an automatic music transcription apparatus for converting the acoustic signal into musical score data, the segmentation means may be divided into an effective section in which the power information is equal to or more than a predetermined value and an invalid section in which the power information is equal to or less than the predetermined value. An automatic music transcription apparatus, comprising: a change point extracting unit that extracts a rising change point of the power information; and a subdivision unit that subdivides the effective section by the extracted rising change point.