JP2876861B2 - Automatic transcription device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an automatic transcription apparatus for converting a music signal into a musical score or a code corresponding to the musical score.
More specifically, the present invention relates to a configuration for constructing chords (chords) in accordance with the progress of music from the transcription results.

[0002]

2. Description of the Related Art Conventionally, music transcription performed by a plurality of musical instruments has been performed by a transcriptionist having musical knowledge. Also, chords (chords) that match the progress of music
Similarly, the transcriber, using his musical knowledge, wrote the transcribed scores.

On the other hand, the present applicant has already proposed a device for automatically transcribing music played by a plurality of musical instruments using a computer (Japanese Patent Application No. 3-11432).
issue). Furthermore, there has been research on manually inputting the melody of music to a computer by a person and constructing chords (chords) that match the input melody (the 32nd Annual Convention of IPSJ). Collection of Lecture Papers, pp. 2137-2138).

[0004]

However, the above-described computer-based chord construction cannot be immediately applied to the automatic transcription of music by a plurality of musical instruments because the input is a single melody note. This is because, when sounds are generated simultaneously by a plurality of musical instruments, a plurality of sounds are included in the automatic transcription result correspondingly.
In addition, in the above-described construction of the chord by the computer, when a person manually inputs the melody, it is necessary to transpose to C major (or A minor) and then input the melody. Has been played in. For the above reasons, it is impossible to construct a code from a result of automatic transcription by a computer simply by combining conventional techniques.

SUMMARY OF THE INVENTION The present invention has been made to solve the above-described problem, and it is possible to accurately calculate a chord (chord) according to the progress of music from a result of automatic transcription of music played by a plurality of musical instruments. It is an object of the present invention to provide an automatic transcription apparatus that can be constructed .

[0006]

In order to achieve this object, an automatic music transcription apparatus according to the present invention takes in a music signal and reads an A / D signal.
A signal capturing unit to be converted, a frequency analysis processing unit that performs frequency analysis at regular time intervals on the data captured by the signal capturing unit, and calculates a power spectrum in a frequency direction within a fixed time; A fundamental frequency candidate extracting unit for extracting a fundamental frequency candidate from the power spectrum calculated by the analysis processing unit; and a processing result of the fundamental frequency candidate extracting unit for converting a note including information such as a pitch, a tone strength, and a tone length. A note information conversion unit for converting information into information; a chord candidate extraction unit for receiving note information from the note information conversion unit and extracting all chords obtained by combining notes present within a certain time; and a chord candidate extraction unit. Associating the key of the music played with the rules of the chord progression, which determines the key of the played music from the appearance of the chord candidates extracted by the section
A rule storage unit for storing Te, in accordance with the tone of music determined by said timing determination section, stored in the rule storage unit
The chord candidate extraction unit in accordance with the chord progression rule
A chord construction unit for selectively combining the chord candidates extracted in step 1 and constructing a chord (chord) according to the progress of the music;
An output unit for outputting a processing result of the code construction unit.

[0007]

According to the structure of the present invention, the signal capturing section captures a music signal and A / D converts the signal into a digital signal that can be handled in a computer.

[0008] The frequency analysis processing section performs a frequency analysis on the data captured by the signal capturing section at predetermined time intervals to calculate a power spectrum in a frequency direction within a predetermined time.

[0009] The fundamental frequency candidate extraction section extracts fundamental frequency candidates from the power spectrum.

The note information conversion section converts the processing result of the fundamental frequency candidate extraction section into note information including information such as pitch, tone strength, and note length.

The chord candidate extracting section receives the note information from the note information converting section and extracts all chords obtained by combining the notes existing within a predetermined time.

The key determining unit determines the key of the played music from the appearance of the chord candidates extracted by the chord candidate extracting unit.

[0013] The rule storage unit stores the key of music and the chord progression.
Stores in association with Lumpur, code construction unit, in accordance with the tone of music determined by said timing determination unit, the route
According to the chord progression rules stored in the
Select chord candidates extracted by the chord candidate extraction unit
Combine and construct chords (chords) that match the progress of the music.

The output unit outputs a processing result of the code construction unit.

[0015]

DESCRIPTION OF THE PREFERRED EMBODIMENTS One embodiment of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram of an automatic transcription apparatus according to the present invention. This apparatus comprises an audio amplifier 1 to which a music signal is input, a low-pass filter 2 for low-pass filtering the amplified music signal, and an A / D for converting the low-pass filtered music signal into a digital signal. A converter 3, an I / O port 4 as an input / output interface, a CPU 5 for controlling the device, and a RAM 6 for temporarily storing data.
And an RO that stores programs for code construction, etc.
M7 and a display 8 for displaying a code construction result.

The audio amplifier 1 is connected to the low-pass filter 2, the low-pass filter 2 is connected to the A / D converter 3, and the A / D converter 3 is connected to the I / O port 4. The I / O port 4 is connected to the CPU 5 and the display 8 in addition to the A / D converter 3. In addition, the CPU 5 has the RA in addition to the I / O port 4.
M6 and ROM7 are also connected.

Next, a functional configuration for constructing a code based on the result of automatic transcription performed by the automatic transcription apparatus of the present invention will be described with reference to FIG. 1 and FIG. This apparatus includes a signal capturing unit 21, an FFT (fast Fourier transform) processing unit 22, and a fundamental frequency candidate extracting unit 23 as functional components.
A note information conversion unit 24, a chord candidate extraction unit 25, a key determination unit 26, a chord construction unit 27, and an output unit 28.

Explaining which of these functional components is comprised in the block diagram of FIG. 1. The signal take-in section 21 comprises an audio amplifier 1, a low-pass filter 2, an A / D converter 3, an I / O port 4, a CPU 5, and the like.

The FFT processing unit 22, the fundamental frequency candidate extracting unit 23, the note information converting unit 24, the chord candidate extracting unit 25, the key determining unit 26, and the chord constructing unit 27
5, a RAM 6, a ROM 7, and the like. The output unit 28 includes the CPU 5, the RAM 6, the display 8, and the like.

Hereinafter, how these functional components function will be described in order.

In the signal receiving section 21, the input music signal is amplified by the audio amplifier 1. This amplified signal is input to the low-pass filter 2,
For example, only frequency components of 5.5 kHz or less pass,
The aliasing distortion during sampling is suppressed. The output signal from the low-pass filter 2 is 12 kHz by the A / D converter 3.
Sampled at z, 16 bits. The sampled data is taken into the CPU 5 via the I / O port 4 and
Stored in AM6.

In the FFT processing unit 22, the CPU 5
The sampled data is read out from 6 and 2 of this data is read out.
One frame is set every 5 msec, and 85.
After applying a 3 msec Hamming window, a log power spectrum is calculated by FFT analysis. Next, the CPU 5
A peak frequency is determined from the calculated logarithmic power spectrum by a parabolic interpolation process.

The fundamental frequency candidate extracting section 23 extracts a fundamental frequency. Specifically, the CPU 5 determines whether a sound in a certain analysis section is a fundamental frequency or a harmonic based on the following three scales. The three scales are as follows: first, the intensity of the sound, and second, the sound is the fundamental frequency.
If the overtone of the sound is included in the peak spectrum (ie, likeness of the fundamental frequency), or thirdly, if the sound is the nth harmonic (2 ≦ n ≦ 8) of another sound, Whether the overtone of the sound that becomes the fundamental frequency is included in the peak spectrum (that is, the likelihood of overtone). The determination is that the first intensity is above a threshold calculated from the intensity of the peak spectrum in the analysis interval, the second fundamental frequency is greater than a certain threshold, and the third harmonic is likely. If the sound is smaller than the threshold value, it is assumed that the sound has the fundamental frequency. The fundamental frequency candidate extraction processing has already been filed by the present applicant (Japanese Patent Application No. 3-11432).

The note information conversion unit 24 converts the sound determined to be the fundamental frequency by the fundamental frequency candidate extraction unit 23 into note information including pitch, note strength, note length, and the like. In the present embodiment, the note information is output separately for a low range and a high range. Since the bass range contains only the lowest tone determined to be the fundamental frequency, note information in the bass range is always a single note. The treble contains all other sounds.

FIG. 3 shows the processing result of the note information conversion unit 24 in this embodiment for a certain section. FIG. 3A shows the processing result for the low frequency range.
(B) shows the processing result for the high frequency range. In this embodiment, note data is MIDI (Musical Instrume).
nt Digital Interface) format. The key number 31 is a value representing the pitch, and in the present embodiment, the center C (keyboard number C4) is represented by 60. Similarly C #
4 indicates 61, D4 indicates 62, and so on. The step time 32 is a value representing a time interval until the next sound is emitted. In this embodiment, one beat is 48. Gate time 3
3 is a value representing the tone length, and the velocity 34 is a value representing the tone strength. The bar line 35 indicates a phrase break of a bar, and the rightmost numeral of the bar line 35 indicates a bar number.

The chord candidate extraction unit 25 includes a note information conversion unit 2
All chords that can be created by combining the note data that are the output results from No. 4 are extracted as chord candidates. Hereinafter, according to the flowcharts of FIGS. 4 and 5,
The operation of the chord candidate extraction unit 25 will be described.

FIG. 4 is a flowchart showing the flow of the process of the chord candidate extraction unit 25. First, the CPU 5 performs initial settings such as setting of a threshold value (S11), and sets a section in which chord candidate extraction processing is performed (S12). Here, 1
/ 2 measures are set as this section. Next, the CPU 5
Counts the note information of the low frequency range existing in the section set in S12 for each sound type and stores it in the RAM 6 (S1).
3). Here, the sound type means a type of sound that does not consider the pitch. That is, C, D, E, F,
G, A, B, C # (= Db: b is a substitute for a flat symbol), D # (= Eb), F # (= Gb), G # (= A
b) and 12 types of A # (= Bb). Therefore, for example, when the sound of the keyboard number C3 and the sound of the keyboard number C4 are present in a certain section, since the sound type of both is C, the count of the sound type C is 2. Similarly, CPU5
Also counts note information in the treble range by note type,
It is stored in the RAM 6 (S14).

In the case where the tone type in the low range is larger than a certain threshold value (S15; Y), or in the case where the tone type in the high range is larger than a certain threshold value (S16; Y), the chord set first is set. Since there is a possibility that the section for performing the candidate extraction processing is too long, the CPU 5 changes the currently set section for performing the chord candidate extraction processing to １ ／ bar. (S1
8). In this case, the sound type is counted again for both the low range and the high range (S19, S20). If the section in which the chord candidate extraction process is performed is changed to １ ／ bar, the remaining １ ／ bar is automatically set as the section in which the chord candidate extraction process is performed. Thereafter, the CPU 5 subsequently performs a chord candidate extraction process (S17).
The details of the processing in S17 will be described in detail again with reference to FIG. When the processing is completed for all sections (S2
1; Y), the processing of the chord candidate extraction unit 25 ends.

FIG. 5 is a flowchart illustrating the details of the chord candidate extraction process in S17 of FIG. First, CPU5
Specifies the tone type of the bass sound (S31). The bass sound is specified in order of C, C #, D,. Now
When the specified tone type of the bass sound is included in the low tone range counted in S13 of FIG. 4 or S19 of FIG. 4 (S32; Y), the CPU 5 specifies the root of the chord (S32; Y). S33). The root of the chord is C, C
#, D,... Are sequentially specified.

Here, the structure of the chord will be briefly described. A triad, which is the basis of a chord, is formed by superimposing a third and fifth higher notes of a certain sound, and this original sound is called a root of the chord.

The root note can be said to be a sound representative of a chord. The third higher sound is called three sounds, and the fifth higher sound is called five sounds. Any of the twelve tone types can be the root of a chord. Next, the CPU 5 determines the bass sound specified in S31,
The relation of the root of the chord specified in S33 is examined. There are certain rules for bass and chords. That is, a sound completely unrelated to a chord does not become a bass sound. In the present embodiment, a rule is provided that the bass sound must match several tones such as the root sound of the chord, and the fifth sound.
If the relation between the base sound and the root of the chord matches this rule (S34; Y), the CPU 5 replaces the sound specified in S33 with the high-pitched sound type counted in S14 or S20 in FIG. It is checked whether or not a sound constituting the chord as the root is included.

First, the CPU 5 checks a major chord. A major chord is a long chord above the root
This is a chord in which the fifth and fifth perfect sounds are superimposed. If the high-pitched tone type includes three tones that make up this chord (S35; Y), the counts of these three tones are summed up and set as the cost of this chord (S36). Furthermore, CPU5
Uses this major chord as one of the chord candidates in RAM6
(S37). Subsequently, the CPU 5 checks whether or not a more complex chord can be created by further adding a note to the chord. In this embodiment, as the sounds to be superimposed, the seventh sound (sound above the root of the seventh note), the major seventh (the sound above the root of the seventh note), the sixth (the sound above the root of the sixth note), and the add Nine's (the sound 9 degrees above the root,
The tone of the second major tone and the tone are the same) are to be processed. If these sounds are included in the treble type,
The CPU 5 turns on the corresponding position of the chord candidate flag stored in the RAM 6 in S37 (S38).

Next, the CPU 5 checks the minor chord. A minor chord is a minor third above the root, complete five
This is a chord with multiple degrees. When the high-pitched tone type includes three tones constituting the chord (S39;
Y), the CPU 5 calculates the cost in the same manner as in the case of the major chord (S40), and sets RA as one of the chord candidates.
It is stored in M6 (S41). The processing for the chord candidate flag is also performed in the same manner as for the major chord (S42).

The CPU 5 finally checks for a special chord. In this embodiment, the special chords include
Augments and diminishes are processed.
A suspension is a chord in which a perfect fourth and fifth perfect note is superimposed on a root note. An augment is a chord in which a third major and a fifth major are superimposed on a root note. Diminish is a chord in which a minor third, fifth fifth, and seventh seventh note are superimposed on the root note. When the treble type includes three or four tones constituting these chords (S
43; Y), in the same manner as in the case of the major chord and the minor chord, the CPU 5 calculates the cost for each chord (S44), turns on the flag at the position corresponding to each chord, and sets the chord candidate. Is stored in the RAM 6 (S45).

With the twelve tone types set as chord roots, the processing from S34 onward is completed for all roots (S46;
Y) Also, this processing is performed using 12 tone types as bass sounds.
When the processing is completed for all bass sounds (S4
7; Y), the chord candidate extraction process in S17 of FIG. 4 ends.

FIG. 6 shows an example of chord candidates extracted by the chord candidate extraction unit 25. Hereinafter, the format of the chord candidates will be described with reference to FIG. The extracted chord is represented by a code name 61.
The lowercase letter b is a substitute for the flat symbol. The symbol ＠ indicates a special chord. Bass sound 62
Corresponds to the note information in the low range. In this embodiment, as described above, the note information is output separately for the low range and the high range. The numeral 63 indicating the type of chord has the following meaning. 0 indicates a major chord, 1 indicates a minor chord, and 2 indicates another special chord. Flag 64
Indicates a sound added to a chord such as Seventh and a special chord type. Each flag is off when it is 0 and on when it is 1. The contents represented by the flag are shown in FIG. The cost 65 of each chord candidate is more likely to be the chord as the number is larger.

In the example 1 shown in FIG. 6, a chord of B flat ("shi")
Flat, "Le", "Fa") and the bass sound is B flat ("S" flat), which is a major chord.
Since the seventh flag is on, it is possible that the chord includes the seventh note of the B flat (the seventh upper minor note, ie, the “la” flat), and the cost of this chord candidate is 14. Similarly, in Example 2 of FIG. 6, the chord of the C minor is G, the base tone is G, the flag of the fractional chord is on for the minor chord (that is, the root of the chord is different from the bass tone), and the cost is 9. In Example 3 of FIG. 6, the special chord of F, the base sound is indicated by F, and the type of special chord is indicated by the flag. In this case, the diminished cost is 6.

FIG. 7 shows the output result of the chord candidate extraction section 25 for the same section as in FIG. The bar number is shown at 71. The start time and the end time of the extracted chord candidates are represented by step times 72.
Chord candidates extracted within the time indicated by 72 are indicated by 73. Each chord candidate is output in the format shown in FIG.

The key determining unit 26 determines the key of the music currently being processed from the appearance of each chord of the chord candidates output from the chord candidate extracting unit 25.

The key of the music is determined as follows.
In a tune of a certain key, it is considered that the chord of the key (for example, the C chord in the C major and the D minor chord in the D minor) appears most frequently. Therefore, for the chord candidates which are the output results from the chord candidate extraction unit 25, the total cost is calculated for each chord for the entire music except for the chord candidates for which the fraction code flag is on and the special chords. Since the chord having the highest total cost has the highest frequency of appearance in the entire music, the tone of the music is determined from this chord. That is, for example, if the total cost of the chords of the B flat is the highest, the key of the music is the B flat major.

The chord constructing section 27 includes a chord candidate extracting section 25
Are combined according to the rules of chord progression. To apply the chord progression rules, the key of the music is required, so the key of the music determined by the key determination unit 26 is used. Hereinafter, the operation of the code construction unit 27 will be described with reference to the flowchart of FIG.

First, the CPU 5 detects the start point of the music (S51). This is the starting point of the song when the chord of the song first appears at the beginning of the bar. In general, a song starts with the chord of the key of the song and ends with the chord of the key of the song.

Next, the CPU 5 extracts four measures of chord candidates from the start point of the music detected in S51 (S52).
The chord construction process is performed in units of chord candidates for four measures. Subsequently, the CPU 5 creates one chord candidate sequence by sequentially connecting the chord candidates for the four measures extracted in S52 (S53).

First, in this embodiment, the chord candidates are
It has been described that extraction is performed for 1/2 bar or 1/4 bar. About this 1/2 bar or 1/4 bar,
Since a plurality of chord candidates are respectively extracted, a plurality of chord candidate strings are formed by this combination. For example, the chord candidates for four measures currently being processed are all 1 /
Assuming that the measures are extracted from two measures, the four measures are divided into eight sections. Assuming that each of these eight sections has n1, n2, n3,... N8 chord candidates, the total number of chord candidate strings formed by this combination is n1 × n2 ×. For all the chord candidate strings, the CPU 5 performs the processing described below.

With respect to the chord candidate sequence created in S53, C
PU5 checks the connection between adjacent chords (S54). There are certain rules for chord progression. In this embodiment, this chord progression rule is represented by a matrix of a connection between two adjacent chords. FIG. 9 shows this matrix. Of the two chords, the first chord is 91
And the later chord at 92. Chords are shown based on the key of the song, not the chord name. That is, for example, assuming that the key of the music is C major, W1 represents C in code name, and W4 represents F. As another example, when the key of the music is A minor, W1 represents A minor, and W4 represents D minor.
Represents a minor. In this way, by expressing a chord based on the key of the music, the matrix can be applied to the music of all keys. The number 1 in the matrix is the chord progression (chord progression from the preceding chord 91 to the subsequent chord 92).
Indicates that it is possible, and 0 indicates that it is not possible.

In S54, if the connection of the chords is OK (S55; Y), if the processing has not been completed to the end of the chord candidate sequence (S58; N), the processing is performed for the next adjacent chords. Move. Conversely, if the connection of the chords is not OK (S55; N), the possibility of chord progression using the borrowed chord is checked (S56). The borrowed chord means that a chord of another key is temporarily used. In this embodiment,
The chord progression is checked using a sub-V chord (V is a substitute for Roman numeral 5), which is one of the representative borrowed chords. If this check is OK (S57; Y), S5
8 is transferred. Conversely, if it is not OK (S5
7; N), the chord candidate string currently being processed is determined to be impossible for chord progression, and the next chord candidate string is created (return to S53).

After completing the check up to the end of the chord candidate string (S58; Y), the CPU 5 checks the end (S59). In the present embodiment, the last chord of the chord candidate sequence is expressed based on the key of the music,
Only the cases of W1, W5 and W6 are allowed.
If this check is OK (S60; Y), the cost of the currently processed chord candidate string is calculated (S6).
1). The cost is calculated by subtracting the count of fractional codes in the chord candidate sequence from the sum of the costs of each chord candidate constituting the chord candidate sequence, and further subtracting the count of chord progression using borrowed chords. . Conversely, if the termination check is not OK (S60; N), S5
7; Return to S53 as in the case of N.

After performing the above processing for all chord candidate strings (S62; Y), the chord candidate string with the maximum cost calculated in S61 is set as a chord for the four measures (S63). Thus, the chord construction process is performed every four measures, and when the process is completed up to the end of the music (S64;
Y), the process of the code construction unit 27 ends.

The output unit 28 outputs the processing result of the code construction unit 27. FIG. 10 shows the output result of the output unit 28 for the same section as in FIGS.

In this embodiment, when a chord candidate sequence is constructed from chord candidates, a two-dimensional matrix indicating whether or not a connection between adjacent chords is possible is used. Thus, the connection of three consecutive chords may be indicated.

In addition to the chord progression check using a matrix, a chord progression including a more complex chord (in this embodiment, indicated as a chord candidate flag) may be checked.

[0052]

As is apparent from the above description,
ADVANTAGE OF THE INVENTION According to the automatic transcription apparatus of this invention, the chord (chord) matched with the progress of music can be constructed | assembled accurately and easily from the automatic transcription result of the music performed by several musical instruments.

[Brief description of the drawings]

FIG. 1 is a block diagram of an embodiment of the present invention.

FIG. 2 is a diagram showing a functional configuration of the embodiment.

FIG. 3 is a diagram showing a processing result of a note information conversion unit.

FIG. 4 is a flowchart illustrating the operation of a chord candidate extraction unit.

FIG. 5 is a flowchart for explaining a chord candidate extraction process in detail.

FIG. 6 is a diagram illustrating an example of chord candidates.

FIG. 7 is a diagram showing a processing result of a chord candidate extraction unit.

FIG. 8 is a flowchart illustrating an operation of a code construction unit.

FIG. 9 is a matrix showing chord progression rules.

FIG. 10 is a diagram illustrating an output result from an output unit.

[Explanation of symbols]

 2 low-pass filter 3 A / D converter 5 CPU 6 RAM 7 ROM 8 display 21 signal capturing unit 22 FFT processing unit 23 fundamental frequency candidate extraction unit 24 note information conversion unit 25 chord candidate extraction unit 26 key determination unit 27 code construction Part 28 Output part

Claims (1)

(57) [Claims] 1. An automatic music transcription apparatus for converting a music signal into a musical score or a code corresponding to a musical score, comprising: a signal capturing unit for capturing and A / D converting a music signal; A frequency analysis processing unit that performs frequency analysis for each time and calculates a power spectrum in a frequency direction within a certain time period, and a fundamental frequency that extracts a candidate for a fundamental frequency from the power spectrum calculated by the frequency analysis processing unit A candidate extraction unit, and a processing result of the fundamental frequency candidate extraction unit,
A note information conversion unit for converting into note information including information such as a note length; a chord candidate for receiving note information from the note information conversion unit and extracting all chords obtained by combining each note existing within a predetermined time. An extraction unit, a key determination unit that determines the key of the played music from the appearance of the chord candidates extracted by the chord candidate extraction unit, and stores the key of the music and the rules of chord progression in association with each other. And
A rule storage unit that, in accordance with the tone of music determined by said timing determination section, the Le
According to the chord progression rules stored in the
Select the chord candidates extracted by the chord candidate extraction unit
Automatic transcription and wherein the binding to the, further comprising a code construction unit for constructing the chords to match the progression of the music, and an output unit for outputting the processing result of the code construction unit.