JP6286933B2 - Apparatus, method, and program for estimating measure interval and extracting feature amount for the estimation - Google Patents

Description

  The present invention relates to an apparatus, a method, and a program for estimating a measure interval and extracting a feature amount for the estimation.

  Tempo (beat interval) and measure interval are elements that compose music. The beat interval is generally expressed in BPM (how many quarter notes there are in one minute) and corresponds to the speed of the entire music. In general, a measure is the smallest unit of music structure, and chords (chords) and rhythm patterns are often assigned to each measure (in some cases, two measures). Therefore, estimating the bar interval is a technique necessary for extracting chords to automatically play electronic musical instruments or to analyze music structures.

  As a first conventional technique for estimating measure intervals and beat intervals, a technique based on the premise that a percussion instrument such as a bass drum or a snare drum cuts a certain beat for a certain period of time has been proposed. More specifically, for example, sound pressure data for each frequency band is created from the music data, the frequency band in which the rhythm is most marked is specified from the sound pressure data, and the change period in the sound pressure data of the specified frequency band is also specified. In addition, a method for estimating a rhythm component has been proposed (for example, Patent Document 1).

  As a second conventional technique for estimating the bar interval / beat interval, the following method has been proposed (for example, Patent Document 2). First, the power for each frame (root mean square) is calculated, and the differential value (according to the embodiment, not the difference between frames, but the gradient when the power of multiple frames before and after is linearly approximated by the least square method is the differential value. As an autocorrelation). The position giving the local maximum of the correlation value (maximum within the expected range of the bar interval) is defined as the bar interval. The beat interval is calculated assuming that the time signature is known. A frame with a large power differential value corresponds to the sound generation time, and the period is estimated by autocorrelation and maximum position detection.

  The following technique has been proposed as a third conventional technique for estimating the bar interval / beat interval (for example, Patent Document 3). First, the sound generation time is detected from the music signal to generate a sound generation time-series signal, and the autocorrelation of the generated signal is calculated. The autocorrelation peak position and peak value are detected. The beat structure of music is analyzed by hierarchizing this peak position and its value into a grouping hierarchy. The tempo that seems to be most appropriate is estimated from the tempo candidate calculated from the peak value of the autocorrelation and the analyzed beat structure.

JP 2000-250534 A JP 05-027751 JP 2002-116754 A

  However, for example, a percussion instrument does not exist in a musical piece, such as a piano solo, and a musical piece whose pronunciation time fluctuates may be difficult to accurately estimate the measure interval and the beat interval.

  Specifically, since the piano solo does not concentrate beats in a specific frequency band, the first prior art described above cannot correctly estimate the bar interval / beat interval for such music. .

In addition, in the method using the periodicity of the pronunciation time as in the second or third prior art described above, the autocorrelation becomes peaky when the pronunciation time has an accurate beat, but it fluctuates at the pronunciation time. If there is, the autocorrelation is lowered, and the bar interval detection accuracy is deteriorated.
FIG. 17 shows an example of a waveform obtained by calculating an autocorrelation signal of an actual piano solo song (hereinafter referred to as “music 1”) by the second prior art technique. Further, as a comparison object, FIG. 18 shows an example of a waveform obtained by calculating an autocorrelation signal of a certain pop music (hereinafter referred to as “music 2”) by the second prior art method. Music 1 has a 3/4 time signature, and music 2 has a 4/4 time signature.

  In FIG. 17 or FIG. 18, the actual beat interval and measure interval of the music 1 or 2 and the maximum position extracted from these music are illustrated. Here, the maximum position is extracted from a calculated autocorrelation value in a time range in which a bar interval exists in a beat interval of about 76 BPM to 140 BPM, for example, a time range of about 1.3 seconds to 4 seconds. Is the maximum value

  First, with respect to the music piece 2 in FIG. 18, the actual bar position matches the detected maximum position. In addition, the time intervals of the autocorrelation value peaks are arranged at almost equal intervals, and the beat structure clearly appears.

  On the other hand, in the musical piece 1 of FIG. 17, a small peak between the bar interval and the maximum position is added, and the detected maximum position appears at the fifth beat, which is completely related to the third beat of the actual bar interval. There is no. In addition, there is no beat structure.

  As described above, the prior art has a problem in that the measure interval and beat interval may not be correctly estimated.

  The present invention makes it possible to perform feature extraction in which features of bar intervals and beat intervals often appear in music such as a piano solo or in music with a beat, and improve the accuracy of extracting bar intervals using this feature amount. The purpose is to do so.

  An example feature amount extraction apparatus includes a time-frequency analysis unit that performs time-frequency analysis on an input acoustic signal and calculates a frequency component signal for each of a plurality of frequency components per unit time, and a unit time A maximum amplitude extraction unit that extracts a frequency component signal having the maximum amplitude as a maximum amplitude signal among frequency component signals for a plurality of frequency components, and an autocorrelation signal of time-series data of the maximum amplitude signal for each unit time are derived. And an autocorrelation deriving unit that extracts the autocorrelation signal as a feature amount.

  In another example of the measure interval estimation device, the autocorrelation signal extracted by the feature amount extraction device is used as a feature amount, and a time having a first time length within a time range starting from a predetermined position of the autocorrelation signal. A peak extraction unit that extracts each peak of the autocorrelation signal that takes the maximum value within each time range specified in the first inspection range while moving the first inspection range that is the range; and each extracted peak The second inspection range, which is the time range having the second time length, is moved within the time range in which a strong and weak pattern period corresponding to the music represented by the acoustic signal is present based on A strong and weak pattern period estimator for selecting a peak having the maximum value in each time range specified in the inspection range from the extracted peaks and estimating a strong and weak pattern period based on the selected peak, and a beat Comprising a beat interval estimation unit that estimates a septum, and a measure interval calculating unit for outputting as a measure interval estimated by calculating the measure interval based on the intensity pattern period and beat interval.

  According to the present invention, it is possible to perform feature amount extraction in which features such as piano solos and beats are effective in features of measure intervals and beat intervals. Extraction of measure intervals is performed using these feature amounts. The accuracy can be improved.

It is a figure which shows an example of the hardware constitutions of embodiment of the bar space estimation apparatus by this invention. It is a block diagram which shows the structural example of the processing function of measure interval estimation apparatus. It is a figure which shows the example of a waveform (music 1) of the autocorrelation signal which concerns on this embodiment. It is a figure which shows the waveform example (music 2) of the autocorrelation signal which concerns on this embodiment. It is a figure which shows the example of a waveform (music 3) of the autocorrelation signal which concerns on this embodiment. It is a figure which shows the waveform example (music 1) of a power autocorrelation signal. It is a figure which shows the waveform example (music 2) of an autocorrelation signal of power. It is a figure which shows the example of a waveform (music 3) of a power autocorrelation signal. It is a figure which shows the waveform example (music 1) of an autocorrelation signal of an amplitude. It is a figure which shows the waveform example (music 2) of an autocorrelation signal of an amplitude. It is a figure which shows the example of a waveform (music piece 3) of an autocorrelation signal of an amplitude. It is a flowchart which shows the process example of the whole bar interval estimation apparatus. It is a flowchart which shows the detailed process example of a feature-value extraction part. It is a flowchart which shows the detailed process example of a peak extraction part. It is a flowchart which shows the detailed process example of an intensity pattern period estimation part. It is a flowchart which shows the detailed process example of a bar space | interval calculation part. It is a figure which shows the example of a waveform (music 1) of the autocorrelation signal by a prior art (patent document 2). It is a figure which shows the example of a waveform (music 2) of the autocorrelation signal by a prior art (patent document 2).

  Hereinafter, embodiments for carrying out the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a diagram illustrating an example of a hardware configuration of an embodiment of a measure interval estimation device according to the present invention. This bar interval estimation device includes a CPU (Central Processing Unit) 101, a ROM (Read Only Memory) 102, a RAM (Random Access Memory) 103, an input unit 104, an output unit 105, an acoustic input unit 106, and an external storage device 107. , Each having a configuration connected to each other by a bus 108.

  The sound input unit 106 takes in music sound data and stores it in the internal RAM 103, a hard disk storage device, or an external storage device 107 such as a solid state drive device as necessary. The sound input unit 106 includes, for example, a network connection device that connects to a music distribution site on the Internet, downloads music sound data, and loads the data into the RAM 103 or the external storage device 107. The sound input unit 106 reads out music sound data recorded on a disk recording medium such as a CD (compact disk) or a DVD (digital versatile disk), and stores it in the RAM 103 or the external storage device 107 as necessary. Including a recording medium driving device. Alternatively, the acoustic input unit 106 includes, for example, a variable recording medium driving device that reads music acoustic data recorded on a portable recording medium such as an SD memory card and stores the data in the RAM 103 or the external storage device 107 as necessary.

  The CPU 101 controls the whole measure interval estimation device according to a control program stored in the ROM 102. The RAM 103 temporarily stores various control data necessary for execution of the control program and music acoustic data for which measure intervals are estimated.

  The input unit 104 detects an input operation by a user using a keyboard, a mouse, various switches, and the like, and notifies the CPU 101 of the detection result.

  The output unit 105 outputs data sent under the control of the CPU 101 to a liquid crystal display or a printer. For example, the output unit 105 displays a measure interval estimation result on the liquid crystal display.

  The external storage device 107 stores music acoustic data, measure interval data, and the like.

  The operation of the measure interval estimation apparatus according to the present embodiment has processing functions shown in the configuration of FIG. 2 to be described later, and a control program for executing the processing of each flowchart shown in FIGS. This is realized by the CPU 101 reading it from the ROM 102 and executing it sequentially. The program may be recorded and distributed in the external storage device 107, for example, or may be acquired from the network by a network connection device (not shown).

  FIG. 2 is a block diagram illustrating a configuration example of processing functions of the measure interval estimation device 201 having the hardware configuration example of FIG. The measure interval estimation device 201 according to the present embodiment includes a feature amount extraction unit 202, a peak extraction unit 206, an intensity pattern period estimation unit 207, a beat interval estimation unit 208, and a measure interval calculation unit 209.

  First, the feature amount extraction unit 202 will be described. The feature amount extraction unit 202 includes a time frequency analysis unit 203, a maximum amplitude extraction unit 204, and an autocorrelation derivation unit 205, which extracts a feature amount when estimating the bar interval from digital data of an input acoustic signal.

  The time-frequency analysis unit 203 performs time-frequency analysis on the digital data of the acoustic signal input from the external storage device 107, for example, and calculates frequency component signals for a plurality of frequency components for each discrete time (for example, analysis frame). To do.

  The maximum amplitude extraction unit 204 extracts a frequency component signal having the maximum amplitude among the frequency component signals for each of a plurality of frequency components as a maximum amplitude signal for each discrete time (for example, an analysis frame).

  The autocorrelation deriving unit 205 derives an autocorrelation signal of time series data of the maximum amplitude signal for each discrete time, and extracts the autocorrelation signal as a feature amount.

  As described above, in the present embodiment, the feature amount extraction unit 202 does not perform autocorrelation of a peaky signal (such as a time fluctuation is steep) such as a pronunciation time, but an autocorrelation of a maximum amplitude signal having a more gradual time fluctuation. Robustness is enhanced by using features.

  Specifically, by using the time-series data of the above-mentioned maximum amplitude signal, the periodicity is estimated based on the strength pattern of the beats in the measure. As for the strong / weak pattern, there are strong / weak / medium / weak and weak / strong / weak / medium called back beats in 4/4 time. Other than that, the first beat is often strong. If the autocorrelation of this strong and weak pattern is obtained, it is expected that a peak appears at a position corresponding to the strong and weak pattern period. In many cases, the intensity pattern period becomes the bar interval, so that the bar interval can be determined based on the peak indicating the intensity pattern period. Even in the case of a musical piece with a strong and weak pattern period of 2 bar intervals, the peak position is detected at a position twice the bar interval. Such a case can be corrected by the function of the bar interval calculation unit 209 described later. is there.

  Next, signals to be used for detecting the strong / weak pattern period will be described. Power and amplitude are often used as the intensity of an acoustic signal. Since music has a wide dynamic range, power is not appropriate because it has a strong influence that is too strong and a weak influence that is too weak. Therefore, in this embodiment, the amplitude is an intensity signal. Further, in the present embodiment, the amplitude having the maximum amplitude among a plurality of frequency component signals obtained by frequency analysis of the analysis frame, for example, is used as the amplitude, instead of the square root of the total power in the analysis frame. This is because, at the timing when a plurality of musical instruments are played, their powers are added together, so that they are not appropriate as intensity signals. In addition, in a section where the sound production interval is close, new sound power is added one after another before the previous sound is completely attenuated. In the present embodiment, for example, by extracting the above-mentioned maximum amplitude signal for each analysis frame, it is possible to generate time-series data in which a strong and weak pattern due to music is most likely to appear.

  In the present embodiment, the autocorrelation signal is derived by calculating the autocorrelation of the time-series data with the maximum amplitude, and this is used as the feature amount including the strength / weakness pattern period corresponding to the bar interval / beat interval information.

  3, 4, and 5 are diagrams illustrating waveform examples of autocorrelation signals that are extracted as feature amounts by the feature amount extraction unit 202 corresponding to the music piece 1, the music piece 2, and the music piece 3. 3 to 5, the actual intensity pattern period, beat interval, and bar interval of the music 1, 2, or 3 and the maximum extracted from these music by the peak extraction unit 206 and the intensity pattern period estimation unit 207 described later. The position is illustrated.

  3 to 5, by using the feature quantity of the autocorrelation signal of the time-series data of the maximum amplitude signal output from the feature quantity extraction unit 202, the beat interval is from time zero except for the vicinity of time zero. It can be seen that it can be easily detected as the first peak appearing within a time range of about 1 second. The beat interval estimation unit 208 in FIG. 2 estimates the beat interval based on such logic.

  Further, in the example of FIG. 3 or FIG. 4, it is assumed that a strong and weak pattern period exists by using the feature quantity of the autocorrelation signal of the time-series data of the maximum amplitude signal output from the feature quantity extraction unit 202. It can be seen that the local maximum position in the time range of about 2 to 4 seconds is in good agreement with the strong and weak pattern period.

  In the present embodiment, the peak inspection unit 206 in FIG. 2 performs a first test that is a time range having a relatively short first time length within a time range starting from a predetermined position (for example, time zero) of the autocorrelation signal. While moving the range, each peak of the autocorrelation signal having the maximum value in each time range specified in the first inspection range is extracted. Thereby, in this embodiment, it becomes possible to perform the robust peak detection which removed the peak resulting from noise.

  Furthermore, in the examples of FIGS. 3 to 5, the waveform of the autocorrelation signal of the time-series data of the maximum amplitude signal output from the feature amount extraction unit 202 is generally downward in the direction in which time increases. It can be seen that each peak corresponding to the strong and weak pattern period has a shape obtained by scraping the waveform around the peak. It can also be seen that the autocorrelation value of the strong and weak pattern period is always larger than the autocorrelation value at twice the time. Therefore, in the present embodiment, the strong and weak pattern period estimation unit 207 has a relatively long second time length within a time range of about 1.3 to 4 seconds, for example, where the strong and weak pattern period is assumed to exist. The peak having the maximum value first in each time range specified in the second inspection range is selected from the peaks extracted by the peak extraction unit 206 while moving the second inspection range. The peak position is estimated as a strong and weak pattern period. As a result, for example, in the waveform example of FIG. 5, for the peak displayed as “maximum position”, a large peak exists in the second inspection range immediately before the peak, and the peak immediately before the peak is a strength pattern. For example, the period is in front of a time range of about 1.3 to 4 seconds. Therefore, in the present embodiment, it is possible to operate so that the peak position displayed as “maximum position” and the peak position immediately before the peak position are not estimated as the intensity pattern period. Actually, the peak position displayed as “strong / weak pattern period” is estimated as the strong / weak pattern period. As described above, in the present embodiment, the waveform characteristic (downwardly descending) of the autocorrelation signal of the time-series data of the maximum amplitude signal output from the feature amount extraction unit 202 is used well, and the pattern pattern period is highly accurate. Can be estimated.

  Furthermore, in the waveform example of FIG. 5, the strong and weak pattern period appears at a position twice the actual bar interval. Even in actual music, there is a case where one strong and weak pattern period is formed in two bars. Therefore, in the present embodiment, the measure interval calculation unit 209 in FIG. 2 uses the beat interval estimated by the beat interval estimation unit 208 to use the beat number (beat interval of the beat interval) estimated by the strength pattern period estimation unit 207. Several times). If the number of beats is smaller than the predetermined value, the measure interval calculation unit 209 calculates the strength pattern period estimated by the strength pattern period estimation unit 207 as the measure interval. On the other hand, if the number of beats is larger than the predetermined value, the measure interval calculation unit 209 calculates a cycle that is half the strength pattern cycle estimated by the strength pattern cycle estimation unit 207 as the measure interval.

  FIG. 6, FIG. 7, and FIG. 8 show the autocorrelation of the power of each analysis frame as a feature amount corresponding to the music 1, the music 2, and the music 3, respectively, as a comparison with FIG. 3, FIG. 4, and FIG. It is a figure which shows the example of a waveform when a signal is calculated. 9, 10, and 11 correspond to music 1, music 2, and music 3 as a comparison with FIGS. 3, 4, 5, 5, 6, 7, and 8, respectively. It is a figure which shows the example of a waveform when the autocorrelation signal of the amplitude (square root of the total power) of each analysis frame is calculated as a feature quantity. 6 and 9, the strong and weak pattern periods are extracted relatively accurately as in the waveform example of FIG. 3 based on the present embodiment. However, for the music piece 2, the waveform example of the power autocorrelation signal in FIG. 7 can easily understand the period of the strong and weak pattern, and the waveform example of the autocorrelation signal of amplitude in FIG. Is a level. However, it is not as remarkable as the waveform example of FIG. 4 based on this embodiment. For music 3, both FIG. 8 and FIG. 11 are more difficult to understand the strength and weakness pattern period than the waveform example of FIG. 5 based on this embodiment. Also, overall, details are impaired in the waveform examples of the autocorrelation signals of power shown in FIGS. The waveform examples of the autocorrelation signals having the amplitudes shown in FIGS. 9 to 11 are slightly better than the waveform examples of the power autocorrelation signals shown in FIGS. 6 to 8, but the waveform examples according to the present embodiment shown in FIGS. Compared, the details are still lost.

  From the above, it can be said that the autocorrelation signal of the maximum amplitude signal extracted by the feature amount extraction unit 202 in this embodiment is excellent as a feature amount for estimating the beat interval, the strength pattern period, and the measure interval. .

  Hereinafter, the details of the control operation of the present embodiment for realizing the processing function of FIG. 2 will be described with reference to the flowcharts of FIGS. In the description of each flowchart, each part of FIG. 1 or FIG. 2 is referred to as needed.

  FIG. 12 is a flowchart showing an example of the overall processing of the measure interval estimation apparatus having the hardware configuration example of FIG. 1 and the processing function configuration example of FIG. This flowchart is realized as an operation in which the CPU 101 executes a measure interval estimation program stored in the ROM 102.

  Feature amount extraction processing (step S101) is performed as processing for realizing the functions of the feature amount extraction unit 202, peak extraction unit 206, strength pattern period estimation unit 207, beat interval estimation unit 208, and measure interval calculation unit 209 in FIG. The control processing is executed in the order of peak extraction processing (step S102), strength pattern period estimation processing (step S103), beat interval estimation processing (step S104), and measure interval calculation processing (step S105).

  FIG. 13 is a flowchart showing a detailed processing example of the feature amount extraction processing in step S101 of FIG.

  First, for example, time frequency analysis processing is performed on digital data of an acoustic signal input from the external storage device 107, and frequency component signals for a plurality of frequency components are calculated for each discrete time (for example, analysis frame) ( Step S201). This process realizes the function of the time-frequency analysis unit 203 in FIG. As the time-frequency analysis processing, short-time Fourier transform, wavelet transform, constant Q filter bank, or the like can be used. Thereby, the complex signal of each frequency channel for every discrete time is obtained.

  Next, maximum amplitude extraction processing is executed (step S202). In this processing, the frequency component signal having the maximum amplitude is extracted as the maximum amplitude signal among the frequency component signals for each of the plurality of frequency components obtained by the time frequency analysis processing in step S201 for each discrete time (for example, analysis frame). Is done. This process realizes the function of the maximum amplitude extraction unit 204 in FIG. At this time, the maximum value may be obtained after first calculating the amplitude of each frequency component. However, it is advantageous in terms of calculation load to obtain the maximum value of power and calculate the square root thereof.

  Finally, an autocorrelation calculation process is executed (step S203). In this process, the autocorrelation of the time series data of the maximum amplitude signal extracted in step S202 is calculated. The autocorrelation can be obtained, for example, by inverse Fourier transform of the power spectrum of the time series data of the maximum amplitude signal.

  FIG. 14 is a flowchart showing a detailed processing example of the peak extraction processing in step S102 of FIG. Here, peak extraction is performed from the feature amount extracted by the feature amount extraction processing in step S101 of FIG. Beat intervals, bar intervals, and strong and weak pattern periods are included in this peak.

  First, for example, a position which is a variable on the RAM 103 (a counter indicating the position of the horizontal axis to be processed in the following flow loop among the horizontal axes of FIGS. 3 to 5) is set as the start position, and the peak container Is cleared (empty) (step S301). The peak container is an array variable on the RAM 103, for example, for storing the extracted peak (position).

  Next, the inspection range corresponding to the position (first inspection range: the horizontal axis range in which the horizontal axis in FIGS. 3 to 5 inspects the position currently being processed) is determined (step S302). ). Since noise may be picked up when a peak is determined simply by comparing autocorrelation signal values between adjacent neighbors, a time range in which noise is difficult to pick up is checked. For example, in the waveform example of the music piece 3 illustrated in FIG. 5, peaks due to noise are observed before and after the beat interval, and the autocorrelation suddenly increases near zero, so it is easy to pick up such noise. Since the noise is often a small protrusion, it can be easily removed if the first inspection range is set appropriately. Further, since the inspection range varies depending on the position, the first inspection range is determined for each position that is currently processed in this way.

  Then, the maximum value of the autocorrelation value within the first inspection range determined in step S302 is acquired (step S303).

  Subsequently, it is determined whether or not the autocorrelation value at the current processing target position is equal to the maximum value acquired in step S303 (step S304).

  If the autocorrelation value at the current processing target position is equal to the maximum value of the autocorrelation value acquired at step S303 and the determination at step S304 is Yes, the autocorrelation value at the current processing target position is a peak, so the position is It adds to a peak container (step S305).

  On the other hand, if the autocorrelation value at the current processing target position is not equal to the maximum autocorrelation value acquired at step S303 and the determination at step S304 is No, the autocorrelation value at the current processing target position is a peak. Because it is not, do nothing.

  Since the process at this position is completed, it is determined whether or not the position is the final position (step S306).

  If the determination in step S306 is not the final position, there is still a position to be processed, so the position is advanced (step S307), and the process returns to step S302 to continue the process.

  When the processing for all positions is completed, the position is equal to the final position in the determination in step S306, the determination in step S306 is Yes, the processing in the flowchart in FIG. 14 is terminated, and the peak in step S102 in FIG. The extraction process ends.

  FIG. 15 is a flowchart showing a detailed processing example of the strength pattern cycle estimation processing in step S103 of FIG.

  First, a peak within a range to be inspected is acquired from the peak group extracted by the peak extraction process in step S102 of FIG. 12 (step S401). Since the intensity pattern period to be estimated corresponds to one of the peak positions extracted by the peak extraction process, a peak within a time range in which an intensity pattern period is assumed to be acquired is acquired in advance. Assumed range is, for example, assuming that one bar interval is within a time range of about 1.3 to 4 seconds, the strong and weak pattern period is 1 or 2 bar intervals, so the time range is about 1.3 to 8 seconds. . This time range can be freely determined according to what kind of tempo and what time signature is assumed.

  Next, the peak to be inspected is set to the first peak acquired in step S401 (step S402).

  Then, an inspection range (second inspection range) is determined (step S403). As described above, the waveform of the autocorrelation signal is generally downward and has a shape obtained by scraping the waveform around the peak for each peak corresponding to the strong and weak pattern period. Further, since the autocorrelation value of the strong and weak pattern period is always larger than the autocorrelation value at twice the time, it becomes the maximum value within the appropriate second inspection range. For example, the second inspection range corresponds to the peak position to be inspected from the position having a value on the horizontal axis slightly smaller than half of the position of the horizontal axis corresponding to the peak position to be inspected in FIGS. The position is set to a value that is slightly larger than twice the position of the horizontal axis.

  Next, the maximum value of autocorrelation within the inspection range determined in step S403 is acquired (S404).

  Subsequently, it is determined whether or not the autocorrelation value at the peak position to be inspected is equal to the maximum autocorrelation value acquired in step S404 (step S405).

  If the value of the autocorrelation at the peak position to be inspected is equal to the maximum value of the autocorrelation acquired in step S404, and the determination in step S405 is Yes, the peak position to be inspected satisfies the condition as a strong and weak pattern period, so the strength pattern A period based on the horizontal axis position of the peak position is set as the period (step S406). And the process of the flowchart of FIG. 15 is complete | finished, and the strong / weak pattern period estimation process of FIG.12 S103 is complete | finished.

  On the other hand, if the autocorrelation value at the peak position to be inspected and the maximum value of the autocorrelation acquired in step S404 are not equal and the determination in step S405 is No, this peak position is not a strong or weak pattern cycle, and the process continues. Then, it is determined whether or not all the peaks have been inspected (step S407).

  If the inspection of all peaks has not been completed and the determination in step S407 is No, the peak position to be inspected is set to the next peak (step S408), the process returns to step S403, and the inspection process is continued.

  Then, when all the peaks have been inspected and the determination in step S407 is Yes, error processing is executed (step S409). And the process of the flowchart of FIG. 15 is complete | finished, and the strong / weak pattern period estimation process of FIG.12 S103 is complete | finished. Since there is usually no state where the strong and weak pattern period was not found, an error occurs.

  As described above, the dynamic pattern period estimation process in step S103 of FIG. 12 is realized.

  Next, in step S104 of FIG. 12, the beat interval is estimated. Since the beat interval to be estimated corresponds to one of the peak positions extracted in the peak extraction process in step S102, the maximum autocorrelation signal value among the peak positions within the time range in which the beat interval is assumed to exist. It is estimated that there is a beat interval. For example, if it is in the range of 75 to 150 BPM where music with many beats will enter, the time range is about 0.4 to 0.8 seconds, and the beat interval is in the range of 60 to 180 BPM. If so, a time range of about 0.33 to 1.0 seconds is set. If the genre of music and the general BPM range in the genre are known in advance, it may be applied.

  FIG. 16 is a flowchart showing a detailed processing example of the measure interval calculation processing in step S105 of FIG. Here, the bar interval is calculated using the strength / pattern pattern estimated in the strength / pattern pattern estimation process in step S103 of FIG. 12 and the beat interval estimated in step S104.

  First, the beat number of the strong and weak pattern period is calculated (S501). Specifically, the dynamic pattern period is divided by the beat interval, and the division result is rounded to an integer value.

  Next, it is determined whether or not the number of beats calculated in step S501 is greater than or equal to a predetermined value (S502). For example, the predetermined value is set to 6 because it is unlikely that more than six quarters are used in many music pieces. Of course, if the genre of the music is known in advance, a predetermined value corresponding to the genre may be set.

  If the calculated number of beats is less than the predetermined value and the determination in step S502 is No, the strength and weakness pattern period can be estimated as one bar interval, so that the strength and weakness pattern period is set as the bar interval (step S503). Then, the process of the flowchart of FIG. 16 ends, and the measure interval estimation process of step S105 of FIG. 12 ends.

  On the other hand, if the calculated number of beats is equal to or greater than the predetermined value and the determination in step S502 is Yes, the strength / weakness pattern cycle can be estimated as a two-measure interval, so that the strength / weakness pattern cycle multiplied by 0.5 is set as the measure interval. (Step S504). In addition, when the value obtained by multiplying the strength pattern cycle by 0.5 is the predetermined value 6 or more, a value obtained by dividing the strength pattern cycle by 3, 4 or the like may be used. In particular, in the case of a certain kind of music, there are many cases where a group of four bars is used, so that a value divided by four may be used. Then, the process of the flowchart of FIG. 16 ends, and the measure interval estimation process of step S105 of FIG. 12 ends.

  As described above, in this embodiment, by using the autocorrelation of the maximum amplitude signal obtained from the time-frequency analysis as a feature quantity, as is clear from FIGS. 3 to 5, even a musical piece such as a piano solo beats. It is possible to keep the characteristics of strong and weak pattern periods and beat intervals, which are highly related to the bar interval, in the feature amount even in the case of a musical piece with a good effect.

  This is not due to the autocorrelation of a peaky signal (such as a steep time fluctuation) such as the pronunciation time, but because the robustness is enhanced by using the autocorrelation of the maximum amplitude signal with a more gradual time fluctuation as a feature value. To do.

  In addition, the method of extracting the strong and weak pattern period in the present embodiment can accurately find the strong and weak pattern period unlike the conventional method of finding a local maximum value.

  Thereby, in a present Example, it becomes possible to improve the estimation precision of measure interval.

Regarding the above embodiment, the following additional notes are disclosed.
(Appendix 1)
A time-frequency analysis unit that performs time-frequency analysis on an input acoustic signal and calculates a frequency component signal for each of a plurality of frequency components per unit time;
A maximum amplitude extraction unit that extracts, as a maximum amplitude signal, a frequency component signal having a maximum amplitude among frequency component signals for each of the plurality of frequency components for each unit time;
An autocorrelation derivation unit for deriving an autocorrelation signal of time-series data of the maximum amplitude signal for each unit time and extracting the autocorrelation signal as a feature quantity;
A feature quantity extraction device comprising:
(Appendix 2)
An apparatus for estimating a bar interval of the acoustic signal using the autocorrelation signal extracted by the feature quantity extraction device according to attachment 1 as a feature quantity,
Within a time range starting from a predetermined position of the autocorrelation signal, while moving a first inspection range that is a time range having a first time length, within each time range specified in the first inspection range. A peak extractor for extracting each peak of the autocorrelation signal taking the maximum value;
The second inspection range is moved while moving the second inspection range, which is a time range having the second time length, within a time range in which a strong and weak pattern period corresponding to the music represented by the acoustic signal is present. Selecting a peak first having a maximum value in each time range specified in the above from each of the extracted peaks, and an intensity pattern period estimation unit that estimates the intensity pattern period based on the selected peak;
A beat interval estimator for estimating a beat interval based on each extracted peak;
A measure interval calculation unit that outputs a measure interval calculated by calculating a measure interval based on the strength and weakness pattern period and the beat interval;
A measure interval estimation device comprising:
(Appendix 3)
The measure interval calculation unit calculates the number of beats of the intensity pattern period based on the beat interval, and calculates the intensity pattern period as the measure interval if the number of beats is smaller than a predetermined value. If the number is large compared to a predetermined value, a period that is half the dynamic pattern period is calculated as the bar interval.
The measure interval estimation apparatus according to Supplementary Note 2, wherein
(Appendix 4)
Perform time-frequency analysis on the input acoustic signal, calculate the frequency component signal for each of multiple frequency components per unit time,
For each unit time, out of the frequency component signals for each of the plurality of frequency components, the frequency component signal having the maximum amplitude is extracted as the maximum amplitude signal,
Deriving the autocorrelation signal of the time-series data of the maximum amplitude signal per unit time, and extracting the autocorrelation signal as a feature amount,
A feature amount extraction method characterized by that.
(Appendix 5)
A method for estimating a bar interval of the acoustic signal using the autocorrelation signal extracted by the feature quantity extraction method according to attachment 4 as a feature quantity,
Within a time range starting from a predetermined position of the autocorrelation signal, while moving a first inspection range that is a time range having a first time length, within each time range specified in the first inspection range. Extract each peak of the autocorrelation signal that takes the maximum value,
Based on each extracted peak, a beat interval is estimated,
The second inspection range is moved while moving the second inspection range, which is a time range having the second time length, within a time range in which a strong and weak pattern period corresponding to the music represented by the acoustic signal is present. First, the peak having the maximum value in each time range specified in the above is selected from each of the extracted peaks, and the strong and weak pattern period is estimated based on the selected peak.
Calculating a measure interval based on the strength and weakness pattern period and the beat interval, and outputting the calculated measure interval,
A measure interval estimation method characterized by the above.
(Appendix 6)
On the computer,
Perform time-frequency analysis on the input acoustic signal, calculate the frequency component signal for each of multiple frequency components per unit time,
For each unit time, out of the frequency component signals for each of the plurality of frequency components, the frequency component signal having the maximum amplitude is extracted as the maximum amplitude signal,
Deriving the autocorrelation signal of the time-series data of the maximum amplitude signal per unit time, and extracting the autocorrelation signal as a feature amount,
A feature quantity extraction program characterized by causing a process to be executed.
(Appendix 7)
A program for estimating a bar interval of the acoustic signal using the autocorrelation signal extracted by the feature amount extraction program according to attachment 6 as a feature amount,
On the computer,
Within a time range starting from a predetermined position of the autocorrelation signal, while moving a first inspection range that is a time range having a first time length, within each time range specified in the first inspection range. Extract each peak of the autocorrelation signal that takes the maximum value,
Based on each extracted peak, a beat interval is estimated,
The second inspection range is moved while moving the second inspection range, which is a time range having the second time length, within a time range in which a strong and weak pattern period corresponding to the music represented by the acoustic signal is present. First, the peak having the maximum value in each time range specified in the above is selected from each of the extracted peaks, and the strong and weak pattern period is estimated based on the selected peak.
Calculating a measure interval based on the strength and weakness pattern period and the beat interval, and outputting the calculated measure interval,
A measure for estimating measure intervals, which is characterized by causing processing to be executed.

101 CPU
102 ROM
103 RAM
DESCRIPTION OF SYMBOLS 104 Input part 105 Output part 106 Sound input part 107 External storage device 108 Bus 201 Bar interval estimation apparatus 202 Feature quantity extraction part 203 Time frequency analysis part 204 Maximum amplitude extraction part 205 Autocorrelation calculation part 206 Peak extraction part 207 Strength pattern period estimation Part 208 beat interval estimation part 209 measure interval estimation part

Claims (7)

A time-frequency analysis unit that performs time-frequency analysis on input acoustic signal data and calculates a frequency component signal for each of a plurality of frequency components per unit time;
A maximum amplitude extraction unit that extracts, as a maximum amplitude signal, a frequency component signal having a maximum amplitude among frequency component signals for each of the plurality of frequency components for each unit time;
An autocorrelation deriving unit for deriving an autocorrelation signal of time-series data of the maximum amplitude signal from the maximum amplitude signal for each unit time extracted by the maximum amplitude extracting unit and extracting the autocorrelation signal as a feature amount When,
A feature quantity extraction device comprising: An apparatus for estimating a bar interval of the acoustic signal data using the autocorrelation signal extracted by the feature quantity extraction device according to claim 1 as a feature quantity,
Within a time range starting from a predetermined position of the autocorrelation signal, while moving a first inspection range that is a time range having a first time length, within each time range specified in the first inspection range. A peak extractor for extracting each peak of the autocorrelation signal taking the maximum value;
The second inspection is performed while moving the second inspection range, which is a time range having the second time length, within a time range in which a strong and weak pattern period corresponding to the music represented by the acoustic signal data is present. An intensity pattern period estimation unit that selects a peak first having a maximum value in each time range specified by the range from each of the extracted peaks, and estimates the intensity pattern period based on the selected peak;
A beat interval estimator for estimating a beat interval based on each extracted peak;
A measure interval calculation unit that outputs a measure interval calculated by calculating a measure interval based on the strength and weakness pattern period and the beat interval;
A measure interval estimation device comprising: The measure interval calculation unit calculates the number of beats of the intensity pattern period based on the beat interval,
If the beat number is small compared to a predetermined value, the strength pattern period is calculated as the bar interval, and if the beat number is large compared to a predetermined value, a half period of the strength pattern period is calculated as the bar interval. To
The measure interval estimation apparatus according to claim 2, wherein: Performs time-frequency analysis on the input acoustic signal data, calculates a frequency component signal for each of a plurality of frequency components per unit time,
For each unit time, out of the frequency component signals for each of the plurality of frequency components, the frequency component signal having the maximum amplitude is extracted as the maximum amplitude signal,
Deriving the autocorrelation signal of the time-series data of the maximum amplitude signal per unit time, and extracting the autocorrelation signal as a feature amount,
A feature amount extraction method characterized by that. A method for estimating a bar interval of the acoustic signal data using the autocorrelation signal extracted by the feature quantity extraction method according to claim 4 as a feature quantity,
Within a time range starting from a predetermined position of the autocorrelation signal, while moving a first inspection range that is a time range having a first time length, within each time range specified in the first inspection range. Extract each peak of the autocorrelation signal that takes the maximum value,
Based on each extracted peak, a beat interval is estimated,
The second inspection is performed while moving the second inspection range, which is a time range having the second time length, within a time range in which a strong and weak pattern period corresponding to the music represented by the acoustic signal data is present. A peak having the maximum value first in each time range specified by the range is selected from each of the extracted peaks, and the intensity pattern period is estimated based on the selected peak.
Calculating a measure interval based on the strength and weakness pattern period and the beat interval, and outputting the calculated measure interval,
A measure interval estimation method characterized by the above. On the computer,
Performs time-frequency analysis on the input acoustic signal data, calculates a frequency component signal for each of a plurality of frequency components per unit time,
For each unit time, out of the frequency component signals for each of the plurality of frequency components, the frequency component signal having the maximum amplitude is extracted as the maximum amplitude signal,
Deriving an autocorrelation signal of the time-series data of the maximum amplitude signal from the extracted maximum amplitude signal for each unit time, and extracting the autocorrelation signal as a feature amount;
A feature quantity extraction program characterized by causing a process to be executed. A program for estimating a bar interval of the acoustic signal data using the autocorrelation signal extracted by the feature quantity extraction program according to claim 6 as a feature quantity,
On the computer,
Within a time range starting from a predetermined position of the autocorrelation signal, while moving a first inspection range that is a time range having a first time length, within each time range specified in the first inspection range. Extract each peak of the autocorrelation signal that takes the maximum value,
Based on each extracted peak, a beat interval is estimated,
The second inspection is performed while moving the second inspection range, which is a time range having the second time length, within a time range in which a strong and weak pattern period corresponding to the music represented by the acoustic signal data is present. A peak having the maximum value first in each time range specified by the range is selected from each of the extracted peaks, and the intensity pattern period is estimated based on the selected peak.
Calculating a measure interval based on the strength and weakness pattern period and the beat interval, and outputting the calculated measure interval,
A measure for estimating measure intervals, which is characterized by causing processing to be executed.