JP6372072B2 - Acoustic signal analysis apparatus, acoustic signal analysis method, and acoustic signal analysis program - Google Patents

Description

  The present invention relates to an acoustic signal analyzing apparatus, an acoustic signal analyzing method, and an acoustic signal for estimating a beat position and a tempo in real time by analyzing the captured acoustic signal while capturing an acoustic signal representing a performance sound of a music piece. The present invention relates to a computer program applied to a computer provided in an analysis apparatus.

  2. Description of the Related Art Conventionally, for example, as shown in Non-Patent Document 1 below, an acoustic signal analyzer that estimates a beat point by analyzing an acoustic signal representing a musical performance sound is known. In this acoustic signal analyzer, a hidden Markov model is set in which the beat time and the beat position within one measure are hidden. The most likely state path is then calculated to determine the beat point.

Geoffroy Peters, Helelne Papadopoulos, “Simultaneos beat and down beer-tracking using a proficient framework: theory and large-scale eval” 19, issue 6, p. 1754-1769

  In the acoustic signal analyzer described in Non-Patent Document 1, the tempo of the music needs to be known. In addition, the beat point cannot be estimated unless the acoustic signal of the entire music is captured. That is, the acoustic signal analyzer described in Non-Patent Document 1 cannot estimate the beat point and tempo in real time.

  The present invention has been made to cope with the above-described problems, and an object of the present invention is to provide an acoustic signal analyzer capable of accurately estimating beat points and tempos in real time. In addition, in the description of each constituent element of the present invention below, in order to facilitate understanding of the present invention, reference numerals of corresponding portions of the embodiment are described in parentheses, but each constituent element of the present invention is The present invention should not be construed as being limited to the configurations of the corresponding portions indicated by the reference numerals of the embodiments.

In order to achieve the above object, the present invention is characterized in that the captured sound signal is analyzed every time a sound signal representing a performance sound of each section when the entire music is divided into a plurality of sections (t _i ) is captured. Thus, an acoustic signal analyzing apparatus (10) for estimating the existence and tempo of beats in each section of the music in real time, classified by a combination of a physical quantity related to the presence of beats in each section of the music and a physical quantity related to tempo Beat point / tempo provisional deciding means (S14) for tentatively determining the presence and tempo of the beat in the current section based on the probability model described as the sequence of the state (q), and the tempo determined in the past section the correction of a provisional decision tempo (ω _in) on the basis of, on the basis of the corrected tempo, tempo of the current interval (ω _target, ω) a Tempo determination means (S15) to be determined, and beat phase calculation means (S17, S18) for calculating the beat phase (θ) corresponding to the time from the current section to the beat point based on the determined tempo The absolute value of the beat phase is set to a predetermined correction value in a section tentatively determined that a beat is present and the beat phase is included in a predetermined range and one or more subsequent sections of the section. The beat phase correcting means (S16) for correcting the beat phase to be reduced, and determining that a beat exists in a section where the sign of the beat phase has changed, and the determined beat point is determined by the tempo determining means. The sound signal analyzing apparatus includes beat point / tempo output means (S19, S20) for outputting together with the tempo.

  In this case, when the difference between the tempo determined in the past section and the tempo twice the tentatively determined tempo is smaller than a predetermined threshold, the tempo determining means has a tempo that is twice the tentatively determined tempo. The tempo of the current section is determined based on the above, and when the difference between the tempo determined in the past section and the tempo of half of the tentatively determined tempo is smaller than a predetermined threshold, half of the tentatively determined tempo The tempo of the current section is determined based on the tempo of the current section, and the difference between the tempo determined in the past section and the tempo twice the tentatively determined tempo is equal to or greater than a predetermined threshold, and in the past section When the difference between the determined tempo and half of the tentatively determined tempo is equal to or greater than a predetermined threshold, the tempo of the current section is determined based on the tentatively determined tempo. When may.

  Further, in this case, the beat phase correction means determines a correction amount for determining a phase angle to be corrected in the section where the beat phase is provisionally determined and the beat phase is included in a predetermined range. Means (S163 to S166) and a section to be corrected in a section provisionally determined that a beat exists, in which a beat phase is included in a predetermined range and one or more sections following the section Correction amount update means (S168) for updating the phase angle to be corrected so that the absolute value of the angle is reduced by the predetermined correction value, and a section provisionally determined to have a beat, It is preferable that correction of the beat phase is started from a section where the beat phase is included in the predetermined range, and the correction of the beat phase is ended when the phase angle to be corrected is equal to or smaller than the predetermined phase angle.

  The tempo temporarily determined based on the probability model as described above may be a tempo that is M times or 1 / M times the true tempo (“M” is an integer). In particular, the tempo may be twice or half the true tempo. In addition, the tentatively determined beat point may be a time point that is shifted by half a beat from the true beat point (that is, a back beat). Further, there is a possibility that it is temporarily determined that a beat point exists in a plurality of frames near the true beat point.

  According to the present invention, the tempo (temporary tempo) value provisionally determined in the current frame is corrected based on the final estimation result of the tempo value in the past frame. This suppresses erroneous estimation that the tempo of the current frame is M times or 1 / M times the true tempo.

  For example, when the temporary tempo value in the current frame can be regarded as the same as the tempo value finally estimated in the previous frame, the final estimated tempo value in the current frame is based on the temporary tempo value. Is set. If the value of the temporary tempo in the current frame is about twice the value of the tempo finally estimated in the previous frame, the final estimated value of the tempo in the current frame is 1 of the temporary tempo. It is set based on a value of / 2. When the temporary tempo value in the current frame is about ½ times the final estimated tempo value in the previous frame, the final estimated tempo value in the current frame is the temporary tempo. Is set based on a value twice as large as. This suppresses erroneous estimation that the current frame tempo is twice or half the true tempo.

  In addition, the beat phase should be “0” in the section where it is temporarily determined that a beat point exists. Therefore, the beat phase was corrected. However, the beat phase is corrected only when the beat phase is included in the predetermined range. For example, in the following cases, it is preferable not to correct the beat phase.

  When the beat point is estimated based on the probability model as described above, it is highly likely that the beat phase is a section close to “0” and that a beat exists in a plurality of sections that are continuous or close to each other. If the beat phase is corrected at this time, the beat point estimation accuracy may be lowered. Therefore, in this case, it may be configured not to correct the beat phase. This suppresses erroneous estimation that a beat exists in a plurality of sections having a beat phase close to “0”. In addition, it is preferable that the beat phase is not corrected in a section where it is considered highly likely that a half-beat shift has occurred.

  Therefore, according to the acoustic signal analyzer configured as described above, the beat point and the tempo can be accurately estimated in real time.

  In addition, the present invention is not limited to the invention of the acoustic signal analysis device, and can also be implemented as an acoustic signal analysis method and a computer program applied to a computer included in the acoustic signal analysis device.

It is a block diagram showing the structure of the acoustic signal analyzer which concerns on one Embodiment of this invention. It is a graph which shows the example of an acoustic signal. It is a flowchart of a beat point / tempo estimation program. It is a conceptual diagram of the hidden Markov model for tentatively determining a beat point and a tempo. It is a block diagram of a comb filter. It is a graph which shows the calculation result of a BPM feature-value. It is a table | surface which shows the structure of a template. It is a conceptual diagram which shows the position of the present flame | frame in a time coordinate system. It is a conceptual diagram which shows the position of the present flame | frame in a circular coordinate system. It is a flowchart of a tempo correction program. It is a flowchart of a beat phase correction program. FIG. 4 is a conceptual diagram showing ranges of regions A and B, and is a conceptual diagram showing an example in which the current beat phase is included in region A. FIG. 4 is a conceptual diagram showing ranges of regions A and B, and is a conceptual diagram showing an example in which the current beat phase is included in region B. 3 is a conceptual diagram showing a range of a region C. FIG. 3 is a conceptual diagram showing a range of a region D. FIG. It is a conceptual diagram which shows the range of the area | region E when a temporary tempo is twice an estimated tempo. It is a conceptual diagram which shows the range of the area | region E when a temporary tempo is 3 times the estimated tempo. It is a graph which shows transition of beat phase (theta) and correction amount (DELTA) (theta), and the estimation result of a beat point. It is a table | surface which shows transition of beat phase (theta) and correction amount (DELTA) (theta). 10 is a graph showing an example in which the temporarily determined tempo is half the true tempo or twice the true tempo. It is a graph which shows the example which is a time point which the tentatively determined beat point shifted | deviated by a half beat from the true beat point. It is a graph which shows the example temporarily determined that a beat point exists in the some flame | frame near a true beat point.

  An acoustic signal analyzer 10 according to an embodiment of the present invention will be described. The acoustic signal analyzing apparatus 10 estimates the position and tempo of the beat in real time by analyzing the captured acoustic signal while capturing the acoustic signal representing the performance sound of the music.

  As shown in FIG. 1, the acoustic signal analyzer 10 includes an input operator 11, a computer unit 12, a display 13, a storage device 14, an external interface circuit 15, and a sound system 16, which are connected via a bus BS. Connected.

  The input operator 11 includes a switch corresponding to an on / off operation (for example, a numeric keypad for inputting a numerical value), a volume or rotary encoder corresponding to a rotation operation, a volume or linear encoder corresponding to a slide operation, a mouse, a touch panel, etc. Composed. These operators are used for starting or stopping beat point / tempo estimation, setting various parameters relating to beat point / tempo estimation, and the like. When the input operator 11 is operated, operation information indicating the operation content is supplied to the computer unit 12 described later via the bus BS.

  The computer unit 12 includes a CPU 12a, a ROM 12b, and a RAM 12c connected to the bus BS. The CPU 12a reads out and executes a beat point / tempo estimation program representing a procedure of beat point / tempo estimation processing described later from the ROM 12b. In addition to the program, the ROM 12b stores various data such as initial setting parameters, graphic data for generating display data representing an image displayed on the display 13, and character data. The RAM 12c temporarily stores data necessary for executing the program.

  The display 13 is configured by a liquid crystal display (LCD). The computer unit 12 generates display data representing contents to be displayed using graphic data, character data, and the like, and supplies the display data to the display unit 13. For example, the computer unit 12 supplies the display unit 13 with display data representing beat points and tempo estimated by beat point / tempo estimation processing described later. The display device 13 displays an image based on the display data supplied from the computer unit 12.

  The storage device 14 includes a large-capacity nonvolatile recording medium such as an HDD, FDD, CD, or DVD, and a drive unit corresponding to each recording medium.

  The external interface circuit 15 includes a connection terminal that enables the acoustic signal analyzer 10 to be connected to an external device such as an electronic music device or a personal computer. The acoustic signal analyzer 10 can be connected to a communication network such as a LAN (Local Area Network) or the Internet via the external interface circuit 15.

  The sound system 16 includes a sound source circuit that generates a digital sound signal, a D / A converter that converts the generated digital sound signal into an analog sound signal, an amplifier that amplifies the converted analog sound signal, and an amplified analog A speaker that converts a sound signal into an acoustic signal and outputs the sound signal is provided. The sound system 16 includes a microphone for collecting a musical sound emitted by playing a musical piece, an A / D converter for converting an analog sound signal representing the collected musical sound into a digital sound signal, and a converted sound. A buffer for temporarily storing sample data representing the digital sound signal is also provided. That is, the sound system 16 samples the musical sound at a predetermined sampling period (for example, 1/444100 sec), and stores the sample data obtained by the sampling in the buffer.

Next, the operation of the acoustic signal analyzer 10 will be described. First, the outline will be described. The acoustic signal analyzer 10 sequentially samples acoustic signals representing musical performance sounds. When the entire music piece is divided into a plurality of frames t _{i = 0, 1, 2,..., I} (see FIG. 2), the sampling corresponding to each frame is finished (ie, a predetermined number of frames). Every time sample data is obtained), the presence or absence of beats and the tempo for the current frame are estimated using the sample data obtained so far. The above “i” is a frame index. The length of each frame is common. The length of each frame is 4 ms, for example. The “frame” described above corresponds to the “section” of the present invention.

  Next, the operation of the acoustic signal analyzer 10 will be specifically described. When the user turns on a power switch (not shown) of the acoustic signal analyzer 10, the CPU 12a reads the beat point / tempo estimation program shown in FIG. 3 from the ROM 12b and executes it.

  Next, CPU12a performs the initial setting for an acoustic signal analysis in step S11. Specifically, an area for temporarily storing an onset feature value XO, a BPM feature value XB, and the like to be described later is secured in the RAM 12c. Further, the CPU 12a sets the processing target frame as the first frame. That is, the value of “i” that is the index of the frame is set to “0”.

  Next, in step S12, the CPU 12a causes the sound system 16 to start sampling of an acoustic signal representing the musical performance sound.

Then, CPU 12a, at step S13, reads audio signals contained in the frame _{t i} (sample data) from the buffer of the sound system 16. However, if all of the sample data contained in the frame t _i has not yet been sampled, waits until all the sample data is sampled. Then, CPU 12a, at step S14, temporarily determines the presence and tempo beat at frame _{t i.}

Here, an outline of the procedure for temporarily determining the presence / absence of a beat and the tempo will be described. First, the CPU 12a uses the read sample data to calculate an onset feature value XO representing a feature related to the presence or absence of a beat and a BPM (beats per minute) feature value XB representing a feature related to a tempo. calculate. It is described as a series of states q _{b, n} classified according to the combination of the value of the beat period b in each frame t _i (value proportional to the reciprocal of the tempo) and the value of the number of frames n up to the next beat. A forward algorithm is applied to the probabilistic model (Hidden Markov Model, see FIG. 4) to detect a state in which the forward variable is maximized. Thereby, the presence / absence of a beat in frame t _i and the tempo are provisionally determined. The beat period b is represented by the number of frames. Therefore, the value of the beat period b is an integer satisfying “1 ≦ b ≦ b _max ”, and in the state where the value of the beat period b is “β”, the value of the number of frames n is “0 ≦ n <β”. It is an integer that satisfies.

Next, the procedure for temporarily determining the presence / absence of a beat and the tempo will be described in detail. First, CPU 12a calculates the onset feature values XO and BPM feature value XB for a frame _{t i.}

The onset feature value XO (t _i ) of the frame t _i is calculated as follows. CPU12a firstly short-time Fourier transform of the acoustic signal of the frame _{t i,} to calculate the signal strength of each frequency bin. Next, the CPU 12a calculates the signal strength of each frequency band fb _x (for example, x = 1, 2,..., 20) using the mel filter bank. Next, the CPU 12a calculates an increase amount R (fb _x , t _i ) of signal strength in each frequency band with respect to the previous frame. As shown in the following equation (1), the sum of the increase amounts of the signal intensity in each frequency band is the onset feature amount XO (t _i ).

The BPM feature value XB (t _i ) of the frame t _i is calculated as follows. First, the CPU 12a inputs onset feature values XO (t ₀ ), XO (t ₁ ),... In this order to the filter bank FBB (see FIG. 5). Filter bank FBB is composed of a plurality of comb filters CF _b respectively provided in accordance with the value of the beat period b. The comb filter CF _b outputs one data every time one data is input. The comb filter CF _b has a FIFO (= First In First Out) memory for storing past output data by the number corresponding to the value of the beat period b, and is stored in the storage means with the input data. The oldest data among the existing data is added at a predetermined ratio (for example, 1: 1 (that is, α = 0.5)) and output. The sequence XD _b (t) of the data XD _b obtained by inputting the sequence XO (t) {= XO (t ₀ ), XO (t ₁ )...} Of the onset feature quantity XO to the filter bank FBB. By reversing {= XD _b (t ₀ ), XD _b (t ₁ )... In time series and inputting them again into the filter bank FBB, the BPM feature quantity series XB _b (t ) {= XB _b (t ₀ ), XB _b (t ₁ ). The BPM feature value XB (t _i ) of the frame t _i is represented as a set of BPM feature values XB _{b = 1, 2...} (T _i ) calculated for each beat period b (see FIG. 6).

Then, CPU 12a, using the forward algorithm, temporarily determines the presence and tempo beat at frame _{t i.}

Specifically, CPU 12a first calculates onset feature quantity XO _{(t i)} and BPM feature value XB observation likelihood LO _{(t i)} of _{(t i)} and the observation likelihood LB a _{(t i),} respectively . Here, the onset feature amount XO (t _i ) follows a normal distribution set according to the value of the number of frames n up to the next beat point. In other words, the observation likelihood LO (t _i ) of the onset feature quantity XO substitutes the onset feature quantity XO as a normal distribution random variable set according to the value of the number of frames n up to the next beat point. Is calculated by For example, when the value of the number of frames n is “0”, a normal distribution having an average value of “3” and a variance of “1” is used. Further, when the value of the beat period b is “β” and the value of the number of frames n is “β / 2”, the normal value is “1” and the variance is “1”. A distribution is used. Further, when the value of the number of frames n is different from both “0” and “β / 2”, a normal distribution having an average value of “0” and a variance of “1” is used.

Further, the observation likelihood LB of the BPM feature quantity XB (t _i ) corresponds to the adaptability of the BPM feature quantity XB to the template TMP _b provided for each beat period b. That is, as shown in the following equation (2), the inner product of the template TMP _b and the BPM feature quantity XB (t _i ) is the observation likelihood LB (t _i ). Note that “ν _b ” in this arithmetic expression is a coefficient that determines the weight of the BPM feature quantity XB (t _i ) with respect to the onset feature quantity XO (t _i ). That is, the larger the value of “ν _b ” is, the greater the importance is placed on the BPM feature value XB (t _i ). In addition, “Z (ν _b )” in this arithmetic expression is a normalization coefficient that depends on “ν _b ”. In other words, the template TMP _b, the coefficient [delta] _b, which are respectively multiplied to the BPM feature value XB _{(t i)} constituting the BPM feature value _XB b _{(t i),} from the series of _{γ {= 1,2 ···}} (See FIG. 7). Coefficient [delta] _b constituting the template TMP _b, of _gamma, as a factor equal to an integer multiple of the index gamma is the beat period b equal to the coefficient and the beat period b is maximum, template TMP _b is set.

Next, the CPU 12a uses a logarithmic observation likelihood LOB (t _i ) (see Equation (3) below) that is a logarithm of the product of the observation likelihood LO (t _i ) and the observation likelihood LB (t _i ). To calculate a forward variable for each state.

  In the present embodiment (from the state where the value of the beat period b is “βs” and the value of the number of frames n is “ηs”, the value of the beat period b is “βe” and the frame The value of the logarithmic transition probability T to the state where the value of the number n is “ηe” is set as follows (see FIG. 4): “ηe = 0”, “βe = βs”, and “ηe” = Βe−1 ”, the value of the logarithmic transition probability T is“ −0.2. ”When“ ηs = 0 ”,“ βe = βs + 1 ”, and“ ηe = βe−1 ”, The value of the logarithmic transition probability T is “−0.6”, and when “ηs = 0”, “βe = βs−1”, and “ηe = βe−1”, the value of the logarithmic transition probability T. Is “−0.6.” When “ηs> 0”, “βe = βs”, and “ηe = ηs−1”, the logarithmic transition probability T is “0”. Logarithm other than the above The value of the transition probability T is “−∞.” That is, when the transition from the state where the value of the number of frames n is “0” (ηs = 0) to the next state, the value of the beat period b is “1”. In this case, the value of the frame number n is set to a value smaller by “1” than the value of the beat period b after the transition. Also, the value of the frame number n is not “0” ( When transitioning from (ηs ≠ 0) to the next state, the value of the beat period b is not changed, and the value of the number of frames n is decreased by “1”.

Then, CPU 12a, of each state in the frame _{t i,} the value of the forward variable temporarily determines the presence or absence of a beat on the basis of the state _{q max} is the maximum. That is, if the value of the number of frames n in the state q _max is “0”, the CPU 12a temporarily determines that a beat exists in the frame t _i . On the other hand, if the value of the number of frames n in the state q _max is “1” or more, it is provisionally determined that no beat exists in the frame t _i . Further, the CPU 12a provisionally determines the tempo based on the value of the beat cycle b of the state q _max . If the frame t _i beats has to the provisionally determined the presence, CPU 12a is a beat event flag BF representing the presence or absence of beats is set to "1". On the other hand, if the frame _{t i} beats was provisionally decided not to exist, CPU12a sets the beat event flag BF to "0".

  The tempo temporarily determined as described above may be a tempo that is half the true tempo or a tempo that is twice the true tempo (see FIG. 20). In addition, the tentatively determined beat point may be a time point shifted from the true beat point by half a beat (that is, a back beat) (see FIG. 21). Further, there is a possibility that a beat point is temporarily determined to exist in a plurality of frames near the true beat point (see FIG. 22). That is, there is a possibility that the beat event flag BF is set to “1” in a plurality of continuous (or close) frames.

  Therefore, the CPU 12a executes steps S15 to S19 described below to correct the estimation result regarding the tempo and the estimation result regarding the presence or absence of beat points.

Here, the beat phase will be described with reference to FIGS. 8 and 9. As described above, the beat period b corresponds to the tempo. The frame number n corresponds to the time until the next beat point. The time coordinate system from the time point before half a beat as viewed from one beat point in FIG. 8 to the time point after half a beat as viewed from the one beat point is a cyclic coordinate system as shown in FIG. It can be expressed as (rotating coordinate system). Then, the angular velocity in this circular coordinate system corresponds to the tempo. The phase angle corresponds to the time until the beat point. In the following description, the temporarily determined tempo is expressed as a temporary tempo ω _in (rad / frame). In addition, the tempo to be finally estimated (that is, the true tempo) is expressed as an estimated tempo ω _target (rad / frame). Also, the tempo as a variable used internally in the tempo correction process is expressed as an internal tempo ω (rad / frame). In addition, a phase angle used internally in the later-described beat phase correction process and beat point determination process is referred to as a beat phase θ (rad). The value of the beat phase θ corresponding to a time point half a beat before the one beat point is “−π (rad)”, and the beat phase corresponding to a time point half a beat after the one beat point The value of θ is “π (rad)”. Further, the beat phase θ basically advances by a phase angle corresponding to the internal tempo ω for each frame. However, as will be described in detail later, correction of the beat phase θ is started when the beat phase θ is included in a predetermined range in a frame in which the beat event flag BF is “1”, and the subsequent frame or frames are corrected. The beat phase θ is corrected.

First, in step S15, the CPU 12a executes a tempo correction process shown in FIG. In the figure, the “judgment” step is indicated by a hexagon. In the tempo correction process shown in the figure, the CPU 12a updates the values of the estimated tempo ω _target and the internal tempo ω as follows. First of all, CPU12a is, if the value of the provisional tempo _{ω in} can be regarded the same as the value of the estimated tempo ω _target in one previous frame, and the value of the value of the estimated tempo ω _target provisional tempo _{ω in} the frame _{t i} Update to the same value. In addition, if the value of the provisional tempo _{ω in} is about twice the value of the estimated tempo ω _target in one previous frame, the value of the estimated tempo ω _target in the frame _{t i} of the provisional tempo _{ω in} 1/2 Update to double value. When the value of the temporary tempo ω _in is about ½ times the value of the estimated tempo ω _target in the previous frame, the estimated tempo ω _target is updated to a value twice the temporary tempo ω _in. . Then, the value of the internal tempo ω is corrected so as to approach the estimated tempo ω _target .

Next, the tempo correction process will be specifically described. The CPU 12a starts tempo correction processing in step S150. Then, in step S151, it is determined whether or not the temporary tempo ω _in is about twice the estimated tempo ω _target . Specifically, it is determined whether or not the difference between the half value of the temporary tempo ω _in and the estimated tempo ω _target is smaller than a predetermined threshold ω _thres (for example, ω _thres = 0.05ω _in ). If the difference between the half value of the temporary tempo ω _in and the estimated tempo ω _target is smaller than the predetermined threshold ω _thres , the CPU 12a determines “Yes”, and the estimated tempo ω _target is temporarily set in step S152. Update to half of tempo ω _in . On the other hand, when the difference between the half value of the temporary tempo ω _in and the estimated tempo ω _target is equal to or larger than the predetermined threshold ω _thres in step S151, the CPU 12a determines “No”, and in step S153, It is determined whether or not the temporary tempo ω _in is about half of the estimated tempo ω _target . Specifically, it is determined whether or not the difference between the value twice the temporary tempo ω _in and the estimated tempo ω _target is smaller than the threshold ω _thres . When the difference between the value twice the temporary tempo ω _in and the estimated tempo ω _target is smaller than the threshold ω _thres , the CPU 12a determines “Yes”, and the estimated tempo ω _{target is set} to the temporary tempo in step S154. Update to twice the value of ω _in . On the other hand, if the difference between the value twice the temporary tempo ω _in and the estimated tempo ω _target is equal to or greater than the threshold ω _{thres in} step S153, the CPU 12a determines “No” and estimates in step S155. The tempo ω _target is updated to the same value as the temporary tempo ω _in . However, in the first frame t ₀ , the estimated tempo ω _target is set to the same value as the temporary tempo ω _in .

Next, in step S156, the CPU 12a updates the value of the internal tempo ω using the update formula shown in FIG. By using this update formula, the value of the internal tempo ω approaches the estimated tempo ω _target exponentially in a plurality of frames following the current frame. The coefficient ρ in the update formula is set in advance so as to satisfy “0 ≦ ρ ≦ 1”. For example, the coefficient ρ is set to “0.5”. The coefficient ρ represents the smoothness of the tempo transition. That is, the smaller the coefficient ρ, the slower the internal tempo ω approaches the estimated tempo ω _target . In step S157, the CPU 12a ends the tempo correction process, and proceeds to step S16 of the main routine.

Next, in step S16, the CPU 12a executes a beat phase correction process shown in FIG. In the figure, the “judgment” step is indicated by a hexagon. In the beat phase correction process, the CPU 12a corrects the beat phase θ as follows. First, a case where the temporary tempo ω _in and the estimated tempo ω _target can be regarded as the same will be described. When the beat phase θ is included in the region A (ζω ≦ θ ≦ θ _thres ) as shown in FIG. 12 or the region B (−θ as shown in FIG. 13 in the frame t _i where the beat event flag BF is “1”. _(thres ≦ θ ≦ −ζω), the beat phase θ is a correction amount Δθ representing the phase angle to be corrected. The coefficient ζ is set in advance so as to satisfy “0 ≦ ζ ≦ 1”. For example, the coefficient ζ is set to “0.5”. In this case, instead of correcting the beat phase θ by the correction amount Δθ in the frame t _i , the beat phase θ is corrected little by little (for example, “ζω”) over the frame t _i and one or more subsequent frames. To do. In the present embodiment, “θ _thres = π / 2”.

  Further, in the region C (−ζω ≦ θ ≦ ζω) shown in FIG. 14, the beat event flag BF is highly likely to be set to “1” in a plurality of continuous or adjacent frames. If the beat phase θ is corrected when the beat phase θ is included in the region C, the beat point estimation accuracy may be lowered. Therefore, in this case, the beat phase θ is not corrected.

When the beat phase θ is included in the region D (θ _thres ≦ θ ≦ π or −π ≦ θ ≦ −θ _thres ) shown in FIG. 15 and the beat event flag BF is set to “1”, There is a high possibility that a beat shift has occurred. Therefore, in this case, the beat phase θ is not corrected.

Next, the case where the temporary tempo ω _in is “M” times the estimated tempo ω _target (M is a natural number equal to or greater than “2”) will be described. In this case, the region E (“2πm / M + ζω / M ≦ θ ≦ 2πm / M + θ _thres / M” and “2πm” in the vicinity of “2πm / M” (“m” is an integer satisfying “0 ≦ m <M”). / M−θ _thres / M ≦ θ ≦ 2πm / M−ζω ”is a region wrapped within the range of“ −π (rad) ”to“ π (rad) ”: FIG. 16 (M = 2) and FIG. M = 3) corresponds to region A and region B (see FIGS. 12 and 13). Therefore, when the beat phase θ is included in the region E in the frame t _i in which the beat event flag BF is “1”, the beat phase θ is multiplied by “M”, and the phase angle is set to “−π (rad)” to Wrapping within the range of “π (rad)”. A value obtained by multiplying the lapped phase angle by “1 / M” is a correction amount Δθ of the beat phase θ.

Next, the beat phase correction process will be specifically described. In step S160, the CPU 12a starts the beat phase correction process. Next, in step S161, the CPU 12a determines whether or not the beat event flag BF is “1”. When the beat event flag BF is “0”, the CPU 12a determines “No” and advances the process to step S167 described later. On the other hand, when the beat event flag BF is “1”, the CPU 12a determines “Yes”, and whether the beat phase θ is included in the predetermined region (that is, the region A, B, or E) in step S162. Determine whether or not. When the beat phase θ is not included in the predetermined area, the CPU 12a determines “No” and proceeds to step S167 described later. On the other hand, when the beat phase θ is included in the predetermined area, the CPU 12a determines “Yes”, and in step S163, based on the arithmetic expression shown in FIG. 9, the temporary tempo ω _in and the estimated tempo ω _target The ratio M is calculated. Next, in step S164, the CPU 12a multiplies the current beat phase θ by the ratio M. In step S165, the CPU 12a wraps the multiplication result within a range of “−π (rad)” to “π (rad)”. However, the value of the beat phase θ in the first frame t ₀ is “0”. Next, in step S166, the CPU 12a calculates the correction amount Δθ of the beat phase θ by multiplying the lapped phase angle by “1 / M”.

Next, in step S167, the CPU 12a determines whether or not the absolute value of the correction amount Δθ is included in a predetermined range (ζω <| Δθ | <θ _thres ). When the absolute value of the correction amount Δθ is included in the predetermined range, in step S168, the CPU 12a updates the beat phase θ and the correction amount Δθ based on the update formula shown in FIG. That is, when the correction amount Δθ is a positive value, the phase angle corresponding to “ζω” is subtracted from the beat phase θ. On the other hand, when the correction amount Δθ is a negative value, a phase angle corresponding to “ζω” is added to the beat phase θ. When the correction amount Δθ is a positive value, the phase angle corresponding to “ζω” is subtracted from the correction amount Δθ. On the other hand, when the correction amount Δθ is a negative value, a phase angle corresponding to “ζω” is added to the correction amount Δθ. Then, in step S169, the CPU 12a ends the beat phase correction process and proceeds to step S17 of the main routine. On the other hand, if the absolute value of the correction amount Δθ is not included in the predetermined range in step S167, the CPU 12a determines “No” and corrects the beat phase in step S169 without correcting the beat phase. The correction process is terminated, and the process proceeds to step S17 of the main routine.

Next, in step S17, the CPU 12a advances the beat phase θ by a phase angle corresponding to the internal tempo ω. Next, in step S18, the CPU 12a wraps the beat phase θ within a range of “−π (rad)” to “π (rad)”. Next, in step S19, the CPU 12a determines whether or not the sign of the beat phase θ has changed. That is, it is determined whether or not the beat phase θ has changed from a negative value to “0” or whether the beat position θ has changed from a negative value to a positive value. When the sign of the beat phase θ has not changed, the CPU 12a determines “No” and advances the process to step S21 described later. On the other hand, if the beat phase θ is “0” or the sign of the beat phase θ is changed from negative to positive, the CPU 12a determines “Yes” and outputs a beat event in step S20. For example, displays on the display unit 13 to the effect that the beat point current frame (frame _{t i)} is present, the value of the estimated tempo omega value (BPM value corresponds to the estimated tempo omega _target) of _target or internal tempo omega (internal BPM value corresponding to tempo ω) is displayed on the display 13 as the tempo of the current frame. In step S21, the frame index i is incremented, and the process proceeds to step S13.

Examples of transition of beat phase θ and correction amount Δθ calculated as described above, and beat point estimation results are shown in FIGS. In this example, in order to simplify the explanation, it is assumed that the temporary tempo ω _in is constant _in all frames and is equal to the estimated tempo ω _target . In FIG. 18, the start timing of the frame in which the beat event flag BF is set to “1” is indicated by a one-dot chain line. In addition, the finally estimated beat point is indicated by “·”. As shown in the figure, correction of the beat phase θ is started only when the beat event flag BF is “1” and the beat phase θ is included in the region A or B. Even when the beat event flag BF is “1”, the correction of the beat phase θ is not started when the beat phase θ is included in the region C or D.

In the acoustic signal analyzer 10, when the value of the temporary tempo ω _in can be regarded as the same as the value of the estimated tempo ω _target in the previous frame, the value of the estimated tempo ω _target in the current frame is set to the value of the temporary tempo ω _in . Update to the same value as the value. When the value of the temporary tempo ω _in is about twice the value of the estimated tempo ω _target in the previous frame, the value of the estimated tempo ω _target in the current frame is ½ times the temporary tempo ω _in . Update to the value of. In addition, if the value of the provisional tempo _{ω in} is 1/2 times the value of the estimated tempo ω _target in one previous frame, the estimated tempo ω _target to 2 times the value of the value of the provisional tempo _{ω in} Update. This suppresses erroneous estimation that the tempo of the current frame is twice or half the true tempo.

When the beat phase θ is included in the region A or the region B in the frame t _i in which the beat event flag BF is “1”, the beat phase θ should be “0”. Therefore, the beat phase θ is corrected. However, instead of correcting the beat phase θ by the correction amount Δθ in the frame t _i , the beat phase θ is corrected little by little over the frame t _i and a plurality of subsequent frames.

  In the area C, the beat event flag BF is highly likely to be set to “1” in a plurality of frames that are continuous or close to each other. If the beat phase θ is corrected when the beat phase θ is included in the region C, the beat point estimation accuracy may be lowered. Therefore, in this case, the beat phase θ is not corrected. Accordingly, it is possible to suppress erroneous estimation that beat points exist in a plurality of frames having a beat phase θ close to “0”.

  Further, when the beat phase θ is included in the region D and the beat event flag BF is set to “1”, there is a high possibility that a shift of half a beat has occurred. Therefore, in this case, the beat phase θ is not corrected.

However, the temporary tempo ω _in may be “M” times the estimated tempo ω _target (M is a natural number equal to or greater than “2”). Therefore, when the beat phase θ is included in the region E in the frame t _i in which the beat event flag BF is “1”, the beat phase θ is multiplied by “M”, and the phase angle is set to “−π (rad)” to Wrapping within the range of “π (rad)”. A value obtained by multiplying the lapped phase angle by “1 / M” is set as a correction amount Δθ of the beat phase θ.

  Thereby, it can suppress that a back beat is mistakenly estimated to be a table beat. Further, it is possible to suppress erroneous estimation that a beat point exists in a plurality of frames near the true beat point. Therefore, according to the acoustic signal analyzer 10 configured as described above, the beat point and the tempo can be accurately estimated in real time.

  Furthermore, in carrying out the present invention, the present invention is not limited to the above embodiment, and various modifications can be made without departing from the object of the present invention.

  For example, the method of temporarily determining the presence / absence of a beat and the tempo is not limited to the above embodiment. Any technique can be adopted as long as it applies a forward algorithm to a probability model using discrete state variables to provisionally determine the presence / absence of a beat and the tempo.

  In the above embodiment, the step S17 for advancing the beat phase θ by the phase angle corresponding to the internal tempo ω is executed after the beat phase correction process, but is executed between the tempo correction process and the beat phase correction process. May be.

DESCRIPTION OF SYMBOLS 10 ... Acoustic signal analyzer, 12 ... Computer part, 13 ... Display, 16 ... Sound system, A, B, C, D, E ... area | region, b ... Beat period , BF · · · beat event flag, M · · · ratio, n · · · frame _{number, q b,} n ··· _{state, t} i · · · frames, [Delta] [theta] · · · correction amount, zeta · · · coefficient , Θ ... beat phase, ρ ... coefficient, ω ... internal tempo, ω _in ... temporary tempo, ω _target ... estimated tempo, ω _thres ... threshold

Claims (5)

By analyzing the captured acoustic signal each time an acoustic signal representing a performance sound of each section when the entire music is divided into a plurality of sections, the existence and tempo of beats in each section of the music are measured in real time. An acoustic signal analyzer for estimating,
Beat points and tempos that temporarily determine the presence and tempo of beats in the current section based on a probability model described as a series of states classified by combinations of physical quantities related to the presence of beats and physical quantities related to the tempo in each section of the music Provisional decision means,
Tempo determination means for correcting the tentatively determined tempo based on the tempo determined in the past interval, and determining the tempo of the current interval based on the corrected tempo;
Beat phase calculation means for calculating a beat phase corresponding to the time from the current section to the beat point based on the determined tempo;
In a section that is temporarily determined to have a beat and the beat phase is included in a predetermined range and one or more of the subsequent sections, the absolute value of the beat phase is decreased by a predetermined correction value. Beat phase correction means for correcting
Beat point / tempo output means for determining that a beat is present in a section in which the sign of the beat phase has changed, and outputting the determined beat point together with the tempo determined by the tempo determination means; Acoustic signal analyzer. The acoustic signal analyzer according to claim 1,
The tempo determination means includes
When the difference between the tempo determined in the past section and the tempo twice the tentatively determined tempo is smaller than a predetermined threshold, the tempo of the current section is based on the tempo twice the tempo determined. Decide
When the difference between the tempo determined in the past interval and half of the tentatively determined tempo is smaller than a predetermined threshold, the tempo of the current interval is determined based on the tempo that is half of the temporarily determined tempo And
The difference between the tempo determined in the past interval and the tempo twice the tentatively determined tempo is equal to or greater than a predetermined threshold, and the tempo determined in the past interval and half of the tentatively determined tempo An acoustic signal analyzing apparatus that determines a tempo of a current section based on the temporarily determined tempo when a difference from a tempo is equal to or greater than a predetermined threshold. In the acoustic signal analyzer according to claim 1 or 2,
The beat phase correcting means is
Correction amount determining means for determining a phase angle to be corrected in a section that is temporarily determined that a beat is present and the beat phase is included in a predetermined range;
An absolute value of the phase angle to be corrected is an interval that is provisionally determined that a beat is present and the beat phase is included in a predetermined range and one or more of the subsequent sections. Correction amount update means for updating the phase angle to be corrected so as to reduce only the correction value,
When the beat phase correction is started from a section where the beat phase is provisionally determined and the beat phase is included in the predetermined range, and the phase angle to be corrected is equal to or smaller than the predetermined phase angle An acoustic signal analyzer that finishes correcting the beat phase. By analyzing the captured acoustic signal each time an acoustic signal representing a performance sound of each section when the entire music is divided into a plurality of sections, the existence and tempo of beats in each section of the music are measured in real time. An acoustic signal analysis method for estimating,
Beat points and tempos that temporarily determine the presence and tempo of beats in the current section based on a probability model described as a series of states classified by combinations of physical quantities related to the presence of beats and physical quantities related to the tempo in each section of the music A tentative decision step;
A tempo determination step of correcting the tentatively determined tempo based on the tempo determined in the past interval, and determining the tempo of the current interval based on the corrected tempo;
A beat phase calculating step for calculating a beat phase corresponding to the time from the current interval to the beat point based on the determined tempo;
In a section that is temporarily determined to have a beat and the beat phase is included in a predetermined range and one or more of the subsequent sections, the absolute value of the beat phase is decreased by a predetermined correction value. A beat phase correction step for correcting
A beat point / tempo output step of determining that a beat exists in a section in which the sign of the beat phase has changed, and outputting the determined beat point together with the tempo determined in the tempo determination step. Acoustic signal analysis method. By analyzing the captured acoustic signal each time an acoustic signal representing a performance sound of each section when the entire music is divided into a plurality of sections, the existence and tempo of beats in each section of the music are measured in real time. A computer program applied to an acoustic signal analyzer to be estimated, the computer included in the acoustic signal analyzer,
Beat points and tempos that temporarily determine the presence and tempo of beats in the current section based on a probability model described as a series of states classified by combinations of physical quantities related to the presence of beats and physical quantities related to the tempo in each section of the music A tentative decision step;
A tempo determination step of correcting the tentatively determined tempo based on the tempo determined in the past interval, and determining the tempo of the current interval based on the corrected tempo;
A beat phase calculating step for calculating a beat phase corresponding to the time from the current interval to the beat point based on the determined tempo;
In a section that is temporarily determined to have a beat and the beat phase is included in a predetermined range and one or more of the subsequent sections, the absolute value of the beat phase is decreased by a predetermined correction value. A beat phase correction step for correcting
Determined as a section beats the sign of the beat phase is changed is present, perform the determined beat positions, and a beat point tempo output step of outputting together with the tempo determined in the tempo determination step acoustic signal minute 析Pu program to be.

Images