JP2017067903A - Acoustic analysis device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To specify an utterance section immediately after the utterance of the utterance source from among acoustic signals with high accuracy.SOLUTION: An utterance section detection part 40 is an element for analyzing an utterance start point TS where the utterance of the acoustic sound is started and an utterance end point TE where the utterance of the acoustic sound is finished from among the acoustic signals XA, and includes: a start point analysis part 44 for specifying a maximum point of intensity of the acoustic sound XE as the utterance start point TS; and an end point analysis part 46 for specifying a minimum point where intensity of the acoustic signal XE is turned oppositely to the increase in the course of decreasing the intensity after the lapse of the utterance start point TS with time, as the utterance end point in the case where a fluctuation index according to a difference between the intensity at the maximum point after the increase and the minimum value of the intensity until the maximum point exceeds a final point threshold value.SELECTED DRAWING: Figure 4

Description

  The present invention relates to a technique for analyzing sound.

  Conventionally, various techniques for analyzing a sound generation section in which sound is generated in an acoustic signal have been proposed. For example, Patent Document 1 discloses a configuration in which a period in which an SN (Signal to Noise) ratio of an acoustic signal satisfies a predetermined condition is specified as a sounding section.

JP 2008-170806 A

  By the way, it is important to specify the sound generation section of the musical instrument for the analysis of sound produced by various musical instruments such as percussion instruments. However, when the sound source is sounded a plurality of times at short intervals, for example, when the percussion instrument is struck quickly, the immediate sounding starts before the sound due to the first sound is sufficiently attenuated. Therefore, under the technique of Patent Document 1, a plurality of pronunciations may be included in one pronunciation period. However, in the actual performance sound analysis scene, it is assumed that the analysis of the characteristics immediately after the start of sounding is important, so even if the sound source is sounded multiple times at short intervals, the sound of the first sounding It is important to specify a pronunciation interval including only high precision. In view of the above circumstances, an object of the present invention is to specify a sound generation section immediately after sound generation of a sound source in an acoustic signal with high accuracy.

  In order to solve the above problems, an acoustic analysis device according to a preferred aspect of the present invention is an acoustic analysis device that analyzes a sound generation start point at which sound generation is started and a sound generation end point at which sound generation ends in the sound signal. An analysis device that includes a start point analysis unit that identifies a maximum point of the intensity of an acoustic signal as a pronunciation start point, and a minimum point at which the intensity reverses to increase in the process of decreasing the intensity of the acoustic signal over time after the start of the sound generation point. And an end point analysis unit that specifies the end point of sound generation when the variation index according to the difference between the intensity at the maximum point after the increase and the minimum value of the intensity up to the maximum point exceeds the end point threshold value. In the above aspect, the minimum point where the intensity reverses to increase in the process in which the intensity of the acoustic signal decreases with time after the sounding start point has elapsed is specified as the sounding end point when the variation index exceeds the end point threshold. In other words, when the sound source is sounded multiple times at short intervals (when the sound immediately after the start sound is sufficiently attenuated), only the first sound corresponding to the sound start point is generated. The pronunciation end point is specified to include. Therefore, it is possible to specify the sound generation section immediately after the sound generation of the sound source in the acoustic signal with high accuracy.

  In a preferred aspect of the present invention, the end point analysis unit, when the minimum point is specified as the pronunciation end point, when the intensity of the acoustic signal decreases from the sound start point until it falls below the attenuation threshold corresponding to the intensity at the sound start point. The point in time when the intensity falls below the attenuation threshold is specified as the pronunciation end point. In the above aspect, when the intensity of the acoustic signal decreases from the sounding start point until the minimum point is identified as the sounding end point and falls below the attenuation threshold corresponding to the intensity at the sounding start point, Is identified as the pronunciation end point. Accordingly, there is an advantage that an appropriate sounding end point can be set according to the degree of attenuation from the sounding start point when the sound signal is attenuated without sounding by the sounding source after the sounding start point has elapsed.

  In a preferred aspect of the present invention, the start point analysis unit sequentially detects the maximum point of the intensity of the acoustic signal, while the variation index according to the difference between the intensity at the maximum point and the minimum value of the intensity up to the maximum point. Is greater than the start point threshold value greater than the end point threshold value, the local maximum point is specified as the pronunciation start point. In the above aspect, the maximum point of the intensity of the acoustic signal is sequentially detected, while the variation index according to the difference between the intensity at the maximum point and the reference value that is the minimum value of the intensity up to the maximum point is the starting point threshold value. When the value exceeds the maximum value, the local maximum point is specified as the pronunciation starting point. Therefore, there is an advantage that it is possible to specify with high accuracy the clear start of the sound source of the sound source among the plurality of maximum points detected from the acoustic signal.

  In a preferred aspect of the present invention, the variation index is a numerical value obtained by dividing the difference between the intensity at the maximum point and the minimum value of the intensity up to the maximum point by the intensity at the maximum point. In the above aspect, the variation index is calculated by dividing the difference between the intensity at the maximum point and the reference value by the intensity at the maximum point. That is, the difference is normalized to a numerical value that does not depend on the volume of the sound signal. Therefore, it is possible to appropriately specify the sound generation start point and the sound generation end point regardless of the volume of the acoustic signal.

  In a preferred aspect of the present invention, when the start point analysis unit detects a second maximum point having an intensity exceeding the first maximum point in a standby section after the first maximum point of the intensity of the acoustic signal, the first maximum is detected. The point is excluded from the pronunciation start point candidates. In the above aspect, when a maximum point with an intensity exceeding the maximum point is detected in the standby section after the maximum point of the intensity of the acoustic signal, the maximum point is excluded from the pronunciation start point candidates. Therefore, even when multiple local maximum points are detected in the process of increasing the intensity of the sound signal from the start of a single sound generation by the sound source, the sound generation section including one local maximum point corresponding to the sound generation is appropriately specified. Is possible.

1 is a configuration diagram of a sound processing apparatus according to a first embodiment of the present invention. It is a block diagram of an acoustic analysis part. It is explanatory drawing of each sound generation area of an acoustic signal. It is a block diagram of a pronunciation area detection part. It is explanatory drawing of operation | movement of a pronunciation area detection part. It is a flowchart of a starting point analysis process. It is a flowchart of an end point analysis process. It is a block diagram of a sound source identification part. It is a flowchart of a harmonic analysis process. It is a flowchart of a sound source identification process. It is explanatory drawing of the starting point analysis process in 2nd Embodiment.

<First Embodiment>
FIG. 1 is a configuration diagram of the sound processing apparatus 12 according to the first embodiment of the present invention. As illustrated in FIG. 1, a plurality of sound collection devices 14 and sound emission devices 16 are connected to the sound processing device 12. Each of the plurality of sound collection devices 14 generates an acoustic signal XA representing the sound around the sound collection device 14. The acoustic signal XA is, for example, a stereo signal with two channels on the left and right. A plurality of acoustic signals XA generated by the plurality of sound collecting devices 14 are supplied to the acoustic processing device 12 in parallel. The A / D converter for converting the acoustic signal XA generated by the sound collection device 14 from analog to digital is not shown for convenience.

  Each sound collecting device 14 is arranged in the vicinity of a different sound source. The sound source is, for example, a musical instrument that produces a musical tone or a singer that produces a singing voice. In the first embodiment, it is assumed that music is played by a singer and a plurality of musical instruments inside an acoustic space such as a recording studio. The sound signal XA generated by each sound collection device 14 predominately contains sound produced from a sound source in the vicinity of the sound collection device 14, but other sound sources with a lower volume than the sound. Can also be included.

  Each sound source in the first embodiment generates a harmonic sound or a non-harmonic sound. The harmonic sound is a harmonic sound in which the harmonic structure in which the fundamental frequency component of the fundamental frequency and a plurality of harmonic components are arranged on the frequency axis is clearly observed. For example, a musical sound of a harmonic instrument such as a stringed instrument or a wind instrument or a human vocal sound such as a singing voice is a typical example of a harmonic sound. On the other hand, non-harmonic sound is non-harmonic sound in which the harmonic structure is not clearly observed. For example, percussion musical sounds such as drums and cymbals are typical examples of non-harmonic sounds.

  Note that the harmonic sound means sound containing a harmonic acoustic component predominantly compared to a non-harmonic acoustic component. Therefore, in addition to sound composed only of harmonic acoustic components, there is also acoustic that contains both harmonic acoustic components and non-harmonic acoustic components, but the harmonics predominate as a whole. Included in the concept of harmonic sounds. Similarly, non-harmonic sound refers to sound that predominately contains non-harmonic acoustic components compared to harmonic acoustic components. Therefore, it contains both harmonic and non-harmonic acoustic components in addition to the sound composed only of non-harmonic acoustic components, but the non-harmonic property is dominant as a whole. Sound is also included in the concept of non-harmonic sound. In the following description, the subscript H (H: Harmonic) may be added to the code of the element related to the harmonic sound, and the subscript P (P: Percussive) may be added to the code of the element related to the non-harmonic sound.

  The sound processing device 12 generates the sound signal XB by sound processing on the plurality of sound signals XA. Specifically, the sound processing device 12 according to the first embodiment generates a left and right two-channel stereo sound signal XB by mixing (mixing) a plurality of sound signals XA. The sound emitting device 16 (for example, a speaker or headphones) emits sound according to the acoustic signal XB generated by the sound processing device 12. In addition, illustration of the D / A converter which converts the acoustic signal XB which the acoustic processing apparatus 12 produced | generated from digital to analog was abbreviate | omitted for convenience. In FIG. 1, each sound collecting device 14 and sound emitting device 16 are illustrated as separate elements from the sound processing device 12, but a plurality of sound collecting devices 14 and sound emitting devices 16 are mounted on the sound processing device 12. It is also possible to do.

  As illustrated in FIG. 1, the sound processing device 12 is realized by a computer system including a control device 122 and a storage device 124. The storage device 124 is a known recording medium such as a magnetic recording medium or a semiconductor recording medium, or a combination of a plurality of types of recording media, and stores a program executed by the control device 122 and various data used by the control device 122. . The control device 122 executes a program stored in the storage device 124, thereby analyzing each of the plurality of acoustic signals XA, and using the analysis result by the acoustic analysis unit 20, the plurality of acoustic signals XA. The acoustic processing unit 30 that generates the acoustic signal XB from the above is realized. A configuration in which some or all of the functions of the control device 122 are realized by a dedicated electronic circuit, or a configuration in which the functions of the control device 122 are distributed to a plurality of devices may be employed.

  The acoustic analysis unit 20 specifies, for each of the plurality of acoustic signals XA supplied from the plurality of sound collection devices 14, the type of sound generation source represented by the acoustic signal XA. Specifically, the acoustic analysis unit 20 generates information (hereinafter referred to as “sound source identification information”) D indicating the type of sound source of each acoustic signal XA. The sound source identification information D is, for example, the name of the sound source (specifically, the instrument name or performance part name).

  FIG. 2 is a configuration diagram of the acoustic analysis unit 20. As illustrated in FIG. 2, the acoustic analysis unit 20 of the first embodiment includes a sounding section detection unit 40, a feature amount extraction unit 50, and a sound source identification unit 60. In the following description, although attention is paid to the processing for an arbitrary one system of acoustic signals XA for convenience, the same processing is executed for each of the plurality of acoustic signals XA.

  The sounding section detection unit 40 in FIG. 2 detects a plurality of sounding sections P for the acoustic signal XA. FIG. 3 shows the relationship between the waveform of the acoustic signal XA and the sound generation section P. As can be understood from FIG. 3, each sound generation section P is a time-axis section where the sound represented by the sound signal XA is generated, and from the point in time when sound generation starts (hereinafter referred to as “pronunciation start point”) TS to the end point This is a section up to TE (hereinafter referred to as “pronunciation end point”).

  The feature quantity extraction unit 50 in FIG. 2 extracts the feature quantity F of the acoustic signal XA. The feature amount extraction unit 50 according to the first embodiment sequentially extracts the feature amount F for each sound generation section P detected by the sound generation section detection unit 40. The feature amount F is an index representing the acoustic feature of the acoustic signal XA in the sound generation section P. The feature amount F of the first embodiment is expressed as a vector including a plurality of different characteristic values f (f1, f2,...). Specifically, the MFCC (Mel-frequency cepstral coefficients) representing the timbre of the acoustic signal XA, the steepness of the acoustic rise in the sound generation section P, the intensity ratio of the harmonic component to the fundamental component, and the sign of the intensity of the acoustic signal XA A plurality of types of characteristic values f such as the number of inversions or the number of zero crossings, which is the frequency, are included in the feature amount F.

  The characteristics of the sound generated by each sound source are particularly prominent immediately after the sound generation start point TS. In the first embodiment, since the feature amount F of the acoustic signal XA is extracted for each sounding start point TS (for each sounding section P) of the sound signal XA, the section in which the sound signal XA is divided regardless of the presence or time of sounding. Compared with the configuration in which the feature amount F is extracted every time, there is an advantage that the feature amount F in which a unique feature is significantly reflected for each type of sound source can be extracted. However, it is also possible to extract the feature amount F for each section obtained by dividing the acoustic signal XA on the time axis regardless of whether or not the sound source is sounded and the time point (therefore, the sounding section detection unit 40 is omitted). . The sound source identification unit 60 generates the sound source identification information D by identifying the type of the sound source of the acoustic signal XA using the feature amount F extracted by the feature amount extraction unit 50.

  The sound processing unit 30 in FIG. 1 generates the sound signal XB by performing sound processing on the plurality of sound signals XA with reference to the sound source identification information D analyzed by the sound analysis unit 20 for each sound signal XA. Specifically, acoustic processing set in advance for each type of sound source indicated by the sound source identification information D of the acoustic signal XA is executed on the acoustic signal XA. As acoustic processing for the acoustic signal XA, for example, effect imparting processing (effector) that imparts various acoustic effects such as reverberation effect and distortion effect, characteristic adjustment processing (equalizer) that adjusts the volume for each frequency band, and sound image localization Examples are a localization adjustment process (pan) for adjusting the position to be performed and a volume adjustment process for adjusting the volume. The type and degree of the sound effect given to the sound signal XA by the effect applying process, the frequency characteristic given to the sound signal XA by the characteristic adjusting process, the position of the sound image adjusted by the localization adjusting process, and the adjustment contents by the volume adjusting process ( Various parameters such as gain) are individually set for each type of sound source indicated by the sound source identification information D. Then, the sound processing unit 30 generates the sound signal XB by mixing (mixing) the plurality of sound signals XA after the sound processing exemplified above. That is, the acoustic processing unit 30 according to the first embodiment realizes automatic mixing that reflects the sound source identification result by the harmonic analysis unit 62. Hereinafter, a specific configuration of each of the sounding section detection unit 40 and the sound source identification unit 60 in the first embodiment will be described.

<Sound generation section detector 40>
FIG. 4 is a configuration diagram of the sounding section detection unit 40. As illustrated in FIG. 4, the sounding section detection unit 40A of the first embodiment includes a signal processing unit 42, a start point analysis unit 44, and an end point analysis unit 46. In the following description, attention is paid to the processing for an arbitrary one-system acoustic signal XA for the sake of convenience, but actually the same processing is executed for each of the plurality of acoustic signals XA.

  The signal processing unit 42 generates an acoustic signal XE by signal processing of the acoustic signal XA supplied from the sound collection device 14. The acoustic signal XE corresponds to an envelope (envelope) on the time axis of the acoustic signal XA. Specifically, the signal processing unit 42 generates the acoustic signal XE by converting each signal value of the acoustic signal XA into an absolute value and then suppressing (smoothing) the high frequency component. The waveform of the acoustic signal XE is illustrated in FIG. In the configuration in which the acoustic signal XE generated by the external device is supplied to the acoustic processing device 12, the signal processing unit 42 can be omitted from the acoustic processing device 12.

  The start point analysis unit 44 in FIG. 4 specifies a sound generation start point TS at which sound generation is started in the sound signal XE. The end point analysis unit 46 specifies a sound generation end point TE at which sound generation ends in the sound signal XE. In the first embodiment, the specification of the sound generation start point TS by the start point analysis unit 44 and the specification of the sound generation end point TE by the end point analysis unit 46 are timed from the start point of the sound signal XE in real time in parallel with the generation of the sound signal XE. It is executed sequentially with the passage of time. The operations of the start point analysis unit 44 and the end point analysis unit 46 will be described below.

<Start point analysis unit 44>
As illustrated in FIG. 5, the start point analysis unit 44 of the first embodiment specifies the maximum point (peak) xH where the intensity (amplitude or power) Q of the acoustic signal XE reverses from increase to decrease as the sound generation start point TS. . However, the start point analysis unit 44 of the first embodiment does not set all the local maximum points xH detected from the acoustic signal XE as the sound generation start point TS, but is a predetermined one of a plurality of local maximum points xH detected from the acoustic signal XE. The local maximum point xH that satisfies the above condition is selectively specified as the pronunciation start point TS.

  Specifically, the start point analysis unit 44 responds to the difference (QH−QREF) between the intensity QH of the acoustic signal XE at the local maximum point xH and the reference value QREF as the local maximum point xH1 illustrated in FIG. When the variation index δ exceeds a predetermined threshold value (hereinafter referred to as “starting point threshold value”) ZS (δ> ZS), the local maximum point xH is determined as the pronunciation starting point TS. On the other hand, like the local maximum point xH0 illustrated in FIG. 5, the local maximum value xH whose fluctuation index δ is lower than the starting point threshold value ZS is not set as the pronunciation starting point TS.

  The reference value QREF is updated as the analysis process of the sound generation start point TS progresses so that it becomes the minimum value of the intensity Q of the sound signal XE after the immediately preceding sound generation start point TS (immediately after the start of processing is the start point of the sound signal XE). The The variation index δ is, for example, a numerical value (δ = (QH−QREF) / QH) obtained by dividing the difference (QH−QREF) between the intensity QH at the local maximum point xH and the reference value QREF by the intensity QH. By dividing by the intensity QH, the variation index δ is normalized to a numerical value that does not depend on the overall volume of the acoustic signal XE. The starting point threshold value ZS is a predetermined positive number selected in advance.

  FIG. 6 is a flowchart of processing (hereinafter referred to as “start point analysis processing”) in which the start point analysis unit 44 specifies the pronunciation start point TS. The start point analysis unit 44 sequentially detects the maximum point xH from the start point of the acoustic signal XE, and starts the start point analysis process of FIG. 6 each time the maximum point xH is detected.

  When starting point analysis processing is triggered by the detection of the local maximum point xH of the acoustic signal XE, the starting point analysis unit 44 responds to the difference (QH−QREF) between the intensity QH at the local maximum point xH and the current reference value QREF. It is determined whether or not the variation index δ exceeds the starting point threshold value ZS (SC1). When the variation index δ is less than the start point threshold value ZS (SC1: NO), the start point analysis unit 44 ends the start point analysis process without specifying the current local maximum point xH as the pronunciation start point TS. On the other hand, when the variation index δ exceeds the start point threshold value ZS (SC1: YES), the start point analysis unit 44 specifies the current local maximum point xH as the pronunciation start point TS (SC2). Then, the start point analysis unit 44 updates the reference value QREF to the intensity QH at the current local maximum point xH (SC3). Since the sound signal XE attenuates when the sound generation start point TS elapses, the reference value QREF decreases with time after the sound generation start point TS elapses. The above is a preferred example of the starting point analysis process.

<End point analysis unit 46>
As described above, the end point analysis unit 46 in FIG. 4 identifies the sound generation end point TE at which sound generation ends in the sound signal XE. FIG. 7 is a flowchart of processing (hereinafter referred to as “end point analysis processing”) in which the end point analysis unit 46 specifies the pronunciation end point TE. The end point analysis process in FIG. 7 is started when the start point analysis unit 44 specifies the pronunciation start point TS (SC2).

  When the end point analysis processing is started in response to the specification of the pronunciation start point TS, the end point analysis unit 46 determines whether or not a predetermined time τ has elapsed from the pronunciation start point TS (SD1). If the predetermined time τ has not elapsed since the sound generation start point TS (SD1: NO), the end point analysis unit 46 determines whether the current intensity Q of the acoustic signal XE falls below a predetermined threshold (hereinafter referred to as “attenuation threshold”) Z0. It is determined whether or not (SD2). The attenuation threshold Z0 is set to a numerical value corresponding to the intensity QH of the acoustic signal XE at the immediately preceding sounding start point TS. Specifically, a numerical value obtained by multiplying the intensity QH at the pronunciation start point TS by a positive number less than 1 (for example, an arbitrary numerical value of 0.4 to 0.6) is suitable as the attenuation threshold Z0. When the intensity Q is lower than the threshold value Z0 (SD2: YES), the end point analysis unit 46 specifies the current time point as the pronunciation end point TE (SD3). In other words, the point in time when the intensity Q of the acoustic signal XE decreases after the sounding start point TS falls below the attenuation threshold Z0 is specified as the sounding end point TE.

  By the way, when the sound source is sounded a plurality of times at short intervals, for example, when a percussion instrument is quickly repeatedly struck, the immediately following sounding starts before the sound of the first sounding is sufficiently attenuated. Therefore, only by specifying the time point when the intensity Q of the acoustic signal XE falls below the attenuation threshold value Z0 as the sounding end point TE, the sound source of the sound source is included in one sounding period P from the sounding start point TS to the sounding end point TE. Result. However, in the scene of analysis of the acoustic signal XA such as the extraction of the feature amount F by the feature amount extraction unit 50 and the identification of the type of the sound source by the sound source identification unit 60, it is important to analyze the characteristics immediately after the start of the sound source. It is. Considering the above circumstances, the end point analysis unit 46 of the first embodiment includes only the first pronunciation corresponding to the pronunciation start point TS even if the pronunciation source is pronounced multiple times at short intervals. As described above (that is, so that the second and subsequent pronunciations are not included in the pronunciation period P), the pronunciation end point TE is specified.

  Specifically, when the intensity Q of the acoustic signal XE exceeds the attenuation threshold Z0 (SD2: NO), the end point analysis unit 46 determines the minimum point at which the intensity Q of the acoustic signal XE reverses from decreasing to increasing after the sounding start point TS. (Dip) It is determined whether xL is detected (SD4). When the minimum point xL is not detected (SD4: NO), the end point analysis unit 46 proceeds to step SD1 to determine whether the time τ has elapsed from the pronunciation start point TS (SD1: YES) or the intensity Q of the acoustic signal XE is Until the attenuation threshold value Z0 falls below (SD2: YES), the occurrence of the minimum point xL is monitored.

  On the other hand, if the local minimum point (hereinafter referred to as “target local minimum point”) xL is detected before the intensity Q of the acoustic signal XE falls below the attenuation threshold value Z0 (SD4: YES), the end point analysis unit 46 It is determined whether or not the maximum point xH immediately after xL has been detected (SD5). When the local maximum point xH is not detected (SD5: NO), the end point analysis unit 46 shifts the processing to step SD1. If the target minimum point xL is detected (SD4: YES) and the intensity Q at the target minimum point xL is lower than the current reference value QREF (Q <QREF), the reference value QREF is the target minimum point. Updated to strength Q at xL. That is, as described above, the reference value QREF is updated so as to be the minimum value of the intensity Q after the sounding start point TS (for example, the intensity Q at the target minimum point xL).

  FIG. 5 illustrates a local maximum point xH2 immediately after the target local minimum point xL. When the local maximum point xH2 is detected (SD5: YES), the end point analysis unit 46 calculates the variation index δ according to the difference (QH−QREF) between the intensity QH at the local maximum point xH2 and the current reference value QREF. It is determined whether or not the end point threshold value ZE is exceeded (SD6). As described above, the variation index δ is a numerical value obtained by dividing the difference (QH−QREF) between the strength QH and the reference value QREF by the strength QH. It is highly possible that the current reference value QREF is the intensity Q at the target minimum point xL. The end point threshold value ZE is set to a predetermined positive number lower than the above-described start point threshold value ZS used for specifying the sound generation start point TS (ZE <ZS).

  When the variation index δ is lower than the end point threshold value ZE (SD6: NO), the maximum point xH is observed immediately after the target minimum point xL, but the intensity Q increases due to the pronunciation of the pronunciation source immediately after the pronunciation start point TS (No. 1). Cannot be estimated until the second and subsequent pronunciations. Therefore, it is necessary to continuously monitor the intensity Q of the acoustic signal XE without determining the pronunciation end point TE yet. Therefore, the end point analysis unit 46 shifts the process to step SD1 until the time τ elapses from the sound generation start point TS (SD1: YES) or until the intensity Q of the acoustic signal XE falls below the attenuation threshold Z0 (SD2: YES). ), Monitoring the occurrence of the minimum point xL.

  On the other hand, when the intensity Q of the maximum point xH increases as the variation index δ exceeds the end point threshold value ZE (SD6: YES), the maximum point xH immediately after the target minimum point xL is the sound source immediately after the sound generation start point TS. It is presumed that the intensity Q is increased due to the pronunciation (ie, the second and subsequent pronunciations immediately after the first pronunciation). Therefore, it is necessary to determine from the sounding start point TS to the target local minimum point xL as the sounding section P, and to exclude the local maximum point xH corresponding to the second and subsequent sounding from the sounding section P. Therefore, the end point analysis unit 46 specifies the target minimum point xL as the pronunciation end point TE (SD7). That is, when the variation index δ exceeds the end point threshold value ZE at the local maximum point xH immediately after the target local minimum point xL, the target local minimum point xL is subsequently determined as the pronunciation end point TE.

  As understood from the above description, the end point analysis unit 46 of the first embodiment varies the target minimum point xL detected in the process in which the intensity Q of the acoustic signal XE decreases with time after the sound generation start point TS elapses. When the index δ exceeds the end point threshold value ZE (SD6: YES), the sound generation end point TE is specified (SD7), and when the variation index δ is less than the end point threshold value ZE (SD6: NO), the sound generation end point TE is not set. Note that the local maximum point xH2 of FIG. 5 detected immediately after the target local minimum point xL is specified as the pronunciation start point TS on the condition that the variation index δ exceeds the start point threshold value ZS, as described with reference to FIG. The When the variation index δ exceeds the start point threshold value Z S, the end point threshold value ZE naturally also exceeds, so the target minimum point x L immediately before the maximum point x H is determined as the pronunciation end point TE.

  On the other hand, the intensity Q of the acoustic signal XE falls below the attenuation threshold Z0 (SD2: YES), or the minimum point xL after the elapse of the sounding start point TS is not specified as the sounding end point TE (SD7). When the time τ elapses from the start point TS (SD1: YES), the end point analysis unit 46 specifies the time point when the time τ elapses from the sound generation start point TS as the sound generation end point TE (SD8). As understood from the above description, the end point analysis unit 46 basically specifies the time point when the intensity Q of the acoustic signal XE falls below the attenuation threshold value Z0 as the sounding end point TE (SD3), but immediately after the sounding start point TS. When the pronunciation of the pronunciation source is estimated (SD6: YES), the minimum point xL is determined as the pronunciation end point TE so that the pronunciation is excluded from the pronunciation period P (SD7), and none of the conditions is satisfied Is specified as a sound generation end point TE (SD8).

  As described above, in the first embodiment, the minimal point xL where the intensity Q reverses to increase in the process in which the intensity Q of the acoustic signal XE decreases with time after the sound generation start point TS elapses, and the variation index δ is the end point threshold value. If it exceeds ZE, it is specified as the pronunciation end point TE. In other words, when the sound source is sounded multiple times at short intervals (when the sound immediately after the sound of the first sound begins to decay before the sound is sufficiently attenuated), only the first sound corresponding to the sound start point TS is sounded. The pronunciation end point TE is specified so as to be included in the section P. Therefore, it is possible to specify the interval immediately after the sound generation important for the analysis of the acoustic signal XA as the sound generation interval P with high accuracy. For the sound source identification by the sound source identification unit 60, the characteristic immediately after the sound generation in which the difference for each type of sound source becomes significant is particularly important. Therefore, the first embodiment that can specify the section immediately after the sound generation as the sound generation section P with high accuracy is particularly suitable.

  Further, in the first embodiment, before the arrival of the minimum point xL where the variation index δ exceeds the end point threshold value ZE, the intensity Q of the acoustic signal XE is reduced to the sounding start point until it falls below the attenuation threshold value Z0 corresponding to the intensity QH at the sounding start point TS. When it decreases compared to TS (SD2: YES), the time point when the intensity Q falls below the attenuation threshold value Z0 is specified as the sound generation end point TE. Therefore, when the sound signal XE attenuates without sound generation after the sound generation start point TS has elapsed, there is an advantage that an appropriate sound generation end point TE can be set according to the degree of attenuation from the sound generation start point TS.

  In the first embodiment, the maximum point xH of the intensity Q of the acoustic signal XE is sequentially detected, while the difference between the intensity QH at the maximum point xH and the reference value QREF that is the minimum value of the intensity Q up to the maximum point xH. When the variation index δ according to (QH−QREF) exceeds the start point threshold value ZS, the local maximum point xH is specified as the sound generation start point TS. Therefore, there is an advantage that the start of clear sound generation of the sound source among the plurality of maximum points xH detected from the acoustic signal XE can be specified with high accuracy as the sound start point TS.

  Further, the variation index δ is obtained by dividing the difference (QH−QREF) between the intensity QH at the maximum point xH and the reference value QREF that is the minimum value of the intensity Q up to the maximum point xH by the intensity QH at the maximum point xH. Is calculated. That is, the difference (QH−QREF) is normalized to a numerical value that does not depend on the volume of the acoustic signal XE. Therefore, it is possible to appropriately specify the sound generation start point TS and the sound generation end point TE regardless of the volume of the acoustic signal XE.

<Sound source identification unit 60>
FIG. 8 is a configuration diagram of the sound source identification unit 60 of the first embodiment. As illustrated in FIG. 8, the sound source identification unit 60 of the first embodiment includes a harmonic analysis unit 62, a first analysis unit 64, a second analysis unit 66, and a sound source identification unit 68.

  The harmonic analysis unit 62 analyzes whether the sound represented by the sound signal XA (hereinafter referred to as “target sound”) corresponds to a harmonic sound or a non-harmonic sound from the feature value F of the sound signal XA. The harmonic analysis unit 62 of the first embodiment calculates the accuracy WH (first accuracy) that the target sound corresponds to the harmonic sound and the accuracy WP (second accuracy) that the target sound corresponds to the non-harmonic sound.

  Specifically, a known pattern recognizer that discriminates between harmonic and non-harmonic sounds by analyzing the feature value F is arbitrarily used as the harmonic analysis unit 62. In the first embodiment, a support vector machine (SVM) which is a representative example of a statistical model using supervised learning is exemplified as the harmonic analysis unit 62. That is, the harmonic analysis unit 62 uses the hyperplane determined in advance by machine learning to which a large number of acoustic learning data including harmonic and non-harmonic sounds is used to generate the target sound of the feature amount F. It is sequentially determined for each feature amount F (for each sounding section P) whether the harmonic sound or the non-harmonic sound is applicable. Then, the harmonic analysis unit 62 determines, for example, the ratio of the number of times that the target sound is determined to be harmonic sound within a predetermined period (the number of times determined as harmonic sound / the total number of determinations within the period) of the harmonic sound. While calculating as the accuracy WH, the ratio of the number of times that the target sound is determined to be a non-harmonic sound is calculated as the accuracy WP of the non-harmonic sound (WH + WP = 1). As understood from the above description, the higher the possibility (likelihood) that the target sound of the acoustic signal XA is a harmonic sound, the higher the probability WH, and the higher the possibility that the target sound is a non-harmonic sound. WP is a large number.

  The first analysis unit 64 analyzes from the feature quantity F of the acoustic signal XA whether the sound source of the target sound of the acoustic signal XA corresponds to a plurality of types of harmonic sound sources. The harmonic sound source means a sound source (for example, a harmonic instrument) that generates harmonic sounds. In FIG. 8, four types of bass (Bass), guitar (Guitar), male singer (male Vo.), And female singer (female Vo.) Are illustrated as harmonic sound sources that are candidates for the sound source of the target sound. Has been. Specifically, the first analysis unit 64 of the first embodiment sets the accuracy of the sound source of the target sound corresponding to the harmonic sound source for each of N types (N is a natural number of 2 or more) of harmonic sound sources. The corresponding evaluation value EH (n) (EH (1) to EH (N)) is set.

  FIG. 9 is a flowchart of processing (hereinafter referred to as “harmonic analysis processing”) in which the first analysis unit 64 sets the evaluation values EH (1) to EH (N). The harmonic analysis process shown in FIG. 9 is executed every time the feature amount F is extracted by the feature amount extraction unit 50 (therefore, every sounding section P).

When the harmonic analysis process is started, the first analysis unit 64 selects each of the _two combinations ( _N C ₂ types) of selecting any two types of harmonic sound sources from the N types of harmonic sound sources selected in advance. For which the sound source of the target sound corresponds to which of the two types of harmonic sound sources of the combination, using the feature value F (SA1). For the above discrimination, a support vector machine using two types of harmonic sound sources as discrimination candidates is preferably used. That is, by applying the feature value F to _N C ₂ support vector machines corresponding to combinations of harmonic sound sources, the sound source of the target sound is selected from two types of harmonic sound sources for each combination.

For each of the N types of harmonic sound sources, the first analysis unit 64 calculates the accuracy CH (n) (CH (1) to CH (N)) that the sound source of the target sound corresponds to the harmonic sound source ( SA2). The accuracy CH (n) of an arbitrary one (nth) harmonic sound source is determined, for example, as the sound source of the target sound corresponds to the nth harmonic sound source out of a total of _N C _two determinations. The ratio of the number of times (number of times determined to correspond to a harmonic sound source / _{NC 2} ). As understood from the above description, the probability CH (n) is larger as the probability (likelihood) that the sound source of the target sound of the acoustic signal XA corresponds to the nth harmonic sound source out of N types is higher. It becomes.

  The first analysis unit 64 sets a numerical value (score) corresponding to the rank of the accuracy CH (n) calculated for each harmonic sound source as an evaluation value EH (n) for each of the N types of harmonic sound sources (SA3). ). Specifically, the numerical value according to the order of the accuracy CH (n) is set as the evaluation value EH (n) of each harmonic sound source so that the evaluation value EH (n) becomes a larger value as the accuracy CH (n) increases. Is granted. For example, the evaluation value EH (n) of the harmonic sound source located at the top in the descending order of the accuracy CH (n) is set to a numerical value ε1 (for example, ε1 = 100), and the accuracy CH (n) is located in the second place. The harmonic sound source evaluation value EH (n) is set to a numerical value ε2 (for example, ε2 = 80) lower than the numerical value ε1, and the evaluation value EH (n) of the harmonic sound source with the accuracy CH (n) in the third place is The numerical value ε3 (for example, ε3 = 60) lower than the numerical value ε2 is set, and the evaluation value EH (n) of the remaining harmonic sound source lower than the predetermined rank is set to the minimum value (for example, 0). As understood from the above description, the evaluation value EH (n) becomes larger as the sound source of the target sound of the acoustic signal XA is more likely to correspond to the nth harmonic sound source among the N types. The above is a preferred example of harmonic analysis processing.

  The second analysis unit 66 in FIG. 8 analyzes from the feature quantity F of the acoustic signal XA which of the plural types of non-harmonic sound sources the sound source of the target sound of the acoustic signal XA corresponds to. A non-harmonic sound source means a sound source that produces non-harmonic sounds (for example, a non-harmonic instrument such as a percussion instrument). In FIG. 8, bass drum (Kick), snare drum (Snare), hi-hat (Hi-Hat), floor tom (F-Tom), and cymbal (Cymbal) are non-tones that are candidates for the sound source of the target sound. Illustrated as a wave source. Specifically, in the second analysis unit 66 of the first embodiment, for each of M types (M is a natural number of 2 or more) of non-harmonic sound sources, the sound source of the target sound corresponds to the non-harmonic sound source. An evaluation value EP (m) (EP (1) to EP (M)) corresponding to the accuracy is set. The difference between the number N of harmonic sound sources and the number M of non-harmonic sound sources is not questioned.

The setting (non-harmonic analysis process) of the M evaluation values EP (1) to EP (M) by the second analysis unit 66 is the harmonic analysis process (evaluation value EH by the first analysis unit 64) illustrated in FIG. (Setting (n)). Specifically, the second calculating unit 66, for each of all combinations _(M C ₂ combinations) selecting two kinds from M kinds of non-harmonic sound source, two types of sound sources the combination of the target sound The accuracy CP (m) corresponding to the sound source of the target sound corresponding to the mth non-harmonic sound source is calculated for each non-harmonic sound source. For the discrimination of the non-harmonic sound source, a support vector machine is preferably used as in the case of the harmonic sound source discrimination in the harmonic analysis process.

  Then, the second analysis unit 66 sets a numerical value corresponding to the rank of the accuracy CP (m) as the evaluation value EP (m) for each of the M types of non-harmonic sound sources. The evaluation value EP (m) of the non-harmonic sound source located at an arbitrary rank of the accuracy CP (m) includes the evaluation value EH (n) of the harmonic sound source located at the same rank in the order of the accuracy CH (n). Equivalent numbers are given. Specifically, the evaluation value EP (m) of the non-harmonic sound source positioned at the top in the descending order of the accuracy CP (m) is set to the numerical value ε1, and the non-harmonic signal having the accuracy CP (m) positioned at the second position. The evaluation value EP (m) of the wave source is set to the numerical value ε2, and the evaluation value EP (m) of the non-harmonic sound source with the accuracy CP (m) located at the third place is set to the numerical value ε3. The evaluation value EP (m) of the remaining harmonic sound source below is set to a minimum value (for example, 0). Therefore, the higher the possibility (likelihood) that the sound source of the target sound of the acoustic signal XA corresponds to the mth non-harmonic sound source among the M types, the larger the evaluation value EP (m) becomes.

  As described above, any one feature quantity F extracted from the acoustic signal XA by the feature quantity extraction unit 50 includes a plurality of characteristic values f1 (first characteristic values) and characteristic values f2 (second characteristic values). Of characteristic value f. The first analysis unit 64 of the first embodiment uses the characteristic value f1 of the feature amount F to analyze the accuracy CH (n) corresponding to each of the N types of harmonic sound sources as the sound source of the target sound. On the other hand, the second analysis unit 66 uses the characteristic value f2 of the feature amount F to analyze the accuracy CP (m) in which the sound source of the target sound corresponds to each of the M types of non-harmonic sound sources. That is, the first analysis unit 64 calculates the characteristic amount F (characteristic value f1) used for calculating the harmonic source accuracy CH (n), and the second analysis unit 66 calculates the non-harmonic source accuracy CP (m). This is different from the applied feature amount F (characteristic value f2).

  Specifically, for the calculation of the accuracy CH (n) by the first analysis unit 64, a characteristic value f1 in which the difference is remarkable for each type of harmonic sound source is used. For example, the characteristic value f1 such as the MFCC representing the tone color or the intensity ratio of the harmonic component to the fundamental component is preferably used for calculating the accuracy CH (n) of the harmonic sound. On the other hand, for the calculation of the accuracy CP (m) by the second analysis unit 66, a characteristic value f2 in which the difference is remarkable for each type of non-harmonic sound source is used. For example, the characteristic value f2 such as the steepness of the acoustic rise and the number of zero crossings is preferably used for calculating the accuracy CP (m) of the subharmonic sound. The characteristic value f1 used by the first analysis unit 64 and the characteristic value f2 used by the second analysis unit 66 can be partially shared.

  The sound source specifying unit 68 in FIG. 8 specifies the type of sound source of the acoustic signal XA according to the results of the above analysis by the harmonic analysis unit 62, the first analysis unit 64, and the second analysis unit 66. The type of the sound source is specified for each sound generation section P. As illustrated in FIG. 8, the sound source identification unit 68 of the first embodiment includes a multiplication unit 682, a multiplication unit 684, and a selection processing unit 686.

  The multiplying unit 682 adds the harmonic sound analyzed by the harmonicity analyzing unit 62 to each of the N evaluation values EH (1) to EH (N) set by the first analyzing unit 64 for the N types of harmonic sound sources. N identification indices R (R = EH (n) × WH) are calculated by multiplying the accuracy WH. On the other hand, in the multiplication unit 684, the harmonic analysis unit 62 analyzes each of the M evaluation values EP (1) to EP (M) set by the second analysis unit 66 for the M types of non-harmonic sound sources. By multiplying the non-harmonic sound accuracy WP, M identification indices R (R = Ep (m) × WP) are calculated. The identification index R is calculated for each of K types (K = N + M) candidate sound sources including N types of harmonic sound sources and M types of non-harmonic sound sources by the processing of the multiplication unit 682 and the multiplication unit 684. As understood from the above description, the accuracy WH corresponds to a weight value for each evaluation value EH (n) of the harmonic sound, and the accuracy WP corresponds to a weight value for each evaluation value EP (m) of the non-harmonic sound. To do. The higher the accuracy WH that the target sound corresponds to the harmonic sound, the greater the identification index R of the harmonic sound source, and the higher the accuracy WP that the target sound corresponds to the non-harmonic sound, the higher the identification index R of the non-harmonic sound source. Relatively dominant.

  The selection processing unit 686 specifies the type of the sound source of the target sound of the acoustic signal XA according to the K identification indexes R calculated by the multiplication unit 682 and the multiplication unit 684, and the sound source identification information indicating the type of the sound source D (for example, instrument name) is generated. Specifically, the selection processing unit 686 selects one type of candidate sound source having the maximum identification index R among the K types of candidate sound sources as the sound source of the target sound, and the sound source identification information D for designating the candidate sound source. Is generated. That is, the type of sound source of the target sound of the acoustic signal XA is identified. By executing the processing exemplified above for each of the plurality of acoustic signals XA, sound source identification information D indicating the type of sound source of the target sound is generated for each acoustic signal XA. Specific examples of the acoustic analysis unit 20 are as described above.

  The acoustic processing unit 30 in FIG. 1 generates an acoustic signal XB by performing acoustic processing on a plurality of acoustic signals XA with reference to the sound source identification information D analyzed by the harmonic analysis unit 62 for each acoustic signal XA. . Specifically, acoustic processing set in advance for each type of sound source indicated by the sound source identification information D of the acoustic signal XA is executed on the acoustic signal XA. As acoustic processing for the acoustic signal XA, for example, effect imparting processing (effector) that imparts various acoustic effects such as reverberation effect and distortion effect, characteristic adjustment processing (equalizer) that adjusts the volume for each frequency band, and sound image localization Examples are a localization adjustment process (pan) for adjusting the position to be performed and a volume adjustment process for adjusting the volume. The type and degree of the sound effect given to the sound signal XA by the effect applying process, the frequency characteristic given to the sound signal XA by the characteristic adjusting process, the position of the sound image adjusted by the localization adjusting process, and the adjustment contents by the volume adjusting process ( Various parameters such as gain) are individually set for each type of sound source indicated by the sound source identification information D. Then, the sound processing unit 30 generates the sound signal XB by mixing (mixing) the plurality of sound signals XA after the sound processing exemplified above. That is, the acoustic processing unit 30 according to the first embodiment realizes automatic mixing that reflects the sound source identification result by the harmonic analysis unit 62.

  FIG. 10 is a flowchart of a process (hereinafter referred to as “sound source identification process”) in which the sound source identification unit 60 of the first embodiment specifies the type of sound source of the target sound for any one system of acoustic signals XA. For each of the plurality of acoustic signals XA, the sound source identification process of FIG. 10 is executed every time the feature amount F is extracted by the feature amount extraction unit 50 (for each sound generation section P).

  When the sound source identification process is started, the harmonic analysis unit 62 analyzes whether the target sound represented by the acoustic signal XA corresponds to the harmonic sound or the non-harmonic sound from the feature value F of the acoustic signal XA (SB1). On the other hand, the first analysis unit 64 calculates the evaluation value EH (n) (EH (1) to EH (N)) for each of the N types of harmonic sound sources by the harmonic analysis process described with reference to FIG. (SB2), the second analysis unit 66 performs evaluation values EP (m) (EP (1) to EP (M) for each of the M types of non-harmonic sound sources by the non-harmonic analysis process similar to the harmonic analysis process. )) Is calculated (SB3). The sound source identification unit 68 identifies the type of sound source of the acoustic signal XA according to the results of the above analysis by the harmonic analysis unit 62, the first analysis unit 64, and the second analysis unit 66 (SB4). . The order of the harmonic analysis by the harmonic analysis unit 62, the harmonic analysis processing by the first analysis unit 64, and the non-harmonic analysis processing by the second analysis unit 66 is arbitrary. For example, after the harmonic analysis process (SB2) and the non-harmonic analysis process (SB3) are performed, the harmonic analysis unit 62 can analyze the harmonics.

  As described above, in the first embodiment, the type of sound source of the target sound is specified by distinguishing the harmonic sound and the non-harmonic sound from each other. Specifically, the harmonic analysis unit 62 analyzes the accuracy (WH, WP) that the target sound corresponds to each of the harmonic sound and the non-harmonic sound, and N types of harmonic sound sources as the sound source of the target sound. The analysis results of the accuracy CH (n) corresponding to each of the first and second analysis portions 64 and the accuracy CP (m) corresponding to each of the M types of non-harmonic sound sources as the sound source of the target sound The type of sound source of the target sound is specified using the result analyzed by 66. Therefore, it is possible to specify the sound source type of the target sound with high accuracy as compared with the configuration in which the sound source type is specified without distinguishing between the harmonic sound and the non-harmonic sound. There is also an advantage that harmonic sound / non-harmonic sound can be identified by the sound processing unit 30 for the unlearned sound source of the first analysis unit 64 and the second analysis unit 66.

  In the first embodiment, the accuracy WH corresponding to the target sound corresponding to the harmonic sound and the evaluation value EH (n) of each harmonic sound source are multiplied, and the accuracy WP corresponding to the target sound corresponding to the subharmonic sound and each non-harmonic sound. The identification index R is calculated for each of K types of musical instruments (N types of harmonic sources and M types of non-harmonic sources) by multiplication with the harmonic sound source evaluation value EP (m). The type of sound source of the target sound is specified according to the above. That is, as the accuracy WH corresponding to the target sound corresponds to the harmonic sound, the harmonic sound source identification index R becomes relatively dominant, and as the accuracy WP corresponding to the target sound corresponds to the non-harmonic sound, the identification index of the non-harmonic sound source increases. R becomes relatively dominant. Therefore, there is an advantage that the type of sound source of the target sound can be specified easily and with high accuracy by comparing the K identification indexes R.

  By the way, for example, the accuracy CH (n) corresponding to the sound source of the target sound corresponding to the harmonic sound source is used as the evaluation value EH (n) and the accuracy CP (m) corresponding to the sound source of the target sound corresponding to the non-harmonic sound source is used. In the configuration used as the evaluation value EP (m) (hereinafter referred to as “comparative example”), the numerical value of the evaluation value EH (n) depends on the number N of types of harmonic sound sources and the numerical value of the evaluation value EP (m) is not Depends on the number M of types of harmonic sound sources. For example, the accuracy CH (n) becomes smaller as the number N of types of harmonic sound sources increases. Therefore, when the number N of harmonic sound sources and the number M of non-harmonic sound sources are different, there is a problem that the evaluation value EH (n) and the evaluation value EP (m) cannot be appropriately compared. In the first embodiment, a numerical value corresponding to the rank of the accuracy CH (n) corresponding to the sound source of the target sound corresponding to the harmonic sound source is set as the evaluation value EH (n) for each harmonic sound source, and the sound source of the target sound A numerical value corresponding to the rank of the accuracy CP (m) corresponding to the subharmonic sound source is set for each subharmonic sound source as the evaluation value EP (m). That is, the evaluation value EH (n) is set to a value that does not depend on the number N of harmonic sound sources, and the evaluation value EP (m) is set to a value that does not depend on the number M of non-harmonic sound sources. Therefore, according to the first embodiment, for example, the evaluation value EH (n) and the evaluation value EP (m) are appropriately set even when the number N of harmonic sound sources is different from the number M of non-harmonic sound sources. There is an advantage that it can be compared. In other words, the restrictions on the number N of harmonic sound sources and the number M of non-harmonic sound sources are relaxed. However, the comparative examples described above are also included in the scope of the present invention.

  In the first embodiment, the first analysis unit 64 uses the characteristic amount F (characteristic value f1) used for calculating the harmonic source accuracy CH (n) and the second analysis unit 66 uses the non-harmonic source accuracy CP. The feature amount F (characteristic value f2) applied to the calculation of (m) is different. Specifically, for example, the characteristic value f1 suitable for identifying harmonics is used for the calculation of the accuracy CH (n) by the first analysis unit 64, and for the calculation of the accuracy CP (m) by the second analysis unit 66, for example. A characteristic value f2 suitable for identifying non-harmonic sound is used. Therefore, the sound source of the target sound is highly accurate compared to a configuration that uses the same type of feature quantity for the calculation of the harmonic source accuracy CH (n) and the non-harmonic source accuracy CP (m). There is an advantage that it can be identified. However, it is also possible for the first analysis unit 64 and the second analysis unit 66 to use a common feature amount F.

Second Embodiment
A second embodiment of the present invention will be described. In addition, about the element which an effect | action and function are the same as that of 1st Embodiment in each form illustrated below, the code | symbol used by description of 1st Embodiment is diverted, and each detailed description is abbreviate | omitted suitably.

  As illustrated in FIG. 11, a plurality of local maximum points xH (xH1, xH2) are observed in the process in which the intensity Q increases immediately after the start of one sound generation by the sound source (for example, sound generation by one percussion instrument percussion). May be. Since the fluctuation index δ of the local maximum point xH1 in FIG. 11 exceeds the starting point threshold value ZS, in the first embodiment in which all the local maximum points xH whose fluctuation index δ exceeds the starting point threshold value ZS are determined as the pronunciation starting point TS, the local maximum point xH1 and the local maximum point Both of the points xH2 are specified as the pronunciation start point TS. However, the intensity of the acoustic signal XE increases to the maximum point xH2 immediately after the maximum point xH1. That is, it is estimated that the local maximum point xH1 and the local maximum point xH2 actually correspond to one sound generation of the sound source. Accordingly, without specifying the local maximum point xH1 as the pronunciation start point TS, only the local maximum point xH2 immediately after is specified as the pronunciation start point TS, and both the local maximum point xH1 and the local maximum point xH2 should be included in one sound generation section P. . Considering the above circumstances, in the second embodiment, when a local maximum point xH2 having an intensity Q exceeding the local maximum point xH1 is detected immediately after one local maximum point xH1 of the intensity Q of the acoustic signal XE, the preceding is performed. The maximum point xH1 is excluded from the pronunciation start point TS candidates.

  Specifically, when any one local maximum point xH1 (first local maximum point) in which the variation index δ exceeds the starting point threshold value ZS is detected by the same method as that in the first embodiment, the starting point analysis unit 44 is shown in FIG. As illustrated, the standby section V is set at a position on the time axis corresponding to the local maximum point xH1. The standby section V is a section in which it is reserved that the local maximum point xH1 is determined as the sound generation start point TS, and is set after the local maximum point xH1. The start point analysis unit 44 of the second embodiment sets a standby section V having a predetermined length starting from the maximum point xH1.

  When the standby section V is set, the start point analysis unit 44 continues searching for the maximum point xH for the acoustic signal XE after the maximum point xH1. As described above, the intensity Q of the acoustic signal XE may increase after the maximum point xH1. When a local maximum point xH2 (second local maximum point) having an intensity exceeding the local maximum point xH1 is detected in the standby section V, the start point analysis unit 44 excludes the preceding local maximum point xH1 from the pronunciation start point TS candidates. When the standby section V passes without detecting the local maximum point xH having an intensity exceeding the detected local maximum point xH, the start point analysis unit 44 finally detects the standby section V before the expiration. The determined local maximum point xH is determined as the pronunciation start point TS.

  As understood from the above description, in the second embodiment, when a local maximum point xH2 having an intensity Q exceeding the local maximum point xH1 is detected in the standby section V after the local maximum point xH1 of the intensity Q of the acoustic signal XE. In addition, the local maximum point xH1 is excluded from the candidates for the pronunciation start point TS. Therefore, even when a plurality of local maximum points xH are detected in the process of increasing the intensity Q of the acoustic signal XE from the start of one sound generation by the sound source, the sound generation section including one local maximum point xH corresponding to the sound generation It is possible to specify P appropriately.

  In the second embodiment, the standby interval V starting from one local maximum point xH1 is set. However, when a local maximum point xH2 having an intensity Q exceeding the local maximum point xH1 is detected, the local maximum point xH2 is set as the starting point. It is also possible to newly set the waiting section V to be performed (that is, to update the waiting section V every time the maximum point xH is detected).

<Modification>
Each aspect illustrated above can be variously modified. Specific modifications are exemplified below. Two or more modes arbitrarily selected from the following examples can be appropriately combined within a range that does not contradict each other.

(1) In each of the above embodiments, the harmonic analysis unit 62 discriminates between harmonic and non-harmonic sounds using the support vector machine, but the harmonic / non-harmonic sound discrimination method by the harmonic analysis unit 62 is as described above. It is not limited to the illustration. For example, a method of discriminating a target sound into a harmonic sound and a non-harmonic sound using a mixed normal distribution that expresses a distribution tendency of the feature amount F of each of the harmonic sound and the non-harmonic sound, or a K-means algorithm is used. A method of discriminating the target sound into a harmonic sound and a non-harmonic sound by clustering may be employed. Similarly, the method in which each of the first analysis unit 64 and the second analysis unit 66 estimates the type of the sound source of the target sound is not limited to the support vector machine exemplified in each of the above-described embodiments, and is well-known pattern recognition. Any technique can be employed.

(2) In the above-described embodiments, the harmonic sound accuracy WH analyzed by the harmonic analysis unit 62 is multiplied by N evaluation values EH (1) to EH (N), and the non-harmonic sound accuracy WP is set to M. Although the evaluation values EP (1) to EP (M) are multiplied, the method of reflecting the accuracy WH of the harmonic sound and the accuracy WP of the non-harmonic sound in the type of the sound source of the acoustic signal XA is not limited to the above examples. . For example, it is determined according to the accuracy WH and the accuracy WP whether the target sound of the acoustic signal XA corresponds to the harmonic sound or the non-harmonic sound, and N evaluation values EH (1) to EH (N) and M It is also possible for the sound source specifying unit 68 to specify the type of the sound source by selectively using any one of the evaluation values EP (1) to EP (M) according to the harmonic discrimination result.

  Specifically, the harmonic analysis unit 62 determines the target sound as a harmonic sound when the accuracy WH exceeds the accuracy WP, and determines the target sound as a non-harmonic sound when the accuracy WP exceeds the accuracy WH. To do. When it is determined that the target sound is a harmonic sound, the sound source identification unit 68 corresponds to the maximum value among the N evaluation values EH (1) to EH (N) calculated by the first analysis unit 64. If the target sound is determined to be a non-harmonic sound while the harmonic sound source to be identified is identified as the type of sound source, the M evaluation values EP (1) to EP (2) calculated by the second analysis unit 66 are determined. The non-harmonic sound source corresponding to the maximum value in M) is specified as the type of sound source. The configuration exemplified above is also referred to as a configuration in which one of the accuracy WH and the accuracy WP is set to 1 and the other is set to 0 in each of the above-described embodiments. When the harmonic analysis unit 62 determines that the target sound is a harmonic sound, non-harmonic analysis processing (calculation of M evaluation values EP (1) to EP (M)) by the second analysis unit 66 is performed. When the harmonic analysis unit 62 analyzes that the target sound is a non-harmonic sound, harmonic analysis processing (N evaluation values EH (1) to EH (N A configuration in which the calculation of () is omitted may be employed.

  As understood from the above examples, the sound source specifying unit 68 specifies the type of sound source of the target sound according to the analysis results by the harmonic analysis unit 62, the first analysis unit 64, and the second analysis unit 66. Whether the analysis results of both the first analysis unit 64 and the second analysis unit 66 are used or only one of the analysis results is used is unquestioned in the present invention.

(3) In each of the above-described embodiments, the starting point threshold value ZS is a fixed value, but the starting point threshold value ZS may be a variable value. For example, a numerical value corresponding to the intensity QH of the acoustic signal XE at the local maximum point xH (for example, a numerical value obtained by multiplying the predetermined value by the intensity QH) is used as the start point threshold value ZS, and in step SC1 in FIG. It is also possible to compare the difference (QH−QREF) between QH and the reference value QREF with the starting point threshold value ZS using the variation index δ. Similarly, the end point threshold value ZE can be a variable value. It is also possible to variably set the start point threshold value ZS or the end point threshold value ZE according to an instruction from the user.

(4) It is also possible to realize the acoustic processing device 12 by a server device that communicates with a terminal device (for example, a mobile phone or a smartphone) via a communication network such as a mobile communication network or the Internet. Specifically, the acoustic processing device 12 generates an acoustic signal XB from the plurality of acoustic signals XA received from the terminal device via the communication network, and transmits the acoustic signal XB to the terminal device by the same processing as in the above-described embodiments. Note that the acoustic analysis unit 20 identifies the type of sound source (sound source identification information D) of each of the plurality of acoustic signals XA received from the terminal device, notifies the terminal device, and the acoustic processing unit 30 mounted on the terminal device. However, it is also possible to generate the acoustic signal XB from the plurality of acoustic signals XA according to the identification result. That is, the sound processing unit 30 can be omitted from the sound processing device 12. In the configuration in which the terminal device is notified of the sound generation start point TS and the sound generation end point TE of the sound generation section P of the acoustic signal XA (for example, the terminal device includes the feature amount extraction unit 50 and the sound source analysis unit 20), the sound processing device 12 The feature amount extraction unit 50 and the sound source identification unit 60 are omitted from the acoustic analysis unit 20.

  As will be understood from the above description, the preferred embodiment of the present invention is an apparatus for analyzing the sound generation start point TS at which sound generation is started and the sound generation end point TE at which sound generation ends (acoustic signal XA). Analysis device). The presence or absence of the acoustic processing unit 30 in the acoustic analysis device is not questioned.

(5) The sound processing device 12 exemplified in the above-described embodiments is realized by the cooperation of the control device 122 and the program as described above. The program may be provided in a form stored in a computer-readable recording medium and installed in the computer. The recording medium is, for example, a non-transitory recording medium, and an optical recording medium (optical disk) such as a CD-ROM is a good example, but a known arbitrary one such as a semiconductor recording medium or a magnetic recording medium This type of recording medium can be included. Further, the program exemplified above can be provided in the form of distribution via a communication network and installed in a computer.

(6) The present invention is also specified as an operation method of the sound processing device 12 according to each of the above-described embodiments. For example, in a method (acoustic analysis method) in which the sounding section detection unit 40 analyzes a sounding start point TS at which sound generation is started and a sounding end point TE at which sound generation ends in the sound signal XA, a computer (single unit) is used. In addition to the above device, a computer system composed of a plurality of devices separated from each other) specifies the local maximum point xH of the intensity Q of the acoustic signal XA as the pronunciation starting point TS (starting point analysis processing in FIG. 6), The intensity xH at the maximum point xH after the increase and the intensity up to the maximum point xH after the intensity Q of the acoustic signal XA decreases with time after the sounding start point TS has elapsed. When the variation index δ according to the difference between the Q and the minimum value QREF exceeds the end point threshold value ZE, the sound generation end point TE is specified (end point analysis process in FIG. 7).

DESCRIPTION OF SYMBOLS 12 ... Sound processing device, 14 ... Sound collection device, 16 ... Sound emission device, 122 ... Control device, 124 ... Memory | storage device, 20 ... Sound analysis part, 30 ... Sound processing part, 40 ... Sound generation section detection unit 42... Signal processing unit 44... Start point analysis unit 46... End point analysis unit 50 .. feature amount extraction unit 60 .. sound source identification unit 62. 64... First analysis unit 64, 66... Second analysis unit, 68... Sound source identification unit, 682.

Claims (5)

A sound analysis device that analyzes a sound generation start point at which sound generation starts and a sound end point at which sound generation ends in an acoustic signal,
A starting point analysis unit that identifies the maximum point of the intensity of the acoustic signal as the starting point of sound generation;
The minimum point at which the intensity reverses to increase in the process in which the intensity of the acoustic signal decreases with time after the start of the sounding start point, the intensity at the maximum point after the increase and the minimum value of the intensity up to the maximum point An acoustic analysis device comprising: an end point analysis unit that identifies the sound generation end point when a variation index corresponding to the difference exceeds an end point threshold value. The end point analysis unit determines the attenuation when the intensity of the acoustic signal has decreased from the sounding start point until the minimum point falls below an attenuation threshold corresponding to the intensity at the sounding start point before specifying the minimum point as the sounding end point. The acoustic analysis device according to claim 1, wherein a point in time when the intensity falls below a threshold is specified as the pronunciation end point. The starting point analysis unit sequentially detects the maximum point of the intensity of the acoustic signal, while a variation index according to the difference between the intensity at the maximum point and the minimum value of the intensity up to the maximum point is the end point threshold value The acoustic analysis device according to claim 1, wherein the local maximum point is specified as the sound generation start point when a start point threshold value greater than the threshold value is exceeded. The acoustic analysis according to any one of claims 1 to 3, wherein the variation index is a numerical value obtained by dividing a difference between the intensity at the local maximum point and the minimum value of the intensity up to the local maximum point by the intensity at the local maximum point. apparatus. When the start point analysis unit detects a second maximum point having an intensity exceeding the first maximum point in a standby section after the first maximum point of the intensity of the acoustic signal, the start point analysis unit generates the first maximum point as the pronunciation The acoustic analysis device according to any one of claims 1 to 4, wherein the acoustic analysis device is excluded from a starting point candidate.

Images