JP6035785B2 - Acoustic analysis apparatus and acoustic analysis method - Google Patents

Description

  The present invention relates to a technique for analyzing an acoustic signal.

  Techniques for detecting the pitch (fundamental frequency) of acoustic signals have been conventionally proposed. For example, Non-Patent Document 1 and Patent Document 1 disclose a technique for detecting a pitch for an acoustic signal in which a plurality of sounds are mixed.

A.P.Klapuri, "Multiple fundamental frequency estimation based on harmonicity and spectral smoothness," IEEE Trans. Speech and Audio Proc., 11 (6), 804-816, 2003

JP 2001-125562 A

  In the techniques of Non-Patent Document 1 and Patent Document 1, a plurality of candidates are estimated for the pitch of an acoustic signal, but a configuration in which a user selects a desired pitch from a plurality of candidates has not been proposed. Therefore, the acoustic component of the erroneously detected pitch is excluded in an auxiliary manner according to an instruction from the user, or the user's desired acoustic component (for example, singing sound or instrument sound) is separated from the acoustic signal. There is a problem that can not be. In view of the above circumstances, an object of the present invention is to enable a user to easily select a pitch specified from an acoustic signal.

  Means employed by the present invention to solve the above problems will be described. In order to facilitate the understanding of the present invention, in the following description, the correspondence between the elements of the present invention and the elements of the embodiments described later will be indicated in parentheses, but the scope of the present invention will be exemplified in the embodiments. It is not intended to be limited.

  The acoustic analysis device of the present invention displays an analysis image in which a plurality of candidate trajectories (for example, candidate trajectory P0) expressing a time change in pitch specified from an acoustic signal (for example, acoustic signal X) are arranged in the frequency-time domain. Display control means (for example, the display control section 24) to be displayed on the display, instruction receiving means (for example, the instruction receiving section 26) for receiving designation of the instruction locus (for example, the instruction locus PA) for the frequency-time domain from the user, Trajectory setting means (for example, trajectory setting unit 28) is provided for displaying a selected trajectory (for example, selected trajectory PB) along the candidate trajectory of the corresponding position in the frequency-time domain. In the above configuration, since the selection trajectory along the candidate trajectory displayed in the frequency-time domain is set according to the instruction from the user (designation of the instruction trajectory), the pitch of the specific acoustic component of the acoustic signal is set. The user can select intuitively and easily. Further, since a selection trajectory is set along a candidate trajectory at a position corresponding to the instruction trajectory designated by the user, an appropriate selection trajectory can be selected without designating the instruction trajectory so as to exactly match the candidate trajectory. .

  The acoustic analysis apparatus according to a preferred aspect of the present invention includes acoustic processing means (for example, the acoustic processing unit 30) that generates a separation signal (for example, the separation signal Y) having a pitch corresponding to the selected locus. In the above configuration, since a separation signal having a pitch corresponding to the selected locus is generated, the user can specify the instruction locus while listening to the reproduction sound corresponding to the selected locus. Specifically, the acoustic processing means generates a separated signal by causing a separation filter for separating an acoustic component corresponding to the selected locus in the acoustic signal to act on the acoustic signal. With the above configuration, there is an advantage that high-quality reproduced sound can be reproduced for the acoustic component corresponding to the selected locus. In another aspect, the sound processing means generates a separation signal indicating a pure tone having a frequency corresponding to the selected locus. That is, the sound processing means generates a separated signal independently of the sound signal. According to the above aspect, there exists an advantage that the processing load of an acoustic processing means is reduced compared with the structure which makes a separation filter act on an acoustic signal.

  In a preferred aspect of the present invention, the display control means uses the pitch likelihood distribution calculated from the acoustic signal (for example, the pitch likelihood L (k, m)) as a candidate trajectory. In the frequency-time domain, the trajectory setting means has each point on the candidate trajectory where the pitch likelihood is maximum within a predetermined range (for example, range R) in the frequency axis direction including each point on the indicated trajectory. Is set as the selection trajectory. According to the above aspect, there exists an advantage that a suitable selection locus | trajectory can be set with a simple process. In another aspect, the display control means displays the pitch likelihood distribution calculated from the acoustic signal in the frequency-time domain so that the pitch likelihood peak trajectory is a candidate trajectory, and the trajectory setting means A distribution of weight values (for example, distribution D of weight values w) is set in the frequency axis direction for each point on the indicated locus, and the weight value at that point is added to the pitch likelihood of each point on the candidate locus. An array of points on the candidate trajectory with the maximum numerical value is set as the selection trajectory. In the above aspect, for example, if the weight value is large even at a point where the pitch likelihood is low among the candidate trajectories, the selection trajectory can be set that sufficiently reflects the instruction trajectory designated by the user. There are advantages.

  The acoustic analysis apparatus according to a preferred aspect of the present invention includes a trajectory classification unit (for example, a trajectory classification unit 32) that divides a plurality of selected trajectories set by the trajectory setting unit into a plurality of trajectory groups. The display mode of each of the plurality of selected trajectories is made different for each trajectory group divided by the trajectory classification means. In the above aspect, since each selection locus is displayed in an individual display manner for each locus group, there is an advantage that the user can easily grasp the selection locus of a desired locus group. In addition, the specific example of the above aspect is later mentioned as 2nd Embodiment, for example. The “display mode” of an image in the present specification means a property of the image that can be visually distinguished by the user, and includes the brightness, color, hue, and the like of the image as typical examples.

  In a preferred aspect of the present invention, when the end point (for example, the end point eA) of the designated locus is designated in the peripheral region (for example, the region C) around the end point (for example, the end point e0) of the candidate track, the trajectory setting means A selection trajectory along the candidate trajectory is set with the end point of. In the above aspect, when the end point of the instruction trajectory is specified in the peripheral region of the end point of the candidate trajectory, the selection trajectory with the end point of the candidate trajectory as the end point is set, so the starting point of pronunciation of each acoustic component of the acoustic signal Alternatively, there is an advantage that a selection locus having the end point as an end point can be easily set. In addition, the specific example of the above aspect is later mentioned, for example as 3rd Embodiment.

  In a preferred aspect of the present invention, the display control means displays different display modes for each candidate trajectory that is different from a stationary part (for example, the stationary part Q1) parallel to the time axis and a varying part (for example, the varying part Q2) other than the stationary part. Is displayed. In the above aspect, since the steady part and the variable part of each candidate trajectory are displayed in different display forms, the user intuitively grasps the acoustic component in which the pitch easily varies and the acoustic component in which the pitch hardly varies. There is an advantage that you can. In addition, the specific example of the above aspect is later mentioned as 4th Embodiment, for example. Further, according to the configuration in which the display control unit highlights the candidate trajectory corresponding to the specific harmonic structure, there is an advantage that the user can intuitively grasp the candidate trajectory corresponding to the specific musical instrument, for example. A specific example of the above aspect will be described later as a fifth embodiment, for example.

  In a preferred aspect of the present invention, the trajectory setting means, when instructed by the user to move one existing selected trajectory in the frequency axis direction, is a candidate having a harmonic overtone relationship with respect to the one selected trajectory. Set the selected trajectory along the trajectory. In the above aspect, since the selection trajectory is set along the candidate trajectory having a harmonic overtone with respect to the existing selection trajectory, the correction of the selection trajectory set to a frequency shifted by one octave from the desired pitch or the harmonic structure Can be easily realized. A specific example of the above aspect will be described later as a sixth embodiment, for example.

  The acoustic analysis apparatus according to each aspect described above is realized by hardware (electronic circuit) such as a DSP (Digital Signal Processor) dedicated to processing of an acoustic signal, or a general-purpose calculation such as a CPU (Central Processing Unit). This is also realized by cooperation between the processing device and the program. The program of the present invention includes a display control process for displaying on a display device an analysis image in which a plurality of candidate trajectories expressing a time change of a pitch specified from an acoustic signal are arranged in a frequency-time domain, and an instruction for the frequency-time domain The computer executes an instruction receiving process for accepting designation of a trajectory from the user and a trajectory setting process for displaying a selected trajectory along a candidate trajectory at a position corresponding to the designated trajectory in the frequency-time domain. According to the above program, the same operation and effect as the acoustic analysis apparatus of the present invention are exhibited. The program of the present invention is provided in a form stored in a computer-readable recording medium and installed in the computer, or is provided in a form distributed via a communication network and installed in the computer.

1 is a block diagram of an acoustic analysis device according to a first embodiment of the present invention. It is a schematic diagram of an analysis image. It is explanatory drawing of operation | movement of a locus | trajectory setting part. It is a block diagram of the acoustic analysis device concerning a 2nd embodiment. It is explanatory drawing of operation | movement of the locus | trajectory setting part in 3rd Embodiment. It is a schematic diagram of the analysis image in 4th Embodiment. It is explanatory drawing of operation | movement of the locus | trajectory setting part in 6th Embodiment. It is explanatory drawing of operation | movement of the locus | trajectory setting part in a modification.

<First Embodiment>
FIG. 1 is a block diagram of an acoustic analysis apparatus 100 according to the first embodiment of the present invention. As shown in FIG. 1, the acoustic analysis device 100 is realized by a computer system including an arithmetic processing device 10, a storage device 12, a display device 14, an input device 16, and a sound emitting device 18.

  The storage device 12 stores a program PGM executed by the arithmetic processing device 10 and various data used by the arithmetic processing device 10. The storage device 12 of the first embodiment stores the acoustic signal X. The acoustic signal X is a sample series indicating a mixed sound of a plurality of acoustic components that may have different pitches (fundamental frequencies). A known recording medium such as a semiconductor recording medium or a magnetic recording medium or a combination of a plurality of types of recording media is arbitrarily employed as the storage device 12. The acoustic analysis device 100 according to the first embodiment is a signal processing device that analyzes the pitch of the acoustic signal X stored in the storage device 12.

  The display device 14 (for example, a liquid crystal display panel) displays an image instructed from the arithmetic processing device 10. The input device 16 is a device that receives an instruction from the user to the acoustic analysis device 100, and includes, for example, a plurality of operators operated by the user. The sound emitting device 18 (speaker or headphones) reproduces sound waves according to the separation signal Y generated by the arithmetic processing device 10. The separated signal Y is a signal obtained by emphasizing (ideally extracting) a specific acoustic component in the acoustic signal X. In addition, illustration of the D / A converter which converts the separation signal Y which the arithmetic processing unit 10 produced | generated from digital to analog was abbreviate | omitted for convenience.

  The arithmetic processing unit 10 executes a program PGM stored in the storage device 12 to thereby analyze a plurality of functions (analysis processing unit 22, display control unit 24, instruction receiving unit 26, trajectory setting). Unit 28 and acoustic processing unit 30). A configuration in which each function of the arithmetic processing device 10 is distributed to a plurality of devices, or a configuration in which a dedicated electronic circuit (DSP) realizes some functions of the arithmetic processing device 10 may be employed.

  The analysis processing unit 22 calculates the pitch likelihood L (k, m) of each frequency on the frequency axis from the acoustic signal X for each unit period (for each frame) on the time axis. The symbol k means any one frequency (frequency band) on the frequency axis, and the symbol m means any one unit period (frame) on the time axis. One pitch likelihood L (k, m) has an accuracy (likelihood) that the kth frequency on the frequency axis corresponds to the pitch (fundamental frequency) of the acoustic signal X in the mth unit period. Equivalent to. For the calculation of the pitch likelihood L (k, m) by the analysis processing unit 22, a known technique (for example, the technique of Non-Patent Document 1 or Patent Document 1) can be arbitrarily employed.

  The display control unit 24 causes the display device 14 to display the image 50 shown in FIG. 2 (hereinafter referred to as “analysis image”) for the user to visually recognize the time change of the pitch estimated from the acoustic signal X. As shown in FIG. 2, the analysis image 50 is a GUI (Graphical User Interface) in which a frequency-time region 52 is arranged. The frequency-time region 52 is a coordinate plane in which a frequency axis AF and a time axis AT intersecting each other are set.

  The display control unit 24 displays the analysis result by the analysis processing unit 22 in the frequency-time region 52. That is, the display control unit 24 displays the distribution of each pitch likelihood L (k, m) calculated by the analysis processing unit 22 for each unit period in the frequency-time region 52 as shown in FIG. Specifically, the display mode of each point in the frequency-time domain 52 (so that the user can visually grasp the level of each pitch likelihood L (k, m) in the frequency-time domain 52 ( For example, brightness, saturation, hue) are variably set according to the pitch likelihood L (k, m). For example, the higher the pitch likelihood L (k, m) in the frequency-time region 52, the higher the gradation is displayed. Therefore, as can be understood from FIG. 2, a locus (hereinafter referred to as “candidate locus”) P0 in which the peaks of the pitch likelihood L (k, m) for each unit period are arranged in the direction of the time axis AT is determined by the user. It is displayed in the frequency-time region 52 in a visually distinguishable state. Since the acoustic signal X is a mixed sound signal of a plurality of acoustic components, a plurality of candidate trajectories P0 in which the position in the direction of the frequency axis AF and the position in the direction of the time axis AT are different are arranged in the frequency-time region 52. Is done. The candidate trajectories P0 can overlap each other on the time axis AT. The frequency at which the pitch likelihood L (k, m) peaks on the frequency axis AF (each frequency on the candidate trajectory P0) is likely to correspond to the pitch of the acoustic signal X.

  The display control unit 24 variably controls the display magnification of the frequency-time region 52 in accordance with an instruction from the user to the input device 16. Further, the display control unit 24 displays the distribution of the pitch likelihood L (k, m) corresponding to a part of the time axis of the acoustic signal X in the analysis image 50, and the pitch of the acoustic signal X The section in which the likelihood L (k, m) is displayed in the analysis image 50 is changed (scrolls the frequency-time region 52 in the direction of the time axis AT) according to an instruction from the user.

  The user appropriately operates the input device 16 while referring to the distribution of the pitch likelihoods L (k, m) (each candidate trajectory P0) in the frequency-time region 52, so that the part (A in FIG. ), A desired trajectory (hereinafter referred to as “instruction trajectory”) PA can be designated at an arbitrary position in the frequency-time region 52. Specifically, the user drags the mouse of the input device 16 so as to roughly follow the candidate trajectory P0 close to the frequency of the desired acoustic component to be separated in the acoustic signal X, thereby indicating the instruction trajectory PA. specify. It is sufficient for the user to designate the instruction locus PA so as to roughly follow the desired candidate locus P0, and it is not necessary to designate the instruction locus PA so as to exactly match the candidate locus P0. The instruction receiving unit 26 in FIG. 1 is an element that receives designation of an instruction locus PA for the frequency-time region 52 from a user. In FIG. 3 (A), the instruction locus PA is shown for convenience, but the instruction locus PA is not displayed in the actual analysis image 50. However, the instruction trajectory PA can be displayed together with the candidate trajectory P0.

  The trajectory setting unit 28 in FIG. 1 sets a selection trajectory PB corresponding to each candidate trajectory P0 displayed on the display device 14 and the instruction trajectory PA received from the user by the instruction accepting unit 26 to set the frequency-time region 52. Place in. The selected trajectory PB and the candidate trajectory P0 are displayed in different display modes (for example, brightness, saturation, hue) so that the user can visually distinguish both trajectories.

  The selection trajectory PB of the first embodiment is a trajectory (curved line or broken line) along the candidate trajectory P0 at a position corresponding to the instruction trajectory PA in the frequency-time region 52. As shown in part (B) of FIG. 3, the trajectory setting unit 28 according to the first embodiment has a frequency axis AF including one point pA on the instruction trajectory PA received from the user by the instruction receiving unit 26. A trajectory in which the point p0 on the candidate trajectory P0 having the maximum pitch likelihood L (k, m) within the predetermined range R in the direction is arranged for each point pA on the designated trajectory PA is set as the selection trajectory PB. . Therefore, the selection locus PB covers a section equivalent to the instruction locus PA on the time axis (a section between end points of the instruction locus PA).

  As understood from the above description, the instruction locus PA roughly designated by the user in the frequency-time region 52 is snapped to the predetermined candidate locus P0. However, when the candidate trajectory P0 does not exist within the range R including the point pA on the indicated trajectory PA (when the indicated trajectory PA is away from any candidate trajectory P0), the trajectory setting unit 28 The designated locus PA itself designated by is determined as the selected locus PB. The trajectory setting unit 28 sets the selection trajectory PB for each designation of the instruction trajectory PA, whereby a plurality of selection trajectories PB are arranged in the frequency-time region 52.

  The user can select one or more selection trajectories PB arranged in the frequency-time region 52 as a processing target and instruct editing. Specifically, the trajectory setting unit 28 executes editing operations such as extending (adding), shortening, deleting, duplicating, and pasting the selected trajectory PB with respect to the selected trajectory PB selected by the user. A method for selecting the selection locus PB to be edited in accordance with an instruction from the user is arbitrary. For example, a configuration in which the trajectory setting unit 28 selects each selected trajectory PB in a region specified by the user in a rubber band (controlling the diagonal length of the rectangular region) or freehand in the frequency-time region 52 is employed. Further, a configuration in which the trajectory setting unit 28 selects each selection trajectory PB within the range specified by the user on the axis of the frequency axis AF and the time axis AT, and a selection trajectory of the time length within the numerical range specified by the user. A configuration in which the trajectory setting unit 28 selects PB is also suitable. It is also possible to select a plurality of selection trajectories PB by sequentially designating each selection trajectory PB while pressing a predetermined operator of the input device 16.

  When the user appropriately operates the input device 16 to instruct to save (export) the selected trajectory PB, the trajectory setting unit 28 designates each selected trajectory PB set in the frequency-time region 52 at the current stage. To be generated (hereinafter referred to as “trajectory file”) and stored in the storage device 12. On the other hand, when the user operates the input device 16 to instruct reading (import) of the trajectory file, the trajectory setting unit 28 sets each selected trajectory PB specified by the trajectory file stored in the storage device 12 to frequency-time. Arrange in the region 52. In the above description, reading of the trajectory file designated and saved (exported) with respect to the analysis result by the analysis processing unit 22 has been exemplified. However, the time trajectory of the pitch specified by various known pitch estimation techniques is described. It is also possible to read the indicated trajectory file (that is, a trajectory file other than the trajectory file generated by the trajectory setting unit 28).

  The acoustic processing unit 30 in FIG. 1 generates a separation signal Y having a pitch corresponding to the selected locus PB set by the locus setting unit 28. Specifically, the acoustic processing unit 30 starts generating the separation signal Y when the user instructs the acoustic analysis device 100 to separate the acoustic signal X by operating the input device 16. The separation signal Y of the first embodiment is an acoustic signal that emphasizes the acoustic component of the pitch indicated by the selected locus PB in the acoustic signal X.

  Specifically, the acoustic processing unit 30 generates a filter (hereinafter referred to as “separation filter”) for emphasizing an acoustic component corresponding to each selected locus PB for each unit period, and sequentially acts on the acoustic signal X. Thus, the separation signal Y is generated. The separation filter is a series of a plurality of coefficient values corresponding to different frequencies on the frequency axis. The separation filter of the first embodiment is a binary mask in which coefficient values corresponding to the pitch indicated by the selection locus PB and its harmonic frequency are set to 1 and the remaining coefficient values are set to 0. The acoustic processing unit 30 generates a separated signal Y by multiplying the frequency spectrum of each unit period of the acoustic signal X by a separation filter and converting it into the time domain. The separated signal Y generated by the acoustic processing unit 30 is supplied to the sound emitting device 18 and reproduced as a sound wave. Therefore, the user can listen to the reproduced sound in which the acoustic component of the pitch of the selection locus PB corresponding to the designated locus PA designated by the user is emphasized.

  On the other hand, the display control unit 24 arranges a reproduction indicator (cursor) 54 indicating the current reproduction point in the frequency-time region 52 as shown in FIG. The position on the time axis of 54 is changed. The user listens to the reproduced sound of the separated signal Y and visually recognizes the analysis image 50 (relationship between each selected locus PB and the reproduction indicator 54), thereby allowing each candidate locus P0 in the frequency-time region 52 and each It is possible to intuitively grasp the relationship between the selected trajectory PB and the sound generation and silencing time points and pitches of each acoustic component of the acoustic signal X. In parallel with listening to the reproduced sound of the separated signal Y and visually recognizing the analysis image 50, the user can edit each selection locus PB and add a new selection locus PB (designation of the instruction locus PA). 100 can be instructed.

  In the first embodiment described above, the candidate trajectory P0 arranged in the frequency-time region 52 is selected as the selection trajectory PB in accordance with an instruction from the user (designation of the instruction trajectory PA). The user can intuitively and easily select the pitch of the specific acoustic component of X. Particularly in the first embodiment, since the selection locus PB is set by snapping the designated locus PA designated by the user to the candidate locus P0, the effect that the user can easily select the desired selection locus PB is exceptional. It is remarkable.

Second Embodiment
A second embodiment of the present invention will be described below. In addition, about the element in which an effect | action and a function are equivalent to 1st Embodiment in each form illustrated below, the code | symbol referred by description of 1st Embodiment is diverted, and each detailed description is abbreviate | omitted suitably.

  FIG. 4 is a block diagram of the acoustic analysis device 100 according to the second embodiment. As illustrated in FIG. 4, the acoustic analysis device 100 according to the second embodiment has a configuration in which a trajectory classification unit 32 is added to the first embodiment. The trajectory classification unit 32 classifies the plurality of selection trajectories PB set in the frequency-time region 52 by the trajectory setting unit 28 into a plurality of groups (hereinafter referred to as “trajectory groups”) G. One trajectory group G includes one or more selected trajectories PB. Specifically, the trajectory classification unit 32 classifies a plurality of selected trajectories PB arbitrarily designated by the user through the operation of the input device 16 into one trajectory group G. For example, the user selects a plurality of selection trajectories PB estimated to correspond to the acoustic components of a desired musical instrument (for example, a plurality of selection trajectories PB existing in the range of the user's desired musical instrument) as one trajectory group G. It is possible to select as an element. Note that the selection trajectory PB classified into each trajectory group G is selected in the same manner as in the first embodiment (for example, the selection trajectory in the region designated by the user in the frequency-time region 52 with a rubber band or freehand). A configuration in which PB is selected may be employed.

  The display control unit 24 changes the display mode of each of the plurality of selection trajectories PB arranged in the frequency-time region 52 for each trajectory group G. Further, the user can individually instruct for each trajectory group G whether or not there is a process that can be executed for each selected trajectory PB (valid / invalid). Specifically, the display control unit 24 controls the presence / absence of the display of the selected locus PB for each locus group G according to an instruction from the user. Therefore, for example, only each selected locus PB of the locus group G instructed by the user is displayed in the frequency-time region 52, and each selected locus PB of the remaining locus group G is controlled to be hidden. As can be understood from the above description, when attention is paid to the display of each selected locus PB, it can be said that a separate layer is set for each locus group G. An analysis image 50 is formed by overlapping a plurality of layers on which the selected locus PB of each locus group G is arranged. In addition, the acoustic processing unit 30 controls the presence / absence of emphasis in the separation signal Y for each trajectory group G according to an instruction from the user. For example, the acoustic processing unit 30 generates the separation signal Y by selectively enhancing only the acoustic component corresponding to each selected locus PB of the locus group G instructed by the user from the acoustic signal X.

  In the second embodiment, the same effect as in the first embodiment is realized. In the second embodiment, since each selected locus PB is classified into a plurality of locus groups G and displayed in a different display manner for each locus group G, each selected locus PB (for example, specific locus) of a particular locus group G is displayed. There is an advantage that the user can intuitively grasp only the selection trajectory PB) corresponding to the acoustic component of the instrument. In addition, since the presence / absence of various processes is individually controlled for each trajectory group G, only the selected trajectory PB of the trajectory group G is heard while listening to the reproduction sound of the acoustic component corresponding to the selected trajectory PB of the specific trajectory group G. There is also an advantage that various and easy operations can be realized, such as selectively displaying and editing.

<Third Embodiment>
FIG. 5 is an explanatory diagram of the operation of the trajectory setting unit 28 in the third embodiment. As shown in part (A) of FIG. 5, the display control unit 24 of the third embodiment specifies each end point e0 of the candidate locus P0 corresponding to each pitch likelihood L (k, m) of the acoustic signal X. . The end point e 0 is a point where the pitch likelihood L (k, m) discontinuously or rapidly changes (increases or decreases) in the time direction in the frequency-time region 52. As understood from the above description, the start point and the end point of the sound component of the sound signal X correspond to the end point e0.

  As shown in part (A) of FIG. 5, the user designates the end point eA of the instruction locus PA in the peripheral area C (for example, a circular area having a predetermined diameter centered on the end point e0) C of the candidate locus P0. In this case, the trajectory setting unit 28 sets a selection trajectory PB along the candidate trajectory P0 with the end point e0 of the candidate trajectory P0 as an end point, as shown in part (B) of FIG. That is, the end point eA of the designated locus PA designated by the user is snapped to the end point e0 of the candidate locus P0. When the end point eA of the instruction locus PA is designated outside the peripheral area C of the end point e0 of each candidate locus P0, the selection locus PB having the end point eA of the instruction locus PA as an end point is set.

  In the third embodiment, the same effect as in the first embodiment is realized. In the third embodiment, since the end point of the selection locus PB is automatically adjusted to the end point e0 of the candidate locus P0, the selection locus PB from the start point to the end point of the sound component of the acoustic signal X is used by the user. Can be set easily. Therefore, there is an advantage that each acoustic component of the acoustic signal X can be emphasized with high accuracy by the separated signal Y from the start point to the end point of the sound generation. The third embodiment can also be applied to the second embodiment.

<Fourth embodiment>
FIG. 6 is a schematic diagram of an analysis image 50 in the fourth embodiment. In FIG. 6, the selection trajectory PB is omitted for convenience. As shown in FIG. 6, the display control unit 24 of the fourth embodiment includes a portion (hereinafter referred to as “steady portion”) that extends substantially parallel to the time axis AT in each candidate locus P 0 in the frequency-time region 52. Q1 and a portion other than the steady portion Q1 (hereinafter referred to as “variable portion”) Q2 are displayed in different display modes (solid line / broken line in FIG. 6). Specifically, the steady portion Q1 of each candidate locus P0 is highlighted (for example, displayed with a thick line or high gradation) with respect to the varying portion Q2. The display control unit 24 can discriminate between the steady portion Q1 and the variable portion Q2, but, for example, the section in which the frequency is continuously maintained within a predetermined width band in the candidate trajectory P0 is steady. A method of determining the portion Q1 is preferable. The point that the selection trajectory PB along the candidate trajectory P0 is set according to the instruction trajectory PA designated by the user is the same as in each of the embodiments described above.

  The degree of time variation of the pitch of each acoustic component is different for each acoustic component (each musical piece part). For example, the singing sound responsible for the main melody of the song is likely to vary in pitch due to vibrato, portamento, etc., but the instrumental sound responsible for accompaniment of the song (for example, the performance sound of a piano, guitar, bass, etc.) There is a general tendency that the pitch hardly changes between pronunciations. In the fourth embodiment, the user visually distinguishes between a steady portion Q1 (for example, a portion that is highly likely to be an accompaniment performance sound) and a variation portion Q2 (for example, a portion that is likely to be a main melody singing sound). However, there is an advantage that the instruction locus PA (and the selection locus PB) can be specified.

<Fifth Embodiment>
The user can designate a specific harmonic structure by appropriately operating the input device 16 with respect to the acoustic analysis device 100 of the fifth embodiment. For example, the harmonic structure of a closed wind instrument such as a clarinet can be specified.

  In the frequency-time region 52 of the analysis image 50 displayed on the display device 14 by the display control unit 24, in addition to the candidate trajectory P0 corresponding to the pitch of each acoustic component of the acoustic signal X, it corresponds to the harmonic frequency of each pitch. Candidate locus P0 can also be displayed at a position on frequency axis AF. The display control unit 24 highlights the candidate trajectory P 0 corresponding to the harmonic structure specified by the user among the plurality of candidate trajectories P 0 in the frequency-time region 52. For example, in the harmonic structure of a closed wind instrument, odd harmonics tend to be more prominent than even harmonics. Therefore, when the user designates the harmonic structure of the closed wind instrument, the display control unit 24 is located at a frequency that is an odd multiple of the specific candidate locus P0 (for example, the candidate locus P0 in which the selection locus PB is set). Candidate locus P0 is highlighted. Further, in the harmonic structure of wind instruments such as oboe, there is a tendency that the lower harmonic component is more remarkable than the fundamental component. Therefore, when the user specifies the harmonic structure of a wind instrument such as oboe, the display control unit 24 determines that the pitch likelihood L (k, m) of the lower harmonic component is the pitch likelihood L (k, m) of the fundamental component. , m), each candidate trajectory P0 having a higher relationship is highlighted.

  In the fifth embodiment, the same effect as in the first embodiment is realized. In the fifth embodiment, the candidate trajectory P0 corresponding to the specific harmonic structure among the plurality of candidate trajectories P0 in the frequency-time region 52 is highlighted, so that the selected trajectory corresponding to the desired harmonic structure is selected. There is an advantage that the user can easily select PB. Note that the fifth embodiment can be applied to the second to fourth embodiments.

<Sixth Embodiment>
FIG. 7 is an explanatory diagram of the operation of the trajectory setting unit 28 in the sixth embodiment. The user operates the input device 16 to instruct movement of the desired selection locus PB in the frequency-time region 52 in the direction of the frequency axis AF (for example, drag the existing selection locus PB with the mouse of the input device 16). Is possible).

  When the movement in the direction of the frequency axis AF is instructed with respect to the selection locus PB [1] illustrated in the part (A) of FIG. 7, the locus setting unit 28 is as illustrated in the part (B) of FIG. Then, the selection trajectory PB [2] is set along the candidate trajectory P0 which has a harmonic overtone relationship (a relationship of an integral multiple of the pitch indicated by the selection trajectory PB [1]) with respect to the selection trajectory PB [1]. Specifically, the trajectory setting unit 28 is in the vicinity of the destination specified by the user among a plurality of candidate trajectories P0 corresponding to each frequency that is an integral multiple of the pitch indicated by the selection trajectory PB [1] before the movement. The selection trajectory PB [2] is set along the candidate trajectory P0. That is, it can also be said that the selected locus PB [1] is snapped to the destination candidate locus P0. The section of the selected locus PB [2] on the time axis AT is the same as the selected locus PB [1]. However, as in the third embodiment, the end point of the selected locus PB [2] can be adjusted to the end point e0 of the destination candidate locus P0. Further, as understood from the part (B) in FIG. 7, when the movement destination selection locus PB [2] is set, the movement source selection locus PB [1] is deleted. In the above example, the selection trajectory PB [1] is automatically snapped to the destination candidate trajectory P0. However, the selection trajectory PB [1] is simply translated to a harmonic-related frequency (that is, the selection trajectory). A configuration in which the shape of PB does not change before and after movement can also be employed.

  In the sixth embodiment, the same effect as in the first embodiment is realized. Further, in the sixth embodiment, since the existing selection locus PB [1] is moved to the selection locus PB [2] having a harmonic relationship, the selection locus PB is shifted to a frequency shifted by one octave from the user's desired pitch. Is set (for example, when the user specifies the designated locus PA at a position shifted by one octave from the pitch of the desired acoustic component), the position of the selected locus PB is easily corrected to the user's desired frequency. There is an advantage that you can. Note that the sixth embodiment can be applied to the second to fifth embodiments.

  In the above example, the movement in the form of deleting the selection trajectory PB [1] before the movement is illustrated, but the selection trajectory PB [2] after the movement is set while maintaining the selection trajectory PB [1] before the movement. (Replication) is also possible. Further, it is possible to easily set the selection trajectory PB corresponding to the fundamental component and each harmonic component by repeating the duplication of the selection trajectory PB a plurality of times. The trajectory classification unit 32 of the second embodiment also classifies the selected trajectory PB [1] and each selected trajectory PB [2] duplicated into one trajectory group G (second embodiment). Is preferred. As understood from the above description, the “movement” of the selection trajectory PB includes “movement” in a narrow sense in which the selection trajectory PB before the operation is deleted, and “duplication” in which the selection trajectory PB before the movement is maintained. It is a concept that includes both.

  In the above example, the configuration in which the selection trajectory PB [2] having a harmonic relationship of an arbitrary order with respect to the selection trajectory PB [1] can be set is illustrated. A configuration that can set whether or not the selected trajectory PB [1] can be moved for each order in accordance with an instruction from the user may be employed. For example, for a closed wind instrument such as a clarinet, it is permitted to set the selected trajectory PB [2] after moving to the position of the frequency of the odd-order overtone with respect to the frequency of the selected trajectory PB [1]. Setting the selection trajectory PB [2] at the position of the harmonic frequency is prohibited. According to the above structure, there exists an advantage that the some selection locus | trajectory PB corresponding to a specific harmonic structure can be set easily.

<Modification>
Each form illustrated above can be variously modified. Specific modifications are exemplified below. Two or more aspects selected from the following examples can be appropriately combined as long as they do not contradict each other.

(1) In each of the above embodiments, the separation signal Y is generated by applying the separation filter corresponding to each selection locus PB to the acoustic signal X. However, the method by which the acoustic processing unit 30 generates the separation signal Y is appropriately changed. Is done. For example, in each of the above embodiments, a binary filter in which each coefficient value is set to binary is exemplified as a separation filter, but a filter (for example, a Wiener filter) in which each coefficient value is set in multiple values is used as a separation filter. It is also possible to do.

  In addition, a configuration in which the acoustic processing unit 30 generates the separated signal Y indicating the pure tone (beep sound) corresponding to the selected locus PB independently of the acoustic signal X is also employed. It is also possible for the acoustic processing unit 30 to generate the separated signal Y of the mixed sound of the pure tone having the frequency corresponding to the selection locus PB and each pure tone having the harmonic frequency independently of the acoustic signal X. As described above, according to the configuration that simply generates the separation signal Y (the configuration that does not execute the acoustic processing on the acoustic signal X), the acoustic processing unit is compared with the configuration that generates the separation filter and acts on the acoustic signal X. There is an advantage that the processing load of 30 is reduced. It is also possible for the acoustic processing unit 30 to selectively execute generation of the separation signal Y using the separation filter and simple generation of the pure tone separation signal Y in accordance with an instruction from the user. As can be understood from the above description, the acoustic processing unit 30 is included as an element that generates the separation signal Y having a pitch corresponding to the selected locus PB.

(2) In each of the above-described embodiments, as described with reference to FIG. 3, the pitch likelihood L (k, m) within a range R including one point pA on the designated locus PA designated by the user. Although the array of the points p0 on the candidate locus P0 that maximizes is set as the selection locus PB, the method by which the locus setting unit 28 sets the selection locus PB is not limited to the above example. For example, as shown in FIG. 8, a distribution D of weight values w (w [1], w [2]) is set in the direction of the frequency axis AF for each point pA on the indicated locus PA, and the frequency-time domain 52. The candidate trajectory in which the numerical value obtained by adding (for example, multiplying) the weight value w at the point p0 in the distribution D to the pitch likelihood L (k, m) of the point p0 on each candidate trajectory P0 in 52 is maximized. It is also possible for the locus setting unit 28 to set the array of the points p0 on P0 as the selected locus PB.

  For example, when attention is paid to the candidate trajectory P0 [1] and the candidate trajectory P0 [2] illustrated in FIG. 8, the point p0 is set to the pitch likelihood L (k1, m) of the point p0 on the candidate trajectory P0 [1]. The value w [1] · L (k1, m) multiplied by the weight value w [1] and the pitch likelihood L (k2, m) of the point p0 on the candidate trajectory P0 [2] at the point p0 Is compared with the numerical value w [2] · L (k2, m) multiplied by the weight value w [2] of the point p0 (candidate locus P0 [1] and candidate locus P0 [2] corresponding to the larger one) One point p0) is adopted as a point on the selection locus PB. The distribution D of the weight value w is selected so that the weight value w is maximized at the point pA on the designated locus PA, and the weight value w becomes smaller as the distance from the point pA increases. For example, a known probability distribution such as a normal distribution can be suitably employed as the weight distribution D. In the above configuration, a candidate trajectory P0 that is low in pitch likelihood L (k, m) but close to the point pA on the indicated trajectory PA is selected as the selected trajectory PB, so that the user's intention (instructed trajectory PA) is sufficient. There is an advantage that the selection trajectory PB reflected in can be set.

(3) A configuration in which the separation signal Y is individually generated for each of the plurality of selection trajectories PB set in the frequency-time region 52 and stored in the storage device 12 may be employed. For example, the acoustic processing unit 30 generates a separation signal Y for each selected trajectory PB by applying a separation filter corresponding to the selected trajectory PB to a section corresponding to each selected trajectory PB in the acoustic signal X. 12.

  Then, the acoustic processing unit 30 supplies the separation signal Y of each selection locus PB to the sound emitting device 18 and reproduces it in accordance with an instruction from the user with respect to the frequency-time region 52. For example, the user moves a specified point in the frequency-time region 52 (for example, the position of the mouse pointer or the touch position when the touch panel is used as the input device 16) along the specific selection locus PB by the operation of the input device 16. Then, the sound processing unit 30 supplies the separation signal Y corresponding to the selected locus PB to the sound emitting device 18 at a reproduction speed corresponding to the moving speed of the designated point. Although a known technique can be arbitrarily adopted for controlling the reproduction speed, a technique (for example, a phase vocoder) that changes the reproduction speed so that the pitch of the reproduced sound is maintained at the pitch of the acoustic signal X is suitable. is there. According to the above structure, there exists an advantage that the user's desired acoustic component (for example, specific musical instrument sound) can be reproduced | regenerated at arbitrary speeds among the acoustic signals X. FIG.

  It is also possible to control the reproduction direction (forward direction / reverse direction) of the separation signal Y according to the moving direction of the designated point. For example, the acoustic processing unit 30 reproduces the separation signal Y in the forward direction when the designated point is moved downstream of the time axis AT (the direction in which the time elapses), and upstream of the time axis AT (the time When the designated point is moved in the retroactive direction), the separation signal Y is reproduced in the reverse direction.

(4) In each of the above-described embodiments, the input device 16 that is separate from the display device 14 is illustrated. However, a touch panel that is integrated with the display device 14 may be employed as the input device 16. For example, the user can specify the instruction locus PA by a drag operation (an operation of tracing the display surface with a finger) on the display surface (operation surface) of the display device 14. However, when the drag operation on the display surface is assigned to the scroll of the frequency-time region 52, it is necessary to distinguish between the drag operation for designating the instruction locus PA and the drag operation for scrolling the frequency-time region 52. For example, when a predetermined tap operation (for example, a double tap or a long tap) is given to the frequency-time region 52, the state transitions to a reception state in which the designation of the instruction locus PA is accepted, and the instruction reception unit 26 is changed by the user. The designation of the instruction locus PA according to the drag operation is accepted. In the reception state, a configuration in which the display mode of each candidate locus P0 is changed (for example, each candidate locus P0 is blinked) is preferable. It is also possible to permit the setting of the selection locus PB (designation of the instruction locus PA) only within the band corresponding to the interval between the two fingers in contact with the display surface of the frequency axis AF. A configuration in which the display control unit 24 changes the display magnification of the frequency-time region 52 (zoom in / zoom out) in accordance with a pinch-in operation or a pinch-out operation that changes the interval between two fingers in contact with the display surface is also suitable. .

  In addition, the selected locus PB is changed to a selected state by a predetermined touch operation (for example, a double tap or a long tap) on the selected locus PB set in the frequency-time region 52, and processing related to the selected locus PB in the selected state is performed. Can be instructed by a touch operation on the display surface. For example, when a tap operation (operation to tap the display surface with a finger) is given to the selected trajectory PB, a configuration in which the acoustic component of the selected trajectory PB is reproduced, or two touches on the display surface A configuration in which the selected trajectory PB in the selected state is expanded or contracted in the direction of the time axis AT according to the finger interval, or a flick operation (an operation to play the finger touching the display surface) is given to the selected trajectory PB in the selected state In such a case, a configuration is adopted in which the selected locus PB is deleted. Further, when a drag operation with two fingers is given to the selection locus PB, the selection locus PB is moved, and when a drag operation with one finger is given to the selection locus PB. It is also possible to change the editing contents for the selected trajectory PB in the selected state according to the type of touch operation on the display surface, such as duplicating the selected trajectory PB. Note that the selection state of the selection locus PB is released when a predetermined tap operation (for example, a tap operation at a position away from the selection locus PB in the selection state) is given to the display surface.

  A configuration in which an arbitrary selection locus PB set in the frequency-time region 52 is selected according to a touch operation on the display surface is also suitable. For example, a configuration for selecting a selection locus PB in a rectangular area (rubber band) diagonally with two fingers in contact with the display surface, or a selection in a closed region that passes through three or more contact points on the display surface A configuration for selecting the locus PB is employed.

(5) In the above-described embodiments, the configuration in which the acoustic component corresponding to the selected locus PB in the acoustic signal X is exemplified, but the acoustic signal X is separated by suppressing the acoustic component corresponding to the selected locus PB. It is also possible to generate the signal Y. For example, the acoustic processing unit 30 sets the coefficient value corresponding to the pitch indicated by the selection trajectory PB and the overtone frequency among a plurality of coefficient values corresponding to different frequencies to 0 and sets the remaining coefficient value to 1. The separated filter is applied to the acoustic signal X.

(6) In each of the above embodiments, the pitch likelihood L (k, m) calculated from the acoustic signal X is displayed in the frequency-time region 52. For example, a known pitch detection technique (single pitch detection or multiple pitch) It is also possible to display the time variation of the pitch detected from the acoustic signal X using the detection) as a candidate locus P 0 in the frequency-time region 52. As can be understood from the above description, the candidate trajectory P0 is included as a trajectory that expresses a time change in pitch specified (typically estimated or detected) from the acoustic signal X.

(7) In each of the above-described embodiments, the acoustic analysis apparatus 100 including the acoustic processing unit 30 is illustrated. However, according to the instruction (instruction trajectory PA) from the user for each candidate trajectory P0 in the frequency-time region 52. Focusing only on the function of setting the selection locus PB, the acoustic processing unit 30 can be omitted. Further, in the configuration in which the pitch likelihood L (k, m) is notified from the external device, the analysis processing unit 22 is omitted.

DESCRIPTION OF SYMBOLS 100 ... Acoustic analysis device, 10 ... Arithmetic processing device, 12 ... Memory | storage device, 14 ... Display device, 16 ... Input device, 18 ... Sound emission device, 22 ... Analysis processing part, 24 ... Display Control unit 26... Instruction accepting unit 28... Trajectory setting unit 30... Acoustic processing unit 32... Trajectory classification unit 50 .. Analyzed image 52. , PA: Instruction locus, PB: Selection locus.

Claims (9)

Display control means for displaying, on a display device, an analysis image in which a plurality of candidate trajectories expressing a time change of a pitch specified from an acoustic signal are arranged in a frequency-time domain;
Instruction accepting means for accepting designation of an instruction locus for the frequency-time domain from a user;
A trajectory setting means for displaying a selection trajectory along the candidate trajectory at a position corresponding to the indicated trajectory in the frequency-time region ;
The trajectory setting means sets the selection trajectory along the candidate trajectory with the end point of the candidate trajectory as an end point when the end point of the instruction trajectory is designated in the peripheral region of the end point of the candidate trajectory.
Acoustic analysis device. The acoustic analysis apparatus according to claim 1, wherein the display control unit displays a stationary part parallel to a time axis and a changing part other than the stationary part in the candidate trajectories in different display modes. Display control means for displaying, on a display device, an analysis image in which a plurality of candidate trajectories expressing a time change of a pitch specified from an acoustic signal are arranged in a frequency-time domain;
Instruction accepting means for accepting designation of an instruction locus for the frequency-time domain from a user;
A trajectory setting means for displaying a selection trajectory along the candidate trajectory at a position corresponding to the indicated trajectory in the frequency-time region ;
The display control means displays the stationary part parallel to the time axis in the candidate trajectories and the changing part other than the stationary part in different display modes.
Acoustic analysis device. The trajectory setting unit sets a selection trajectory along a candidate trajectory having a harmonic overtone with respect to the one selected trajectory when the user instructs to move the existing one selected trajectory in the frequency axis direction. The acoustic analysis device according to any one of claims 1 to 3 . Display control means for displaying, on a display device, an analysis image in which a plurality of candidate trajectories expressing a time change of a pitch specified from an acoustic signal are arranged in a frequency-time domain;
Instruction accepting means for accepting designation of an instruction locus for the frequency-time domain from a user;
A trajectory setting means for displaying a selection trajectory along the candidate trajectory at a position corresponding to the indicated trajectory in the frequency-time region ;
The trajectory setting unit sets a selection trajectory along a candidate trajectory having a harmonic overtone with respect to the one selected trajectory when the user instructs to move the existing one selected trajectory in the frequency axis direction. Do
Acoustic analysis device. The acoustic analysis apparatus according to claim 1, further comprising: an acoustic processing unit that generates a separation signal having a pitch corresponding to the selected locus. Computer system
An analysis image in which a plurality of candidate trajectories expressing the time change of the pitch specified from the acoustic signal are arranged in the frequency-time domain is displayed on the display device,
The designation of the instruction locus for the frequency-time domain is accepted from the user,
Displaying a selection trajectory along the candidate trajectory at a position corresponding to the indicated trajectory in the frequency-time region;
In the display of the selection locus, when the end point of the instruction locus is specified in the peripheral region of the end point of the candidate locus, the selection locus along the candidate locus is set with the end point of the candidate locus as an end point.
Acoustic analysis method. Computer system
An analysis image in which a plurality of candidate trajectories expressing the time change of the pitch specified from the acoustic signal are arranged in the frequency-time domain is displayed on the display device,
The designation of the instruction locus for the frequency-time domain is accepted from the user,
Displaying a selection trajectory along the candidate trajectory at a position corresponding to the indicated trajectory in the frequency-time region;
In the display of the analysis image, the stationary part parallel to the time axis and the fluctuation part other than the stationary part are displayed in different display modes among the candidate trajectories.
Acoustic analysis method. Computer system
An analysis image in which a plurality of candidate trajectories expressing the time change of the pitch specified from the acoustic signal are arranged in the frequency-time domain is displayed on the display device,
The designation of the instruction locus for the frequency-time domain is accepted from the user,
Displaying a selection trajectory along the candidate trajectory at a position corresponding to the indicated trajectory in the frequency-time region;
In the display of the selection trajectory, when the user instructs to move one existing selection trajectory in the frequency axis direction, the selection trajectory along the candidate trajectory having a harmonic overtone relationship with respect to the one selection trajectory is displayed. Set
Acoustic analysis method.

Images