JP5135982B2 - Music processing apparatus and program - Google Patents

Description

  The present invention relates to a technique for processing music.

For example, a disc mug (DJ) reproduces one after another while connecting a plurality of music pieces without interruption. In order to realize such reproduction, for example, Patent Document 1 discloses a technology for smoothly connecting a plurality of pieces of music by controlling the timing of each reproduction so that the beat positions of successive pieces of music coincide with each other. It is disclosed.
JP 2003-108132 A

  By the way, in order to organize a music piece having a natural and sophisticated atmosphere from a plurality of music pieces, selection of an appropriate piece of music is an important requirement in addition to the adjustment of the timing for reproducing the music piece. That is, even if the beat positions of the music pieces are simply matched as in the technique of Patent Document 1, for example, if the musical features of the music pieces are significantly different, a natural music piece is formed in terms of hearing. Can not do it. Against the background of the above circumstances, one object of the present invention is to solve the problem of generating music that does not feel uncomfortable from a plurality of music.

In order to solve the above problems, a configuration adopted by the music processing device according to the present invention will be described .

In the configuration exemplified below, since the sub-element selected according to the similarity index value with the main element is used for processing the main element, the user can compare the similarity and harmony of each piece of music. Even if it is not known, it is possible to generate a natural musical piece in terms of hearing without significantly impairing the tone of the main musical piece. In addition, since a chart showing the feature quantity of the main element and the feature quantity of the sub-element is displayed on the display device, the user can see the magnitude of the feature quantity of each of the main music and the sub-music by visually checking the chart. Can be grasped intuitively.

  In a preferred aspect of the present invention, the segment is, for example, a section in which music is segmented at the time of synchronization with the beat of the music. More specifically, a section in which music is divided with one or more beats as a unit, or a section in which the interval between successive beats is divided into a plurality of times (for example, a time length corresponding to 1/2 beat or 1/4 beat). (Section) is a segment. According to this aspect, since the section into which the music is divided at the time of synchronization with the beat is used as a segment, a natural musical sound can be generated while maintaining the rhythm feeling of the main music. However, a segment defined asynchronously with the beat of the music may be used as a segment.

The music processing device according to the first aspect of the present invention includes a storage unit that stores musical tone data of each segment into which a song is divided and an acoustic feature amount of each segment for each of a plurality of songs, and a main song Similarity calculation for each sub-element, for each sub-element, for each main element, for each main element, a similarity index value that is an index of similarity of the feature quantity with sub-elements that are other than the main element Means for sequentially selecting, for each main element, a plurality of sub-elements having similar similarity index values, and the selection means for the main element Processing means for sequentially processing based on the musical sound data of the selected sub-elements, the feature amount of the main element, and the feature values of each of the plurality of sub-elements selected for the main element by the selection means Display control means for displaying the chart shown on the display device. According to the above aspect, since the selection means selects a plurality of sub-elements for each main element, it is possible to generate a variety of musical sounds as compared with a configuration in which only one sub-element is selected. Also, by confirming the chart displayed on the display device, the user can easily grasp the size of the feature quantity between the main element and each sub element.

The music processing device according to the second aspect of the present invention is a storage means for storing the musical tone data of each segment into which the music is divided and a plurality of acoustic feature quantities of each segment for each of the plurality of songs, For each sub-element, for each sub-element, for each main element that is an element of the main musical piece, an index of similarity of the plurality of feature quantities with a sub-element that is an element other than the main element. Similarity calculation means for calculating, selection means for sequentially selecting sub-elements whose similarity index values are similar for each main element, and musical sound data of each main element are selected by the selection means. Processing means for sequentially processing based on the musical sound data of the sub-element selected for the element, the feature value of the main element, and the feature value of the sub-element selected by the selection means for the main element Display control means for individually displaying the chart on the display device for each of the plurality of feature values. . According to the above aspect, since the similarity between the main element and the sub-element is determined based on a plurality of feature quantities, the degree of similarity with the main music is compared with a configuration with one kind of feature quantity. However, there is an advantage that a sufficiently high music can be easily generated.

  The music processing device according to a preferred aspect of the present invention includes first setting means for variably setting the first adjustment value for each feature amount in accordance with an instruction from the user, and the similarity calculation means includes: The similarity index value between the main element and the sub-element is calculated by an operation including weighting for each feature amount according to the adjustment value. According to the above aspect, since weighting according to the first adjustment value is executed for each feature amount when calculating the similarity index value, the user appropriately selects the feature amount to be prioritized by the selection unit. There is an advantage that you can.

  In a more specific aspect, the first setting means sets the first adjustment value of each of the plurality of feature amounts indicating the same kind of characteristics of the segment to the same value. According to the above aspect, a complicated operation for individually adjusting the first adjustment values of each of the plurality of feature amounts showing the same type of characteristics is not necessary. However, a configuration in which the first adjustment value of each feature amount showing the same type of characteristic is individually adjusted is also preferable.

  In a preferred aspect of the present invention, the similarity calculation means calculates the similarity index value by replacing the feature quantity selected by the user among the plurality of feature quantities of the main segment with a numerical value designated by the user. To do. According to the above aspect, there exists an advantage that a user can adjust the music tone of the music which a music processing apparatus produces | generates precisely.

The music processing device according to the third aspect of the present invention includes a storage means for storing, for each of a plurality of music pieces, musical sound data of each piece into which music pieces are divided and an acoustic feature amount of each piece, and a user Second setting means for variably setting the second adjustment value for each piece of music in accordance with an instruction from and a sub-piece that is a piece other than the main piece for each main piece that is a piece of the main piece Is a means for calculating a similarity index value for each sub-element, which is an index of similarity of the feature quantity, and changes according to a second adjustment value set for the music to which each sub-element belongs. Similarity calculating means for calculating the similarity index value between the main element and the sub-element, and selecting means for sequentially selecting sub-elements having similar similarity index values for each main element; , Based on the musical sound data of the sub-elements selected by the selection means for the main element Processing means for sequentially processing, and display control means for displaying on the display device a chart showing the feature quantity of the main element and the feature quantity of the sub-element selected by the selection means for the main element. . In the above aspect, the similarity index value is adjusted for each piece of music according to the second adjustment value set by the second setting means. That is, the frequency with which the sub-pieces of each music piece are used for processing the main music piece increases or decreases according to the second adjustment value. Therefore, it is possible to generate various music pieces that match the user's intention.

The music processing apparatus according to the present invention is realized by hardware (electronic circuit) such as a DSP (Digital Signal Processor) dedicated to each processing, and a general-purpose arithmetic processing apparatus such as a CPU (Central Processing Unit) and a program It is also realized through collaboration with. Also according to the program of the present invention, the same operations and effects as the music processing device according to the present invention are exhibited. The program of the present invention is provided to the user in a form stored in a computer-readable recording medium and installed in the computer, or is provided in a form distributed via a communication network and installed in the computer. The

<A: First Embodiment>
FIG. 1 is a block diagram showing a configuration of a music processing device according to one embodiment of the present invention. The music processing apparatus 100 of this embodiment is an apparatus that processes a single music (hereinafter referred to as “main music”) using a plurality of music. As shown in FIG. 1, the music processing device 100 is realized by a computer system including a control device 10 and a storage device 30.

  The control device 10 executes various processes by executing a program and controls each element of the music processing device 100. The storage device 30 stores a program executed by the control device 10 and various data used by the control device 10. A known recording medium such as a semiconductor storage device or a magnetic storage device is arbitrarily employed as the storage device 30. As shown in FIG. 1, the storage device 30 stores song data for each of a plurality of songs.

  FIG. 2 is a conceptual diagram showing the composition of music. A section S is defined in the music. The section S is a part composed of a plurality of measures including, for example, a characteristic section (for example, a chorus section) in the music. For example, when the user operates the input device 42, the section S It is defined by setting the end point. However, the selection method of the section S and the time length of the section S are arbitrary. For example, the whole piece of music may be set as one section S, or a plurality of sections S may be extracted from one piece of music.

  As shown in FIG. 2, the section S is divided into a plurality of sections (hereinafter referred to as “elements”) P with the time point synchronized with the beat point of the music as a boundary. For example, a segment corresponding to one or a plurality of beats or a segment obtained by equally dividing one beat into a plurality of segments (that is, a segment corresponding to 1/2 beat or 1/4 beat) is used as the segment P. However, the segment P may be defined regardless of the beat point.

  As shown in FIG. 1, the music data of one piece of music includes, for each of a plurality of pieces P belonging to the section S of the piece of music, music data A representing the waveform of the musical sound belonging to the piece P, and the piece. And N types of feature values F (F1 to FN) reflecting the acoustic and musical features of P (N is a natural number). A specific example of each feature amount F will be described later. In this embodiment, all the music data is stored in one storage device 30, but a configuration in which a plurality of music data is distributed and stored in a plurality of storage devices, and the musical sound data A and feature amounts F1 to FN of each music. Are also stored in separate storage devices.

  As shown in FIG. 1, an input device 42, a display device 44, and a sound emitting device 46 are connected to the control device 10. The input device 42 is a device (for example, a mouse or a keyboard) for a user to input an instruction to the music processing device 100. For example, by appropriately operating the input device 42, the user designates M pieces of music (M is a natural number) to be processed by the music processing device 100, and selects the main piece of music from among the M pieces of music. select. The display device 44 displays an image under the control of the control device 10.

  The sound emitting device 46 emits sound based on the musical sound data A sequentially output from the control device 10. For example, the sound emitting device 46 includes a D / A converter that generates an analog signal from the musical sound data A, an amplifier that amplifies a signal output from the D / A converter, and a sound wave corresponding to the signal output from the amplifier. Including sound output devices (speakers and headphones).

  Next, specific functions of the control device 10 will be described. As shown in FIG. 1, the control device 10 executes a program stored in the storage device 30 to thereby execute a plurality of elements (similarity calculation unit 12, selection unit 14, processing unit 16, display control unit 18, first control unit 18). It functions as a setting unit 21 and a second setting unit 22). Note that each element of the control device 10 is also realized by an electronic circuit such as a DSP dedicated to musical tone processing. Further, the control device 10 may be distributed and mounted on a plurality of integrated circuits.

  The similarity calculation unit 12 uses, for each of a plurality of pieces P (hereinafter referred to as “main piece Pm”) constituting the main piece, elements other than the main piece Pm in M pieces of music (including the main piece). A similarity index value R, which is an index of similarity between the feature quantities F1 to FN of the piece (hereinafter referred to as “subelement piece Ps”) and the feature quantities F1 to FN of the main piece Pm, is calculated for each subelement Ps. . The selection unit 14 selects, for each main element Pm of the main musical piece, a predetermined number (four in this embodiment) of sub-elements Ps whose similarity index values R between the main elements Pm are similar. . For example, the top four sub-elements Ps are selected in descending order of similarity indicated by the similarity index value R.

  The processing unit 16 sequentially selects each main element Pm in the order of arrangement in the main music piece, and the musical piece data A of the main element Pm is selected as a predetermined number of sub-elements selected by the selection unit 14 for the main element Pm. After replacing the musical tone data A of the segment Ps, the data is output sequentially. More specifically, the processing unit 16 first acquires the musical sound data A of each sub-element Ps selected by the selection unit 14 from the storage device 30, and secondly, a time substantially equal to the main element Pm. The musical tone data A of each sub-element Ps is squeezed along the time axis so as to be long. For the companding of the musical sound data A, a known technique for adjusting the time length without changing the pitch of the musical sound is arbitrarily adopted. Thirdly, the processing unit 16 adds the musical sound data A of each sub-element Ps after the companding, and sequentially outputs it to the sound emitting device 46. Therefore, instead of the musical sound of each main element Pm of the main musical piece, the musical sound of another sub-element Ps whose acoustic characteristics are similar to the main element Pm is reproduced from the sound emitting device 46. When the processing is completed for the last main element Pm in the section S of the main musical piece, the processing unit 16 repeats the above processing in order from the first main element Pm of the section S.

  The display control unit 18 is means for displaying various images on the display device 44. The display control unit 18 of this embodiment continuously displays the status screen 441 of FIG. 3 and the operation screen 442 of FIG.

  The state screen 441 shown in FIG. 3 is a screen for displaying the magnitudes of the feature quantities F1 to FN for each of the main element Pm and the four sub-elements Ps selected by the selection unit 14 for the main element Pm. N unit areas U (U1 to UN) corresponding to each of the feature amounts F1 to FN are arranged. The status screen 441 displays the main element Pm each time the main element Pm to be processed according to the sub-element Ps is sequentially selected from the main music (that is, as the main music progresses). Are sequentially updated in accordance with the feature amounts F1 to FN and the feature amounts F1 to FN of each sub-element Ps.

  Specific contents of each feature amount F exemplified in the status screen 441 of FIG. 3 will be described below. The acoustic energy (Energy) displayed in the unit region U1 as the feature amount F1 is the energy of the musical sound (sound wave) in each segment P. The low-frequency acoustic energy ratio (Energy Band Ratio Bass) displayed in the unit region U2 as the feature amount F2 is a relative ratio of acoustic energy of components of 100 Hz or less to the sum of acoustic energy of the segments P. The middle band energy energy ratio (Energy Band Ratio Middle) displayed in the unit area U3 as the feature amount F3 is a relative ratio of the acoustic energy of the middle band component to the sum of the acoustic energy of the segments P. The spectral centroid displayed in the unit region U4 as the feature amount F4 is a frequency corresponding to the geometric centroid in the power spectrum of the musical sound in the segment P.

  PCP (Pitch Class Profile) is a spectrum (octave profile) obtained by dividing the power spectrum of the musical sound of the segment P into frequency bands corresponding to one octave and superimposing each of the frequency bands (hereinafter referred to as octave profiles). Intensity (for example, the intensity of the peak in the pitch class). The intensity in one pitch class corresponds to one feature amount F. In this embodiment, a frequency band obtained by further dividing the 12 frequency bands corresponding to semitones among the frequency bands of one octave into three equal parts is defined as a pitch class. Therefore, the PCP is displayed in the unit areas U5 to U40 as 36 feature amounts F5 to F40. PCP is described in detail in JP-A-2000-298475.

  The MFCC (Mel-Frequency Cepstrum Coefficients) expresses the acoustic characteristics of the musical tone of the segment P by a plurality (in this embodiment, 13) of numerical values. One numerical value of MFCC corresponds to one feature amount F. That is, the MFCC is displayed in the unit areas U41 to U53 as thirteen feature amounts F41 to F53.

  FIG. 4 is an enlarged schematic view of the unit area Ui corresponding to the feature amount Fi (i = 1 to N). Each of the unit areas U1 to UN includes a first area 51 and a second area 52 arranged in the vertical direction. In the first area 51 of the unit area Ui, a chart showing the feature quantity Fi of the main element Pm and the feature quantities Fi of each of the four sub-elements Ps selected by the selection unit 14 for the main element Pm. Be placed. In the first region 51 of this embodiment, the feature amount Fi of each segment P is displayed as a bar graph. That is, a white vertical line 511 whose length is selected according to the feature value Fi of the main element Pm and a colored (for example, red) color whose length is selected according to the feature value Fi of each sub-element Ps. Four vertical lines 512 are arranged in the horizontal direction.

  An operation element (slider) 521 is arranged in the second area 52 of the unit area Ui. The user can move the operator 521 up and down independently for each unit area Ui (for each feature value Fi) by appropriately operating the input device 42 (for example, dragging the mouse). In addition, the user gives a predetermined operation to the input device 42 (for example, by dragging the mouse while pressing a specific key), so that a plurality of operators 521 corresponding to each feature amount F showing the same kind of characteristics. Can be moved collectively to the same position. For example, the operating elements 521 in the 36 unit areas U5 to U40 corresponding to the PCP move to the same position all at once by appropriately operating any of the operating elements 521 in the unit areas U5 to U40.

  The first setting unit 21 in FIG. 1 sets an adjustment value X (X1 to XN) for each of the feature amounts F1 to FN. More specifically, the first setting unit 21 sets the adjustment value Xi of the feature value Fi according to the position of the operation element 521 in the unit area Ui on the status screen 441 (that is, according to an instruction from the user). The adjustment value Xi changes continuously according to the position of the operation element 521. Further, when a plurality of operation elements 521 corresponding to the same kind of feature amount F are collectively operated, the first setting unit 21 sets a plurality of adjustment values X corresponding to each feature amount F to the same value. For example, when the operation elements 521 of the unit areas U5 to U40 are collectively operated, the adjustment values X5 to X40 corresponding to the plurality of feature amounts F5 to F40 representing the PCP correspond to the positions of the operation elements 521. Set to a common number.

  Next, FIG. 5 is a schematic diagram illustrating a specific example of the operation screen 442. As shown in FIG. 5, on the operation screen 442, the name (Name) of M pieces of music (8 pieces in this embodiment) selected by the user and the M controls corresponding to each of the M pieces of music are displayed. (Slider) 56 is arranged. The user can move the operator 56 up and down independently for each music piece by appropriately operating the input device 42. The second setting unit 22 in FIG. 1 individually sets an adjustment value Y corresponding to the position of the operation element 56 corresponding to the music on the operation screen 442 for each of the M pieces of music. For example, the adjustment value Y is set to zero when the operation element 56 is at the lower end, and increases as the operation element approaches the upper end.

  Next, a specific configuration of the similarity calculation unit 12 in FIG. 1 will be described. The similarity calculation unit 12 includes a first calculation unit 121 and a second calculation unit 122. The first calculation unit 121 calculates a basic index value R0 that is a basis of the similarity index value R for each sub-element Ps of M pieces of music for each main element Pm. Similar to the similarity index value R, the basic index value R0 is a numerical value that is an index of similarity between the feature amounts F1 to FN of the main element Pm and the subelement Ps.

More specifically, the first calculation unit 121 determines the basic index of the main element Pm and the sub-element Ps so that the contribution (influence) to the basic index value R0 increases as the adjustment value Xi increases. Calculate the value R0. That is, the first calculation unit 121 calculates the basic index value R0 after performing weighting according to the adjustment value Xi for the feature value Fi. For example, when an index value Δi that increases as the difference between the feature quantity Fi of the main element Pm and the feature quantity Fi of the sub-element Ps increases, the basic index value R0 is expressed by the following equation (1). It is calculated as the reciprocal of the distance D.
D = (X1 · Δ1 + X2 · Δ2 + X3 · Δ3 + ... + XN · ΔN) ^1/2 (1)

  The index value Δi is, for example, the square of the difference value between the feature value Fi of the main element Pm and the feature value Fi of the sub-element Ps. That is, the basic index value R0 is the distance (Euclidean distance) between the coordinates specified in the N-dimensional space for each of the main segment Pm and the sub-segment Ps with the feature amounts F1 to FN as coordinate values, and the feature amount Fi. Is defined as the reciprocal of the distance D when weighted according to the adjustment value Xi. Therefore, the acoustic features of the main element Pm and the sub-element Ps are more similar as the basic index value R0 between them is larger.

  The second calculation unit 122 adjusts the basic index value R0 calculated for each sub-element Ps according to the adjustment value Y set by the second setting unit 22 for the music to which the sub-element Ps belongs. The rejection index value R is calculated. More specifically, the second calculation unit 122 calculates a multiplication value of the basic index value R0 calculated for each sub-element Ps of one musical piece and the adjustment value Y set by the second setting unit 22 for the musical piece. Calculated as similarity index value R.

  As described above, the musical sound data A of each main element Pm of the main music is processed (replaced) according to a plurality of sub-elements Ps whose acoustic features are similar to the main element Pm. Therefore, even if the user is not familiar with the similarity and harmony of each music piece, it is possible to generate a natural music piece in terms of hearing without greatly impairing the tone of the main music piece. In addition, each musical piece is divided into segments P in units of beats, and the sub-segment Ps selected by the selection unit 14 is adjusted to the time length of the main segment Pm and used for processing the main segment Pm. Therefore, the rhythm of the main song is not impaired.

  Further, since the feature quantity Fi of the main element Pm and the feature quantities Fi of each of the plurality of sub-elements Ps selected by the selection unit 14 are displayed as a chart on the status screen 441, the user displays the status screen 441. By visually recognizing, the characteristic amount Fi of the main element Pm and each sub-element Ps can be intuitively grasped. In particular, since the vertical line 511 of the feature value Fi of the main element Pm and the vertical line 512 of the feature value Fi of the sub-element Ps are displayed in different modes (colors), the feature value Fi of the main element Pm There is an advantage that the user can easily confirm the size of the feature value Fi of each sub-element Ps.

  Further, since the adjustment values X1 to XN of the feature amounts F1 to FN are variably set in accordance with an instruction from the user, priority is given to the determination of the similarity between the main element Pm and the subelement Ps. The user can select the characteristic amount F to be used. Therefore, it is possible to easily compose a musical piece having a musical tone intended by the user. For example, since the PCP reflects the harmony of the musical sound, as the adjustment values X5 to X40 for the PCP (feature values F5 to F40) are increased, the sub-element Ps whose harmony is similar to the main music is increased. The possibility that the selection unit 14 will select (in other words, the possibility that a musical piece with a sense of harmony similar to the main musical piece will be reproduced) increases. Alternatively, since the MFCC reflects the tone color of the musical tone, the selection unit 14 can select the sub-element Ps whose tone color is similar to that of the main music as the adjustment values X41 to X53 relating to the MFCC (feature amounts F41 to F53) are increased. Increases nature. Further, since the adjustment value X of each feature amount F is set by the movement of the operation element 521 on the status screen 441, there is an advantage that the user can intuitively grasp the feature amount F to be prioritized.

  Further, a plurality of similar feature quantities F such as PCP and MFCC can be collectively set to the same value according to an operation by a user. Accordingly, it is possible to eliminate the troublesome work of setting each feature amount F individually for PCP and MFCC. On the other hand, since it is also possible to individually set a plurality of the same kind of feature amounts F, convenience for a user who wants to set each adjustment value X finely is not impaired.

  Further, since the similarity index value R is adjusted for each piece of music according to the adjustment value Y, the selection unit 14 may select the sub-element Ps of the music set with a large value of the adjustment value Y ( Frequency). That is, by increasing or decreasing the adjustment value Y of a specific music piece by operating the input device 42, the frequency at which the main element Pm is replaced with the sub-element Ps of the music piece increases or decreases. Therefore, compared to a configuration in which the adjustment value Y is fixed (or a configuration in which the basic index value R0 is output to the selection unit 14 as the similarity index value R), various musical compositions are organized in accordance with the user's intention. It is possible. In addition, since the adjustment value Y of each music piece is variably set by the movement of each operation element 56 on the operation screen 442, there is also an advantage that the user can intuitively grasp the music outputted preferentially.

<B: Second Embodiment>
Next, a second embodiment of the present invention will be described. In addition, about the element which an effect | action and function are equivalent to 1st Embodiment in this form, the same code | symbol as the above is attached | subjected and each detailed description is abbreviate | omitted suitably.

  FIG. 6 is a schematic diagram showing a specific example of the status screen 441. The status screen 441 includes N unit areas U1 to UN corresponding to the feature amounts F1 to FN, as in the first embodiment. FIG. 7 is an enlarged schematic view of one unit region Ui. As shown in FIG. 7, the unit region Ui includes a third region 53 in addition to the first region 51 and the second region 52 similar to those in the first embodiment. A manipulator (slider) 531 is arranged in the third region 53. The user can move the operation element 531 up and down independently for each unit area Ui (for each feature value Fi) by appropriately operating the input device 42 during reproduction of music.

  The first setting unit 21 sets the adjustment value Xi according to the position of the operation element 521 in the second area 52 and sets the instruction value Zi for the feature amount Fi according to the position of the operation element 531 in the third area 53. To do. Similar to the operation element 521, in addition to the operation of independently moving each operation element 531, an operation for moving a plurality of operation elements 531 corresponding to the same type of characteristic at a time is possible.

  As shown in FIG. 6, an operator 54 (button) is arranged in the vicinity of each unit area Ui. However, for a plurality of feature amounts F representing the same type of characteristics (for example, PCP and MFCC), one operator 54 is arranged for a plurality of unit areas Ui corresponding to the feature amount F.

  When any one of the operators 54 is operated, the display control unit 18 displays a selection image 55 shown in FIG. The selection image 55 is an image for the user to select a method for calculating the similarity index value R (basic index value R0). In the selection image 55, a plurality of options are arranged. The user can select any option for one feature quantity F or a plurality of feature quantities F showing the same kind of characteristics by appropriately operating the input device 42. The selection image 55 includes “Euclid (Euclidean distance)”, “Cosine”, “Ignore”, and “Replace” as options.

  For one or a plurality of feature values F for which “Euclid” is selected, the first calculation unit 121 uses the same method (Equation (1)) as the first embodiment to determine the distance between the main element Pm and the sub-element Ps. D is calculated, and the reciprocal of the distance D is selected as the index value r1. An index value r1 is calculated for each sub-element Ps for each main element Pm.

  For a plurality of feature amounts F for which “Cosine” is selected, the first calculation unit 121 uses a vector having each feature amount F of the main element Pm as an element and each feature amount F of the sub-element Ps as an element. The cosine cos θ of the elevation angle θ with the vector to be calculated is calculated for each sub-element Ps as the index value r2. For example, when “Cosine” is selected for a plurality of feature amounts F5 to F40 representing PCP, the first calculation unit 121 uses 36-dimensional vectors and subelements defined by the feature amounts F5 to F40 of the main element Pm. The index value r2 is calculated based on the vector defined by the feature values F5 to F40 of the piece Ps. As the feature values F of the main element Pm and the feature values F of the sub-element Ps are more similar (the smaller the elevation angle θ of both vectors), the cosine cosθ becomes a larger value, so the index value r1 (distance D) Similarly, the index value r2 is an index of similarity between the main element Pm and the subelement Ps. In order to select the cosine cos θ, a plurality of feature amounts F are necessary. Therefore, selection of “Cosine” is prohibited for the feature amount F (acoustic energy and spectral center of gravity) that represents the characteristics of the single piece P.

  Each feature amount F for which “Ignore” is selected is ignored in calculating the basic index value R0. That is, the first calculation unit 121 calculates the basic index value R0 by setting the feature amount F to zero for both the main element Pm and the sub-element Ps. Note that the adjustment value Xi may be forcibly set to zero for the feature value Fi for which “Ignore” is selected.

  For one or a plurality of feature amounts F for which “Replace” is selected, the first calculation unit 121 uses the feature amount Fi of the main element Pm as an instruction value Zi (that is, a unit region) designated by the user for the feature amount Fi. (The numerical value corresponding to the position of the Ui operator 531). The feature amount F of the sub-element Ps is not replaced. The first calculation unit 121 determines the distance D between each sub-element Ps and the replaced main element Pm for the plurality of feature quantities F for which “Replace” has been selected by the same method (formula ( 1)), and the reciprocal of the distance D is selected as the index value r3. For a plurality of feature values F for which “Replace” is selected, the cosine cos θ between the sub-element Ps and the replaced main element is calculated as the index value r3 in the same manner as when “Cosine” is selected. May be.

  The first calculation unit 121 calculates a basic index value R0 based on the index value r1, the index value r2, and the index value r3. As in the first embodiment, the basic index value R0 increases as the main element Pm and the subelement Ps are similar (that is, as the index values (r1, r2, r3) increase). A basic index value R0 is calculated for each sub-element Ps for each main element Pm. Subsequent operations are the same as those in the first embodiment.

  As described above, in the present embodiment, since the calculation method of the basic index value R0 (and also the similarity index value R) is changed, each feature amount F and the calculation method of the basic index value R0 are described. By selecting various combinations on a trial basis, it is possible to generate music that exactly matches the user's intention.

<C: Modification>
Various modifications as exemplified below can be added to the above embodiments. Two or more aspects may be arbitrarily selected from the following examples and combined.

(1) Modification 1
In each of the above embodiments, each main element Pm of the main musical piece is always replaced with the sub-element Ps. However, the musical sound of the main element Pm may be reproduced. For example, when a maximum value among a plurality of similarity index values R calculated for each sub-element Ps for one main element Pm is below a predetermined threshold (sub-elements sufficiently similar to the main element Pm) When Ps does not exist), the selection unit 14 acquires the musical sound data A of the main segment Pm from the storage device 30 and outputs it to the sound emitting device 46. According to the above configuration, it is avoided that the tone of the reproduced sound is too far from the main element.

  The method by which the selection unit 14 selects the sub-element Ps is changed as appropriate. For example, the selection unit 14 selects only one sub-element Ps having the maximum similarity index value R, or the selection unit 14 selects any number of sub-elements Ps whose similarity index value R exceeds a threshold. The configuration to be selected is also adopted.

(2) Modification 2
In each of the above forms, the segment P other than the main segment Pm that is currently selected as the target of processing among the main songs is set as a candidate for selection by the selection unit 14 as the sub-segment Ps. A configuration in which only the segment P of the (M-1) pieces of music excluding the main song is the sub-segment Ps is also preferable. Since each element P in the same music often has similar acoustic characteristics, in the configuration of the first embodiment, the element P in the main music is used as the sub-element Ps similar to the main element Pm. Likely to be selected. On the other hand, according to the configuration in which the segment P of the main song is excluded from the selection target by the selection unit 14, it is possible to generate various songs using the segment P of the song other than the main song.

(3) Modification 3
In each of the above embodiments, the configuration in which the main element Pm is replaced with the sub-element Ps is illustrated, but the method of processing the main element Pm based on the sub-element Ps is not limited to the replacement of the musical sound data A. For example, the musical sound data A of the main element Pm and the musical sound data A of each sub-element Ps selected by the selection unit 14 may be mixed and output at a predetermined ratio.

(4) Modification 4
The method for calculating the similarity index value R is arbitrary. For example, in the above description, the similarity index value R increases as the similarity between the main element Pm and the sub-element Ps increases, but the similarity index value R includes the main element Pm and the sub-element Ps. It may be a numerical value that decreases as the degree of similarity increases. In addition, a configuration in which the first calculation unit 121 does not perform adjustment according to the adjustment value X, or a configuration in which the second calculation unit 122 is omitted (a configuration in which the basic index value R0 is output to the selection unit 14 as the similarity index value R). Is also adopted.

(5) Modification 5
The chart displaying the feature amount F of the main element Pm and each sub-element Ps is not limited to a bar graph. For example, a chart in which points are arranged at positions corresponding to the feature amounts F of the main element Pm and the sub-elements Ps, or a line graph in which the points are linearly connected may be employed.

(6) Modification 6
The type of feature amount F is not limited to the above examples. The position of the center of gravity of the energy distribution on the time axis of the musical sound in the segment P (Audio Centroid), the standard deviation (Dynamic Complexity) of the overall volume in the segment P, the intensity of the component whose frequency exceeds a predetermined value (High Frequency Content), the start time (Region Begin) and the end time (Region End) of the segment P for the music (section S) are employed. Further, HPCP (Harmonic Pitch Class Profile) may be used as the feature amount F. The HPCP is a numerical sequence obtained by superimposing the intensity of each peak in each of a plurality of frequency bands obtained by dividing the power spectrum of the musical sound of the segment P every octave (that is, PCP for only the peak of the power spectrum). ). Furthermore, the power spectrum of the musical sound in the segment P such as kurtosis (Spectral Kurtosis), distortion (Spectral Skewness), rough slope (Spectral Decrease), distribution width (Spectral Spread) in the spectrum of the musical tone in the segment P A numerical value characterizing the shape may be used as the feature amount F. In addition, as with the PCP and MFCC in the above-described form, the feature amount F representing one characteristic of a plurality of pieces P is the intensity (Barkbands) in each of a plurality of bands defined on the frequency axis, The energy of the segment P in each of a plurality of frequency bands (mel frequency bands) defined on the mel scale is adopted (a numerical group before discrete cosine transform at the time of MFCC calculation). A configuration in which the feature amount F of the element P is only one type is also employed.

  A configuration in which the feature value F of the sub-element Ps selected by the selection unit 14 is close to the feature value F of the main element Pm is also suitable. For example, the musical sound data A of the sub-element Ps so that the acoustic energy (feature amount F1) of the sub-element Ps selected by the selection unit 14 approaches (ideally matches) the acoustic energy of the main element Pm. Is used to increase or decrease the amplitude of the waveform represented by (for example, multiply by a predetermined value). In addition, the musical tone data A of the sub-element Ps is such that the distribution of intensity (barkbands) on the frequency axis of the musical sound of the sub-element Ps approaches (or matches) the distribution of the intensity of the musical sound of the main element Pm. According to the configuration in which filter processing (for example, filter processing using a bandpass filter bank) is performed on the waveform to be represented, the timbre of the sub-element Ps can be brought close to the main element Pm.

  A configuration in which the sub-element Ps selected by the selection unit 14 is reproduced with the pitch close to the main element F is also suitable. For example, by comparing the PCP (or HPCP) between each sub-element Ps and the main element Pm, the difference in pitch between them is identified in semitone units, and the sub-element Ps with little difference in pitch While the similarity calculation unit 12 calculates the similarity index value R (basic index value R0) so that the possibility of selection increases, the pitch of the sub-element Ps selected by the selection unit 14 is calculated with the main element Pm. A configuration in which the processing unit 16 converts (pitch shift) and reproduces according to the difference in pitch is also suitably employed. According to the above configuration, each sub-element Ps is reproduced in a state corrected to a pitch approximate to the melody of the main element Pm, so that a reproduction sound that sufficiently maintains the tone of the main music can be generated. it can. In the above description, the pitch of the sub-element Ps is converted so as to match the pitch of the main element Pm. However, a configuration in which the pitch of the sub-element Ps is converted by an arbitrary amount on the basis of the pitch of the main element Pm is also possible. Adopted.

It is a block diagram which shows the structure of the music processing apparatus which concerns on 1st Embodiment of this invention. It is a conceptual diagram for demonstrating the segment of a music. It is a schematic diagram which shows the specific example of a status screen. It is a schematic diagram which shows the specific example of a unit area | region. It is a schematic diagram which shows the specific example of an operation screen. It is a schematic diagram which shows the specific example of the status screen in 2nd Embodiment of this invention. It is a schematic diagram which shows the specific example of a unit area | region.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 ... Music processing apparatus, 10 ... Control apparatus, 12 ... Similarity calculation part, 121 ... 1st calculating part, 122 ... 2nd calculating part, 14 ... Selection part, 16 ... Processing part, 18 …… Display control unit, 21 …… First setting unit, 22 …… Second setting unit, 30 …… Storage device, 42 …… Input device, 44 …… Display device, 46 …… Sound emitting device, 441 …… Status screen, 442 ... operation screen, U (Ui) ... unit area.

Claims (9)

Storage means for storing the musical tone data of each segment into which the musical piece is divided and the acoustic feature amount of each segment for each of a plurality of musical pieces;
For each main element, for each main element, a similarity index value that is an index of the similarity of the feature quantity with each sub-element, which is an element other than the main element, is calculated. Non-calculation means,
Selecting means for sequentially selecting a plurality of sub-elements having similar similarity index values for each main element;
Processing means for sequentially processing the musical sound data of each main element based on the musical sound data of the sub-elements selected by the selecting means for the main element;
A music processing apparatus comprising: a display control unit configured to display on a display device a chart indicating a feature amount of the main element and a feature amount of each of the plurality of sub-elements selected by the selection unit for the main element. Storage means for storing the musical tone data of each segment into which the musical piece is divided and a plurality of acoustic feature quantities of each unit for each of the plurality of musical pieces;
For each sub-element, for each sub-element, for each main element that is an element of the main musical piece, an index of similarity of the plurality of feature quantities with a sub-element that is an element other than the main element. Similarity calculation means to calculate,
Selection means for sequentially selecting sub-elements having similar similarity index values for each main element;
Processing means for sequentially processing the musical sound data of each main element based on the musical sound data of the sub-elements selected by the selecting means for the main element;
Display control means for individually displaying on the display device for each of the plurality of feature quantities a chart showing the feature quantities of the main element and the feature quantities of the sub-elements selected by the selection means for the main element. A music processing device. Comprising first setting means for variably setting the first adjustment value for each feature amount in accordance with an instruction from the user;
The similarity calculation means calculates the similarity index value of the main element and the sub-element by a calculation including weighting for each feature amount according to the first adjustment value.
The music processing apparatus according to claim 2 . The first setting means sets a first adjustment value of each of a plurality of feature amounts indicating the same kind of characteristics of the segment to the same value.
The music processing apparatus according to claim 3 . The similarity calculation means calculates a similarity index value by replacing a feature amount selected by the user among a plurality of feature amounts of the main element with a numerical value designated by the user.
The music processing apparatus according to claim 2 . Storage means for storing the musical tone data of each segment into which the musical piece is divided and the acoustic feature amount of each segment for each of a plurality of musical pieces;
A second setting means for variably setting the second adjustment value for each song in accordance with an instruction from the user;
Means for calculating the similarity index value for each sub-element, which is an index of the similarity of the feature quantity with each sub-element, which is an element other than the main element, for each main element that is an element of the main musical piece A similarity calculation means for calculating a similarity index value between the main element and the sub-element so as to change according to a second adjustment value set for the music to which each sub-element belongs. ,
Selection means for sequentially selecting sub-elements having similar similarity index values for each main element;
Processing means for sequentially processing the musical sound data of each main element based on the musical sound data of the sub-elements selected by the selecting means for the main element;
A music processing device comprising: display control means for displaying on a display device a chart showing the feature quantity of the main element and the feature quantity of the sub-element selected by the selection means for the main element. A computer comprising storage means for storing the musical tone data of each segment into which a song is divided and the acoustic feature quantity of each segment for each of a plurality of songs,
For each main element, for each main element, a similarity index value that is an index of the similarity of the feature quantity with each sub-element, which is an element other than the main element, is calculated. Reject calculation processing,
A selection process for sequentially selecting a plurality of sub-elements having similar similarity index values for each main element;
A processing process that sequentially processes the musical sound data of each main element based on the musical sound data of the sub-elements selected for the main element in the selection process;
A program for executing a display control process for displaying on the display device a chart showing the feature quantity of the main element and the feature quantities of each of the plurality of sub-elements selected for the main element in the selection process. A computer comprising storage means for storing the musical tone data of each segment into which a song is divided and a plurality of acoustic feature quantities of each segment for each of the plurality of songs,
For each sub-element, for each sub-element, for each main element that is an element of the main musical piece, an index of similarity of the plurality of feature quantities with a sub-element that is an element other than the main element. Similarity calculation processing to calculate,
A selection process for sequentially selecting sub-elements having similar similarity index values for each main element;
A processing process that sequentially processes the musical sound data of each main element based on the musical sound data of the sub-elements selected for the main element in the selection process;
A display control process for individually displaying a chart showing the feature quantity of the main element and the feature quantity of the sub-element selected for the main element in the selection process on the display device for each of the plurality of feature quantities ; A program that executes A computer comprising storage means for storing the musical tone data of each segment into which a song is divided and the acoustic feature quantity of each segment for each of a plurality of songs,
A second setting process for variably setting the second adjustment value for each song in accordance with an instruction from the user;
A process for calculating, for each sub-element, a similarity index value that is an index of the similarity of the feature quantity with each sub-element, which is an element other than the main element, for each main element that is an element of the main music piece An similarity calculation process for calculating an similarity index value between the main element and the sub-element so as to change according to a second adjustment value set for the music to which each sub-element belongs. ,
A selection process for sequentially selecting sub-elements having similar similarity index values for each main element;
A processing process that sequentially processes the musical sound data of each main element based on the musical sound data of the sub-elements selected for the main element in the selection process;
A program for executing a display control process for displaying on the display device a chart showing the feature quantity of the main element and the feature quantity of the sub-element selected for the main element in the selection process.

Images