JP2008233493A - Music editing device and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To edit new music by replacing phoneme pieces without impairing basic constitution of original music. <P>SOLUTION: A dividing unit 111 divides music data representing the original music on which editing is based into a plurality of pieces of element piece data of shorter time length. A harmony information extracting unit 112 finds harmony information by element piece data, and chord sequence extracting unit 113 extracts a chord sequence of the original musical piece based upon respective pieces of harmony information. An element piece replacing unit 123 of an editing unit 120 selects element piece data within a range wherein the chord sequence extracted by the chord sequence extracting unit 113 can be maintained as an object to be replaced when performing editing processing for replacing element piece data constituting the music data representing the original musical piece with other element piece data. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a music editing apparatus and program for editing music by connecting phoneme pieces.

There is a technique called audio mosaicing as a technique related to song editing. In this audio mosaicing technique, various pieces of music are divided into phonemes having a short time length, and segment data indicating the waveform of each phoneme is collected to construct a segment database. Then, desired segment data is selected from the segment database, the selected segment data is connected on the time axis, and a new song is edited. For example, Non-Patent Document 1 is a document related to this type of technology.
Ari Lazier, Perry Cook, “MOSIEVIUS: FEATURE DRIVEN INTERACTIVE AUDIO MOSAICING”, [on line], Proc of the 6th Int. Conference on Digital Audio Effects (DAFx-03), London, UK, September 8-11, 2003, [Heisei Search on March 6, 19], Internet <URL: http: //soundlab.cs.princeton.edu/publications/mosievius_dafx_2003.pdf> Bee Suan Ong, Emilia Gomez, Sebastian Streich, “Automatic Extraction of Musical Structure Using Pitch Class Distribution Features”, [online], Learning the Semantics of Audio Signals (LSAS) 2006, [Search March 2, 2007], Internet <URL : http: //irgroup.cs.uni-magdeburg.de/lsas2006/proceedings/LSAS06_053_065.pdf>

  By the way, in editing a song, it is desirable to edit a song that has a different impression from the original song by changing the phoneme pieces that are parts of the song without compromising the basic structure of the existing song. There is a case. However, in the conventional technique, since the user can freely replace the phoneme, the basic music that the original song had when the user performed an inappropriate phoneme replacement operation. There was a problem that the composition was damaged and the song obtained by editing was far from the original song.

  The present invention has been made in view of the circumstances described above, and a music editing apparatus and program capable of exchanging phoneme pieces and editing a new music without impairing the basic configuration of the original music The purpose is to provide.

The present invention provides a user interface for receiving instructions related to song editing from a user and providing the user with information related to song editing, segment data indicating a waveform of a phoneme, and at least one representing an attribute of the phoneme A segment database that stores at least one type of attribute data that includes at least harmony information indicating the harmony of the phoneme, and a song data that indicates the original music that is the basis for editing, and a shorter time length. Based on the harmony information extracted by the harmony information extracting means, the harmony information extracting means for extracting the harmony information for each piece of data divided by the dividing means, the dividing means for dividing the plurality of segment data, A chord sequence extracting means for extracting the chord sequence of the original music, and the user interface In accordance with an instruction given in the above, edit processing for editing song data indicating the original song, wherein the segment data obtained from the song data indicating the original song is exchanged with other piece data Is performed by the code sequence extraction means from the segment data stored in the segment database based on the harmony information associated with the segment data stored in the segment database. There is provided a music editing apparatus and a program for causing a computer to function as the music editing apparatus, comprising editing means for selecting, as an exchange object, segment data within a range in which a code sequence can be maintained.
According to this invention, since the exchange of segment data is allowed within the range in which the chord progression maintains the same, editing that changes the music can be performed without impairing the basic characteristics of the music.

Embodiments of the present invention will be described below with reference to the drawings.
FIG. 1 is a block diagram showing the configuration of a music editing apparatus according to an embodiment of the present invention. This music editing apparatus is obtained by installing a music editing program according to an embodiment of the present invention on a computer such as a personal computer.

  In FIG. 1, a CPU 1 is a control center that controls each part of the music editing apparatus. The ROM 2 is a read-only memory that stores a control program for controlling basic operations of the music editing device, such as a loader.

  The display unit 3 is a device for displaying an operation state of the device, input data, a message for an operator, and the like, and is configured by, for example, a liquid crystal display panel and a drive circuit thereof. The operation unit 4 is a means for receiving commands and various types of information from the user, and includes various types of operators. In a preferred embodiment, the operation unit 4 includes a keyboard and a pointing device such as a mouse.

  The interface group 5 includes a network interface for performing data communication with other devices via a network, a driver for transmitting / receiving data to / from an external storage medium such as a magnetic disk or a CD-ROM, and the like. It is comprised by.

  The HDD (hard disk device) 6 is a non-volatile storage device for storing information such as various programs and databases. The RAM 7 is a volatile memory used as a work area by the CPU 1. The CPU 1 loads a program in the HDD 6 into the RAM 7 and executes it in accordance with a command given via the operation unit 4.

  The sound system 8 is means for outputting a song edited by the song editing apparatus or a song in the middle of editing as a sound, and a D / A converter for converting a digital voice signal, which is sound sample data, into an analog voice signal; The amplifier includes an amplifier that amplifies the analog audio signal and a speaker that outputs the output signal of the amplifier as sound. In the present embodiment, the sound system 8, the display unit 3 and the operation unit 4 described above serve as a user interface that receives instructions related to song editing from the user and provides information related to song editing to the user. Fulfill.

  Information stored in the HDD 6 includes a song editing program 61, a song database 62 that is a collection of song data, and a segment database 63 that is a collection of segment data. In a preferred embodiment, the song editing program 61 and song data are downloaded from a site in the Internet, for example, via an appropriate one in the interface group 5 and installed in the HDD 6. In another embodiment, the music editing program 61 and the music data are traded in a state stored in a computer-readable storage medium such as a CD-ROM or MD. In this aspect, the music editing program 61 and music data are read from the storage medium via an appropriate interface group 5 and installed in the HDD 6.

  Each piece of music data in the music database 62 may be time series sample data obtained by sampling and digitizing an audio waveform of a music performance sound, or may be compression-coded data of this time series sample data. .

  Each piece data in the piece database 63 is obtained by dividing music data into data having a shorter time length such as one beat. In the segment database 63, each segment data is accompanied by attribute data indicating each attribute. This attribute data includes at least harmony information indicating the sense of harmony of the sound. In addition, in a preferred embodiment, in addition to harmony information, there are various types such as Spectral Centroid that expresses the pitch of the sound, Loudness that expresses the sense of volume, Brightness that expresses the brightness of the sound, and Noisiness that expresses the sense of roughness. The feature amount is included in this attribute data. In addition to the information related to the matter existing in the segment data as described above, the information related to the matter existing outside the segment data, for example, the name of the player, the genre, etc. Thus, information specified by the user may be included in this attribute data.

  FIG. 2 is a block diagram showing the configuration of the song editing program 61 in this embodiment. As shown in FIG. 2, the song editing program 61 includes an analysis unit 110, an editing unit 120, and a database management unit 130, each of which is a program. The database management unit 130 is a program for accessing the song database 62 and the segment database 63 in accordance with instructions from the analysis unit 110 and the editing unit 120. The analysis unit 110 is a program that acquires song data from the song database 62 via the database management unit 130 and analyzes the song data to generate various data useful for editing new song data. The editing unit 120 is a program for editing new song data by using the analysis result obtained from the analysis unit 110 and referring to the segment database 63 via the database management unit 130.

  FIG. 3 is a block diagram showing the configuration of the analysis unit 110 and the editing unit 120 in the music editing program 61. As illustrated in FIG. 3, the analysis unit 110 includes a division unit 111, a harmony information extraction unit 112, and a code sequence extraction unit 113.

  Here, the dividing unit 111 is a program that divides music data given through the database management unit 130 into a plurality of pieces of piece data having a shorter time length and stores them in the piece data storage area in the RAM 7. In a preferred embodiment, the time length of the segment data is a length in units of beats, such as 1 beat, 2 beats, or (1/2) beats. In this aspect, a known beat detection algorithm can be used as an algorithm for dividing music data into segment data.

The harmony information extraction unit 112 obtains, for example, HPCP (Harmonic) from each piece data in the piece data storage area.
Pitch Class Profile) is a program that extracts harmony information indicating a sense of harmony of sound, etc., and stores it in a harmony information storage area in the RAM 7.

  The chord sequence extraction unit 113 finds a regular repetition pattern of the harmony information from the harmony information sequence generated from the segment data sequence by the harmony information extraction unit 112, and generates chord sequence data indicating the chord sequence of the original music. The program stored in the code sequence storage area in the RAM 7. This chord sequence includes information indicating each chord constituting the chord sequence of the original musical piece and which section in the segment data string obtained from the musical piece data corresponds to each chord.

Although not shown, the analysis unit 110 analyzes each piece of data in the piece data storage area, and indicates Spectral representing a sense of pitch.
It has a program that extracts various feature quantities from segment data, such as Centroid, Loudness that expresses volume, Brightness that expresses the brightness of sound, and Noisiness that expresses the sense of roughness. These feature amounts become a part of attribute data indicating attributes of the segment data together with the harmony information. In the preferred embodiment, as already described, the attribute data also includes information related to the song data to which the segment data belongs, such as the player name and genre. In this case, for example, when the analysis unit 110 analyzes the song data, information such as the player name is input by operating the operation unit 4 and included in the attribute data together with the feature amount obtained by the analysis. .

  When an analysis unit 110 obtains a segment data string and attribute data such as harmony information from a piece of music data, and receives a data storage instruction via the operation unit 4, the analysis unit 110 obtains the segment data and its attribute data. The correspondence is stored in the segment database 63 via the database management unit 130. The segment data and the attribute data stored in the segment database 63 are referred to when the editing unit 120 performs editing processing of new song data.

  The segment data string, harmony information, and chord sequence data generated from the song data in the analysis unit 110 are also used when editing the original song data that is the source thereof and editing new song data. The That is, in the present embodiment, the analysis unit 110 plays a role of generating segment data and attribute data used for future editing of song data from the song data and storing them in the segment database 63, and song data of a certain original song When the editing unit 120 performs editing processing for generating music data of a new music by editing the data, specifically, the data used for the editing processing (specifically, the segment data string and the code sequence data) is used for the music of the original music. It plays a role of generating from data.

  Next, the editing unit 120 will be described. As shown in FIG. 3, the editing unit 120 includes a code sequence display unit 121, a playback unit 122, and a segment replacement unit 123. The code sequence display unit 121 is a program that causes the display unit 3 to display the code sequence data extracted by the code sequence extraction unit 113 of the analysis unit 110 and stored in the code sequence storage area. The reproduction unit 122 is a program that supplies the segment data string in the segment data area to the sound system 8 and reproduces it as sound.

  The segment exchanging unit 123 performs an editing process for exchanging a part of the segment data of the song to be edited stored in the segment data storage area with other segment data in accordance with an instruction given via the operation unit 4. It is a program to be performed. In the present embodiment, the segment exchanging unit 123 performs segment data conversion within a range in which the code sequence indicated by the code sequence data in the code sequence storage area is maintained. The feature of this embodiment is in the mode of the segment data exchange process performed by the segment exchange unit 123.

  Next, the operation of this embodiment will be described by taking as an example the case of editing a new song based on a certain original song. First, the user gives an instruction to execute the music editing program 61 to the CPU 1 by operating the operation unit 4. When the CPU 1 is executing the song editing program 61, when the user instructs a process of editing a new song based on an existing song by operating the operation unit 4, the song editing program 61 An icon indicating each piece of music data stored in 62 is displayed on the display unit 3 to prompt the user to specify an original music piece as a base. When the user designates an icon indicating a desired original music by operating the operation unit 4, the music editing program 61 reads the music data of the original music from the music database 62 and delivers it to the analysis unit 110.

  Thereby, the dividing unit 111 of the analyzing unit 110 divides the music data of the original music to generate a unit data string and stores it in the unit data storage area in the RAM 7. Further, the harmony information extraction unit 112 extracts harmony information such as HPCP information from each piece data in this piece data storage area, and stores it in the harmony information storage area in the RAM 7. Further, the chord sequence extraction unit 113 finds a repetitive pattern of the harmony information from the harmony information string in the harmony information storage area, generates chord sequence data of the original music, and stores it in the chord sequence storage area in the RAM 7.

  There are various known methods for obtaining the code sequence data from the harmony information sequence. In this embodiment, for example, the code sequence data is generated from the harmony information sequence as follows.

  First, let the harmony information string be H (k) (k = 0 to n−1). Here, k is an index corresponding to the time from the beginning of the song, 0 is the beginning position of the song, and n-1 is the end position of the song. Then, any two pieces of harmony information H (i) and H (j) are extracted from the n pieces of harmony information H (k) (k = 0 to n−1), and the similarity L between the two pieces is obtained. (I, j) is calculated. This operation is performed for all i, j combinations within the range of i = 0 to n−1 and j = 0 to n−1, and an n-by-n similarity matrix as shown in FIG. L (i, j) (i = 0 to n−1, j = 0 to n−1) is created.

  Next, a triangular matrix L (i, j) (i = 0 to n−1) which is a part of the similarity matrix L (i, j) (i = 0 to n−1, j = 0 to n−1). , J ≧ i), a continuous region where the similarity L (i, j) is equal to or greater than a threshold value is obtained. In FIG. 4B, a region indicated by a thick black line exemplifies a continuous region having a high similarity obtained by this operation (hereinafter referred to as a high similarity continuous region for convenience). In the present embodiment, when a plurality of such high similarity continuous regions are obtained, the harmony information string H (k) (k = Find patterns of harmony information that appear repeatedly in 0-n-1).

  For example, in the example shown in FIG. 4B, the similarity matrix L (i, j) (i = 0 to n−1, j = 0 to n−1) is a collection of similarities between the same harmony information. In addition to the high similarity continuous region L0, high similarity continuous regions L1, L2, and L3 are included. Here, the high similarity continuous region L1 is a harmony information sequence H (i) (i) in which the harmony information sequence H (j) (j = k2 to k4-1) in the middle of the song starts from the beginning of the song. = 0 to k2-1). The high similarity continuous area L2 is a section where the harmony information string H (j) (j = k4 to k5-1) of the section immediately after the section corresponding to the high similarity continuous area L1 in the music starts from the beginning of the music. This is similar to the harmony information string H (i) (i = 0 to k1-1). In addition, the high similarity continuous area L3 has a harmony information string H (j) (j = k5 to n-1) in the section immediately after the section corresponding to the high similarity continuous area L2 in the music. It is similar to the harmony information string H (i) (i = k2 to k3-1) in the section.

  Focusing on the overlapping relationship of the occupation ranges on the i-axis of these high similarity continuous regions L1, L2, and L3, the following can be understood. First, the harmony information string H (j) (j = k2 to k4-1) of the section corresponding to the high similarity continuous region L1 is the harmony information string H (i) (i = 0 to 0) of the section starting from the beginning of the song. k2-1), but the harmony information string H (i) (i = 0 to k1-1) of a part of the section is a harmony information string H () of the section corresponding to the high similarity continuous region L2. j) (j = k4 to k5-1) is also similar. That is, the section from which the harmony information sequence H (i) (i = 0 to k2-1) starting from the beginning of the song is divided into a first half section A and a second half section B, and corresponds to the high similarity continuous region L1. It is estimated that the same code as the sections A and B is repeated in the section, and the same code as the section A is repeated in the high similarity continuous region L2.

  Next, the harmony information string H (j) (j = k5 to n-1) of the section corresponding to the high similarity continuous region L3 is the harmony information string H (i) (i = k2 to k3-1) of the preceding section. ), But the occupied range on the i-axis of the high similarity continuous region L3 is the occupied range on the i-axis of the other high similarity continuous regions L1 and L2 excluding the high similarity continuous region L0. And do not overlap. In this case, in the music, the section C that is the source of the harmony information string H (i) (i = k2 to k3-1) is a single section that is not further subdivided, and the high similarity continuous region L3 It is estimated that the same code as in section C is repeated.

  The code sequence extraction unit 113 performs the above-described processing, the sections corresponding to the various codes in the harmony information string H (k) (k = 0 to n−1) (in the example shown in FIG. 4B, the section A, B, A, B, A, C), and the chords played in each section are obtained based on the harmony information belonging to each section. As a result, code sequence data which is time-series data of codes is obtained. A method for generating a code sequence based on such harmony information is disclosed in Non-Patent Document 2, for example.

  When the segment data string, the harmony information string, and the code sequence data are generated from the song data by the processing of the analysis unit 110, the code sequence display unit 121 of the editing unit 120 displays the code sequence indicated by the code sequence data on the display unit 3. The playback unit 122 supplies the segment data string to the sound system 8 and plays it back as sound. Thus, in the present embodiment, the user can grasp the contents of the original music to be edited (more specifically, the contents of the music indicated by the segment data string) through both visual and auditory senses.

  In order to make it easier for the user to grasp the original music piece, the display of the chord sequence and the reproduction as the sound of the segment data string may be synchronized. That is, when sending each piece data to the sound system, the playback unit 122 determines which code in the code sequence the piece data belongs to, and notifies the code sequence display unit 121 of the code. When displaying the code sequence on the display unit 3, the code sequence display unit 121 displays the notified code in a display mode different from other codes. According to this aspect, the user can grasp which code in the chord sequence the sound currently being played is.

  Next, the user designates the segment data desired to be replaced with other segment data by operating the operation unit 4. Specifically, for example, when the piece data desired to be exchanged belongs to the code G7, the user displays the code G7 in the code sequence displayed on the display unit 3 as shown in FIG. The position is indicated by operating the mouse or the like in the operation unit 4. Then, the segment exchange unit 123 causes the display unit 3 to display a graphic indicating the segment data string in the section of the code G7. As illustrated in FIG. 5B, this figure may be one in which marks indicating individual piece data are arranged in a line. Alternatively, for example, referring to feature quantities such as Spectral Centroid, Loudness, Brightness, Noisiness, etc. corresponding to each piece of data constituting the piece data string, as illustrated in FIG. A graph showing a change in one or more feature quantities may be generated and displayed on the display unit 3. The user determines where in the graphic displayed in this way there is a portion corresponding to the piece data desired to be exchanged, and indicates the position by operating the operation unit 4. Here, the user may designate one piece of piece data, or may designate a plurality of piece data.

  Based on the indicated position, the element exchanging unit 123 recognizes element data to be replaced with other element data in the element data string in the element data storage area. Next, the segment exchanging unit 123 refers to the code sequence data in the code sequence storage area to determine in which code section the segment data is. Then, when it is determined that the piece data to be replaced with other piece data is, for example, the piece data in the section of the code G7, the piece exchange unit 123 corresponds to each piece data in the piece database 63. Each piece of harmony information is referred to, and from the piece data in the piece database 63, the piece data that can be used for the code G7 is selected as an exchange target candidate. Then, an icon indicating each piece data that is a candidate for replacement is displayed on the display unit 3. At this time, even if the color or shape of the icon is changed according to a specific feature amount of each piece data, the user can easily understand what kind of piece data is a candidate for exchange. Good.

  In this state, the user can input additional conditions for narrowing down candidates for replacement by operating the operation unit 4. This additional condition may be a player name, a genre, or the like. Alternatively, it may be a range or degree that a specific feature amount should take. For example, an additional condition for specifying a candidate for replacement such as how bright it is and how rough it is is specified. When the additional condition is given in this way, the element exchanging unit 123 selects a condition that satisfies the additional condition from the current replacement target candidates and causes the display unit 3 to display the new replacement target candidate.

  When the user designates one piece of piece data in the candidate to be exchanged by operating the operation unit 4, the piece exchange unit 123 reads out the designated piece data from the piece database 63, and the piece data storage area Replace with the segment data that was to be exchanged with other segment data.

  Unless otherwise specified, the playback unit 122 continues to repeatedly send the segment data string in the segment data storage area to the playback unit 122 while the operation for exchanging the segment data is being performed. Therefore, the user can confirm the effect of exchanging segment data with his / her ear. Thereafter, the user selects one piece of data from the exchange target candidates presented by the piece exchange unit 123 and the instruction of the piece data to be exchanged with other piece data until the desired song is obtained. It is sufficient to repeat the operation. In order to allow the user to quickly confirm the effect of exchanging the segment data, after exchanging the segment data, the sound system starts from the segment data slightly before the exchanged segment data. The playback unit 122 may be configured to start sending to 8.

  When a desired song is obtained by repeating the exchange of segment data, the user inputs a song data storage instruction by operating the operation unit 4. Thus, the song editing program 61 stores the segment data string in the segment data storage area at that time in the song database 62 as new song data.

  As described above, according to the present embodiment, when the user tries to exchange the piece data constituting the piece of music data with other piece data, the piece exchange unit 123 of the piece editing program 61 uses the original piece of music. Only the segment data within the range in which the code sequence can be maintained is presented to the user as a candidate for exchange. Therefore, according to this embodiment, the user can edit a new song by exchanging phoneme pieces without losing the basic configuration of the original song.

  Although one embodiment of the present invention has been described above, other embodiments are possible for the present invention. For example:

(1) The music editing program 61 may replace part or all of the program with an electronic circuit.

(2) In the above embodiment, the unit data is exchanged in units of unit data. However, the unit data strings may be exchanged in units larger than that, for example, code units. More specifically, in this aspect, for each code indicated by the code sequence data, the analysis unit 110 stores the segment data string in the segment of the code in the segment database 63. At this time, information indicating the code is stored as harmony information in the segment database 63 together with the segment data string. Then, the element exchanging unit 123 causes the user to specify a code desired to be exchanged with another element data string, and presents the element data string corresponding to the same code as the code to the user as an exchange target candidate, Is selected as an exchange target. In this aspect, fine editing processing in units of segment data cannot be performed, but there is an advantage that the operation is simple because the unit data string is replaced in units of code.

It is a block diagram which shows the structure of the music editing apparatus which is one Embodiment of this invention. It is a block diagram which shows the structure of the music edit program 61 in the embodiment. It is a block diagram which shows the structure of the analysis part 110 and the edit part 120 in the same music edit program 61. FIG. It is a figure which illustrates the method of extracting a code sequence from a harmony information sequence in the embodiment. It is a figure which illustrates the GUI screen which prompts a user to select the segment data exchanged with other segment data in the embodiment.

Explanation of symbols

1 ... CPU, 2 ... ROM, 3 ... display unit, 4 ... operation unit, 5 ... interface group, 6 ... HDD, 7 ... RAM, 8 ... sound system, 61 ... song editing program 62 …… Song database, 63 …… Unit database, 110 …… Analysis unit, 120 …… Editing unit, 130 …… Database management unit, 111 …… Division unit, 112 …… Harmony information extraction unit, 113 …… Code sequence extraction unit 121... Code sequence display unit 123.

Claims (3)

A user interface that receives instructions relating to song editing from the user and provides the user with information related to song editing;
Element data indicating the waveform of a phoneme element and at least one type of attribute data representing the attribute of the phoneme element, including at least harmony information indicating the harmony of the phoneme element A database and
A dividing means for dividing the song data indicating the original song as a base for editing into a plurality of segment data having a shorter time length;
Harmony information extracting means for extracting harmony information for each segment data divided by the dividing means;
Chord sequence extraction means for extracting the chord sequence of the original music based on the harmony information extracted by the harmony information extraction means;
In response to an instruction given via the user interface, the unit performs editing processing of music data indicating the original music piece, and the piece data obtained from the music data indicating the original music is changed to other piece data. The code sequence extraction from each piece data stored in the unit database based on harmony information associated with each piece data stored in the unit database. Editing means for selecting, as an exchange object, segment data within a range in which the chord sequence extracted by the means can be maintained.   The editing means receives, via the user interface, an additional condition that specifies an attribute that the exchange target should have, and based on attribute data associated with each piece data stored in the piece database, 2. The music piece editing apparatus according to claim 1, wherein the piece data having an attribute designated by the additional condition is selected from the piece data within a range in which the chord sequence can be maintained. Computer
A dividing means for dividing the song data indicating the original song as a base for editing into a plurality of segment data having a shorter time length;
Harmony information extracting means for extracting harmony information for each segment data divided by the dividing means;
Chord sequence extraction means for extracting the chord sequence of the original music based on the harmony information extracted by the harmony information extraction means;
In response to an instruction given through a user interface, the unit performs editing processing of music data indicating the original music, and converts the piece data obtained from the music data indicating the original music to other piece data When performing the editing processing to be exchanged, the code sequence extraction means from among the segment data stored in the segment database based on the harmony information associated with each segment data stored in the segment database A music editing program which functions as an editing means for selecting, as an exchange target, segment data within a range in which an extracted chord sequence can be maintained.

Images