JP4380720B2 - Song editing apparatus and program - Google Patents

Description

  The present invention relates to a music editing apparatus and program for editing music by connecting phoneme pieces.

As a technique related to song editing, there is a technique called audio mosaicing. In this audio mosaicing technology, various pieces of music are divided into phonemes having a short time length, and segment data indicating the waveform of each phoneme is collected to constitute a segment database. Then, desired segment data is selected from the segment database, the selected segment data is connected on the time axis, and a new song is edited. In addition, there exists a nonpatent literature 1 as literature regarding this kind of technique, for example.
Ari Lazier, Perry Cook, “MOSIEVIUS: FEATURE DRIVEN INTERACTIVE AUDIO MOSAICING”, [on line], Proc of the 6th Int. Conference on Digital Audio Effects (DAFx-03), London, UK, September 8-11, 2003, [Heisei Search on March 6, 19], Internet <URL: http: //soundlab.cs.princeton.edu/publications/mosievius_dafx_2003.pdf>

  By the way, in editing a song, it is desirable to edit a song that has a different impression from the original song by changing the phoneme pieces that are parts of the song without compromising the basic structure of the existing song. There is a case. However, in the conventional technique, since the user can freely replace the phoneme, the basic music that the original song had when the user performed an inappropriate phoneme replacement operation. There was a problem that the composition was damaged and the song obtained by editing was far from the original song.

  The present invention has been made in view of the circumstances described above, and a music editing apparatus and program capable of exchanging phoneme pieces and editing a new music without impairing the basic configuration of the original music The purpose is to provide.

The present invention is a sequence of a plurality of segment data each representing an audio waveform of a speech segment, an original song storage means for storing a segment data sequence of an original song serving as a base for editing, and a segment data sequence of an edited song Using the segment data in the segment data string of the original song stored in the original song storage unit, the segment data string of the edited song is stored in the edited song storage unit. The editing means for creating, when creating one piece data in the piece data string of the edited music, from the position corresponding to the piece data to be created in the piece data series of the original music Provided is a music editing apparatus comprising an editing means for controlling selection of segment data so that one segment data within a predetermined range is used, and a program for causing a computer to function as the music editing apparatus To do.
According to this invention, when creating each piece data of the piece data string of the edited song, one piece within a predetermined range from the position of the piece data piece to be created in the piece data string of the original music piece. The piece data is used as the piece data to be created. Therefore, an edited music that maintains the basic composition of the original music can be obtained.

Embodiments of the present invention will be described below with reference to the drawings.
FIG. 1 is a block diagram showing the configuration of a music editing apparatus according to an embodiment of the present invention. This music editing apparatus is obtained by installing a music editing program according to an embodiment of the present invention on a computer such as a personal computer.

  In FIG. 1, a CPU 1 is a control center that controls each part of the music editing apparatus. The ROM 2 is a read-only memory that stores a control program for controlling basic operations of the music editing device, such as a loader.

  The display unit 3 is a device for displaying an operation state of the device, input data, a message for an operator, and the like, and is configured by, for example, a liquid crystal display panel and a drive circuit thereof. The operation unit 4 is a means for receiving commands and various types of information from the user, and includes various types of operators. In a preferred embodiment, the operation unit 4 includes a keyboard and a pointing device such as a mouse.

  The interface group 5 includes a network interface for performing data communication with other devices via a network, a driver for transmitting / receiving data to / from an external storage medium such as a magnetic disk or a CD-ROM, and the like. It is comprised by.

  The HDD (Hard Disk Device) 6 is a non-volatile storage device for storing information such as various programs and databases. The RAM 7 is a volatile memory used as a work area by the CPU 1. The CPU 1 loads a program in the HDD 6 into the RAM 7 and executes it in accordance with a command given via the operation unit 4.

  The sound system 8 is means for outputting a song edited by the song editing apparatus or a song in the middle of editing as a sound, and a D / A converter for converting a digital voice signal, which is sound sample data, into an analog voice signal; The amplifier includes an amplifier that amplifies the analog audio signal and a speaker that outputs the output signal of the amplifier as sound. In the present embodiment, the sound system 8, the display unit 3 and the operation unit 4 described above serve as a user interface that receives instructions related to song editing from the user and provides information related to song editing to the user. Fulfill.

  Information stored in the HDD 6 includes a song editing program 61, a song database 62 that is a collection of song data, and a segment database 63 that is a collection of segment data. In a preferred embodiment, the song editing program 61 and song data are downloaded from a site in the Internet, for example, via an appropriate one in the interface group 5 and installed in the HDD 6. In another embodiment, the music editing program 61 and the music data are traded in a state stored in a computer-readable storage medium such as a CD-ROM or MD. In this aspect, the music editing program 61 and music data are read from the storage medium via an appropriate interface group 5 and installed in the HDD 6.

  Each piece of music data in the music database 62 may be time series sample data obtained by sampling and digitizing an audio waveform of a music performance sound, or may be compression-coded data of this time series sample data. . Each piece data in the piece database 63 is obtained by dividing music data into data having a shorter time length such as one beat.

  In the present embodiment, the music editing program 61 reads various pieces of piece data from the piece database 63 in accordance with instructions given via the operation unit 4 and connects them on the time axis to edit new song data. It has. In addition to this function, the song editing program 61 in the present embodiment is a function for editing new song data by changing the segment data that is a part to other segment data based on the existing song data. It has.

  FIG. 2 is a block diagram showing the configuration of the portion related to the latter function of the above two functions in the music editing program 61. As shown in FIG. 2, the music editing program 61 includes a dividing unit 110, an editing unit 120, and a reproducing unit 130, each of which is a program.

  Here, the dividing unit 110 divides the song data read from the song database 62 based on an instruction given via the operation unit 4 into a plurality of segment data having a shorter time length, and the original song in the RAM 7 is divided. This is a program that causes the CPU 1 to execute processing stored in the storage area 71. In a preferred embodiment, the time length of the segment data is a length in units of beats, such as 1 beat, 2 beats, (1/2) beat, (1/4) beat, or the like. In this aspect, a known beat detection algorithm can be used as an algorithm for dividing music data into segment data. Alternatively, the waveform indicated by the music data may be analyzed to detect an attack portion, and each piece of data indicating the waveform of each section of 0.5 seconds from each detected attack portion may be used as segment data.

  The editing unit 120 creates the segment data string of the edited song in the edited song storage area 72 in the RAM 7 using the segment data in the segment data string of the original song stored in the original song storage area 71. It is a program that causes the CPU 1 to execute processing. More specifically, when the editing unit 120 creates one piece of data in the piece data sequence of the edited music and stores it in the edited music storage area 72, the editing unit 120 creates the data in the piece data series of the original music. A sliding window W having a predetermined length including the position corresponding to the segment data is obtained. Specifically, this sliding window W is time t−, where t is the relative appearance time (hereinafter referred to as the creation target position) of the segment data to be created with reference to the beginning of the edited song. This window occupies a range from Δt to time t + Δt, and the range is determined by the position to be created in the edited music. In the present embodiment, the editing unit 120 determines the segment data so that one segment data in the sliding window W is used when the creation target position in the edited music is determined and the sliding window W is determined thereby. Control selection. The feature of this embodiment is the selection control of the piece data of the edited music using this sliding window W. Although there are various modes of selection control of the piece data of the edited music performed by the editing unit 120, details thereof will be clarified in the operation description of the present embodiment in order to avoid duplication of description.

  In accordance with an instruction given via the operation unit 4, the reproducing unit 130 displays the segment data string of the original song stored in the original song storage area 71 or the segment data string of the edited song stored in the edited song storage area 72. This is a program for causing the CPU 1 to execute processing to be sent to the sound system 8 and reproduced as sound.

  Next, the operation of this embodiment will be described by taking as an example the case of editing a new song based on a certain original song. First, the user gives an instruction to execute the music editing program 61 to the CPU 1 by operating the operation unit 4. When the CPU 1 is executing the song editing program 61, when the user instructs a process of editing a new song based on an existing song by operating the operation unit 4, the song editing program 61 An icon indicating each piece of music data stored in 62 is displayed on the display unit 3 to prompt the user to specify an original music piece as a base. When the user designates an icon indicating a desired original music by operating the operation unit 4, the music editing program 61 reads the music data of the original music from the music database 62 and delivers it to the dividing unit 110.

  The dividing unit 110 divides the song data delivered in this way to generate a segment data string, stores the segment data string in the original song storage area 71, and starts up the editing unit 120.

  In the present embodiment, there are three operation modes of the editing unit 120: a manual mode, a fully automatic mode, and a semi-automatic mode. After being started by the dividing unit 110, the editing unit 120 first causes the display unit 3 to display a GUI (Graphical User Interface) that allows the user to select one of these three modes. The editing unit 120 receives an instruction regarding mode selection via the operation unit 4 and operates in the instructed mode. Hereinafter, the operation in each mode will be described.

<Operation in manual mode>
FIG. 3 is a diagram illustrating the operation of the editing unit 120 in the manual mode. When starting the operation in the manual mode, the editing unit 120 copies the segment data string of the original music in the original music storage area 71 into the edited music storage area 72. Further, the editing unit 120 causes the display unit 3 to display a graphic showing the segment data string of the original music piece and a graphic showing the segment data string of the edited music piece, in order to facilitate the user's operation. These figures are preferably figures that are useful for intuitively grasping the segment data string that is time-series information, such as stripes. In a state where these graphics are displayed on the display unit 3, the user operates, for example, a pointing device of the operation unit 4 to indicate one point in the graphic indicating the segment data string of the original music, and It is possible to instruct the creation target position of the piece data.

  When the editing unit 120 receives an instruction related to the creation target position from the operation unit 4 (step S101), the editing unit 120 obtains a sliding window W having a predetermined width corresponding to the creation target position, and performs this sliding in the segment data string of the original music piece. A piece of segment data in the window W, that is, a candidate that can be segment data to be created is presented to the user (step S102).

  In a preferred embodiment, the editing unit 120 displays, for example, an area where the sliding window W is located in a color or pattern different from the other when displaying a stripe indicating the segment data string of the original music on the display unit 3. Part 3 is displayed. In this aspect, the user designates a desired point in the region where the sliding window W is located in the stripe displayed on the display unit 3 by operating the operation unit 4 in order to select the segment data. The segment data to be arranged at the creation target position in the edited music can be designated.

  Until the segment data is instructed by the user, the editing unit 120 includes the sliding window W and has a slightly wider range as the playback range, and the segment data sequence of the original music Each piece of data within the reproduction range is repeatedly sent to the sound system 8 to emit sound (step S103). At this time, in the stripe display indicating the segment data string of the original music piece on the display unit 3, the portion indicating the segment data currently being emitted may be displayed in a different color from the other portions. .

  When the segment data in the original music sliding window W is instructed by the operation of the operation unit 4, the editing unit 120 is instructed by the user from the segment data string in the original song storage area 71. Segment data is selected and written into the edited song storage area 72 as segment data at the position to be created (step S104). When the user designates the segment data outside the sliding window W in the original music, the editing unit 120 ignores this instruction.

  When the editing unit 120 writes the segment data in the sliding window W selected in accordance with an instruction from the user to the edited song storage area 72, the editing range of the segment data string of the edited song after writing is the same as the playback range. Each piece of data is repeatedly sent to the sound system 8 to emit sound (step S105). The user can determine whether or not the selection of the segment data is appropriate by listening to the music emitted from the sound system 8. If it is determined that the selection of the segment data is inappropriate, the user may specify the same creation target position as the previous time (step S101) and redo the selection of the segment data (step S104). When the segment data string in the edited song storage area 72 has a desired content, the user gives an instruction to store data by operating the operation unit 4. Thus, the editing unit 120 stores the segment data string in the edited song storage area 72 in the song database 62 as song data of a new song.

  In another preferred embodiment, the editing unit 120 receives an instruction related to the creation target position from the operation unit 4 (step S101), and when the sliding window W corresponding to the creation target position is obtained, the original music is within the sliding window W. Two types of feature values are obtained for each piece of piece data. Then, as illustrated in FIG. 4, the display unit 3 displays a two-dimensional map that is mapped based on the two types of feature amounts of each piece of data in the sliding window W. Examples of the feature values used to create a two-dimensional map include Spectral Centroid, which expresses the pitch of the sound, Loudness, which expresses the volume, Brightness, which expresses the audible brightness of the sound, and Noisiness, which expresses the gritty feel of the sound. It is done. The two types of feature amounts used for creating the two-dimensional map may be fixed types or may be selected by the user by operating the operation unit 4. Of the segment data in the sliding window W in the two-dimensional map, the segment data at the same position as the creation target position is different from the plot corresponding to the other segment data as illustrated in FIG. Plots are preferred. If there is a request to replace the current segment data (that is, the segment data at the position corresponding to the creation target position in the original music) with a brighter sound, for example, the segment that satisfies such a request This is because selection of data becomes easy.

  In a state where the two-dimensional map of each piece data in the sliding window W is displayed on the display unit 3, the user can designate one point in the two-dimensional map by operating the operation unit 4. When this instruction is given, the editing unit 120 selects the piece of data displayed on the two-dimensional map that is closest to the designated point, and writes it in the edited music storage area 72 as the piece of data at the creation target position. (Step S104).

Also in this mode, in order to allow the user to grasp the selection result of the segment data, when the segment data is selected and stored in the edited song storage area 72 according to the instruction from the user, the editing is performed in the same manner as the mode described above. It is preferable to send the musical piece data string within the reproduction range to the sound system 8 for sound emission.
The above is the operation of the editing unit 120 in the manual mode.

<Operation in fully automatic mode>
FIG. 5 is a diagram illustrating the operation of the editing unit 120 in the fully automatic mode. The editing unit 120 has a plurality of types of selection patterns in order to perform processing in the fully automatic mode. Here, the selection pattern is a time-series pattern of the selection pattern value Δt that specifies at which position in the sliding window W the segment data is to be selected. When the width of the sliding window W is W, the selection pattern value Δt can take a value within a range from −W / 2 to + W / 2. The plurality of types of selection patterns possessed by the editing unit 120 include a periodic pattern and an aperiodic pattern (for example, a pseudo random number sequence). In addition, periodic selection patterns corresponding to various periods and various patterns having different selection pattern values for one period are prepared.

  When starting the operation in the fully automatic mode, the editing unit 120 displays a menu of selection patterns that can be used by the user on the display unit 3 and selects the selection pattern used to create the segment data string of the edited song. (Step S201).

  When the user finishes selecting the selection pattern, the editing unit 120 creates an edited song having the same length as the original song based on the segment data string stored in the original song storage area 71 and the selection pattern selected by the user. A segment data string is created in the edited song storage area 72 (step S202). Here, when there are n types of selection patterns selected by the user, the editing unit 120 uses these selection patterns to generate n types of edited music piece data strings from the original music piece data strings. .

  The content of the process of generating a segment data string of one type of edited music using one type of selection pattern is as follows. In other words, the editing unit 120 sequentially updates the creation target position t from the start point to the end point of the edited song one by one. Is selected and written in the edited music storage area 72.

  When the creation target position t is located within a section within W / 2 from the start point of the edited song, the editing unit 120 does not refer to the selection pattern and creates the creation target position from the segment data string of the original song. The segment data at the same position as t is selected and written into the edited song storage area 72 as the segment data at the editing song creation target position t. The same applies to the case where the creation target position t is located in a section within W / 2 before the end point of the edited song.

  In the section between the time point W / 2 after the start point of the edited music and the time point W / 2 before the end point of the edited music, the editing unit 120 updates the creation target position t every time. Then, the selection pattern value Δt of the selection pattern is sequentially read out one by one, and according to the selection pattern value Δt, the one used for the segment data at the creation target position in the edited music is selected from the segment data string of the original music. More specifically, when the read selection pattern value Δt is a positive value, the editing unit 120 selects the piece data at a position after | Δt | from the same position as the current creation target position t in the original music piece. This is selected and used as segment data at the creation target position t in the edited music. In addition, when the read selection pattern value Δt is a negative value, the editing unit 120 selects the piece data that is at a position | Δt | before the same position as the current creation target position t in the original music, This is used for segment data at the creation target position t in the edited music. When the selected pattern value Δt that has been read is 0, the editing unit 120 selects the segment data at the same position as the current creation target position t in the original music piece, and the segment at the creation target position t in the edited music piece. Used for data.

When the segment data string of n types of edited music is created in the edited music storage area 72 using the n types of selection patterns selected by the user as described above, the editing unit 120 displays the n audition buttons. And one full playback button are displayed on the display unit 3. When the user instructs the k-th audition button by operating the operation unit 4, the editing unit 120 sends the segment data string of the k-th edited song stored in the edited song storage area 72 to the sound system 8, and edits the edited song. Sound. When the user designates the play all button by operating the operation unit 4, the editing unit 120 sequentially sends the segment data strings of n types of edited music stored in the edited music storage area 72 to the sound system 8. Play the edited song. In the case where there is a favorite edited song to be emitted in this way, the user notifies the editing unit 120 of the edited song number by editing the operation unit 4. The editing unit 120 reads out the segment data string of the notified edited song from the edited song storage area 72 and stores it in the song database 62 as song data of a new song.
The above is the operation of the editing unit 120 in the fully automatic mode. This fully automatic mode has an advantage that the trial and error music editing can be easily performed.

<Operation in semi-automatic mode>
FIG. 6 is a diagram illustrating the operation of the editing unit 120 in the semi-automatic mode. This semi-automatic mode is a mode positioned between the manual mode and the fully automatic mode described above. In this semi-automatic mode, the same selection pattern as that in the full-automatic mode is used. In this semi-automatic mode, the editing unit 120 causes the display unit 3 to display a GUI as illustrated on the left side of FIG. This GUI includes a plurality of selection pattern designation buttons 31 indicating selection patterns that can be used by the user, and a stripe 32 indicating a segment data string of the original music.

  When the user wants to change a part of the original music and create new music data, the user specifies the section in the stripe 32 using the pointing device of the operation unit 4. In addition, the pointing device designates one corresponding to the selection pattern used to obtain the segment data of the section among the plurality of selection pattern designation buttons 31.

  When the above operation is performed, the editing unit 120 uses the selection pattern specified by the user and the segment data string in the original music storage area 71 for the segment specified by the user to edit the segment data of the edited song. Is created and stored in the edited song storage area 72. This operation is the same as the operation in the fully automatic mode. For the sections other than the section specified by the user, the segment data in the original song storage area 71 is stored in the edited song storage area 72 as it is as the segment data of the edited song.

  By repeating the above operation, the user can accumulate a plurality of types of edited song segment data strings in the edited song storage area 72. At that time, the editing unit 120 causes the display unit 3 to display a list of edited songs stored in the edited song storage area 72. As illustrated on the right side of FIG. 6, the edited song list for one song has a stripe indicating the number of the edited song and which section of the original song has been changed in the edited song (indicated by diagonal lines in FIG. 6). And a list of information indicating the selection pattern used to change the section is displayed on the display unit 3.

  When the user designates the number of one edited song by operating the operation unit 4 in a state where the edited song list for at least one song is displayed on the display unit 3, the editing unit 120 stores the edited song list in the edited song storage area 72. Of the segment data string of the edited music piece, the data corresponding to the number designated by the user is sent to the sound system 8, and the edited music piece is emitted. The user repeats such an operation. As a result, if there is a favorite edited song to be emitted, the user edits the number of the edited song by operating the operation unit 4. Notification to the unit 120. The editing unit 120 reads out the segment data string of the notified edited song from the edited song storage area 72 and stores it in the song database 62 as song data of a new song.

  The above is the operation of the editing unit 120 in the semi-automatic mode. This semi-automatic mode is also advantageous in that it allows easy editing of trial and error songs. In addition, this semi-automatic mode has an advantage that an edited music in which only a desired section of the original music is changed can be obtained.

  As described above, according to the present embodiment, when the user tries to create a segment data string of an edited song by exchanging the segment data of the original song with other segment data, the segment of the original song One piece of data within the range of the sliding window W including the position corresponding to the piece data to be created in the data string is used as the piece data to be created. Therefore, the segment data string of the edited music maintaining the basic configuration of the original music can be obtained.

  Although one embodiment of the present invention has been described above, other embodiments are possible for the present invention. For example:

(1) The music editing program 61 may replace part or all of the program with an electronic circuit.

(2) The selection pattern used in the fully automatic mode and the semi-automatic mode does not need to be a predefined pattern. For example, white noise generated in the power supply unit of the music editing apparatus or the surrounding environmental noise may be detected, and this noise may be normalized to generate a selection pattern within a range of −W / 2 to + W / 2.

(3) You may comprise so that the width | variety of the sliding window W can be designated by operation of the operation part 4. FIG. In the above embodiment, the range of the sliding window W is determined so that the position corresponding to the creation target position is located at the center of the sliding window W. However, the relative positional relationship between the creation target position and the sliding window W is determined. You may comprise so that adjustment is possible.

It is a block diagram which shows the structure of the music editing apparatus which is one Embodiment of this invention. It is a block diagram which shows the structure of the music edit program 61 in the embodiment. It is a figure which shows the operation | movement in the manual mode of the edit part 120 in the same music edit program 61. FIG. It is a figure which shows the example of a display of the segment data of the selection candidate in the manual mode. FIG. 6 is a diagram showing an operation of the editing unit 120 in a fully automatic mode. 6 is a diagram showing an operation in a semi-automatic mode of the editing unit 120. FIG.

Explanation of symbols

1 ... CPU, 2 ... ROM, 3 ... display unit, 4 ... operation unit, 5 ... interface group, 6 ... HDD, 7 ... RAM, 8 ... sound system, 61 ... song editing program 62 …… Song database, 63 …… Unit database, 110 …… Division unit, 120 …… Editing unit, 130 …… Playback unit.

Claims (4)

A plurality of segment data strings each representing an audio waveform of a phoneme segment; original music storage means for storing a segment data string of the original music serving as a base for editing;
Edited song storage means for storing a segment data string of the edited song;
An editing unit that creates a segment data string of an edited song in the edited song storage unit using the segment data in the segment data sequence of the original song stored in the original song storage unit, When creating one piece of data in a piece data sequence of an edited song, one piece of data within a predetermined range from the position corresponding to the piece data to be created in the piece data row of the original music Editing means for controlling the selection of segment data so that is used. A user interface for receiving instructions related to song editing from the user and providing the user with information related to song editing;
The editing means receives an instruction regarding a segment data to be created in the segment data string of the edited music from the user via the user interface, and the creation instructed by the user in the segment data string of the original music Obtain segment data within a predetermined range from the position of the target segment data, notify the user of the segment data within the predetermined range via the user interface, and from the user via the user interface 2. The music editing apparatus according to claim 1, wherein, in accordance with a given instruction, one piece data is selected from the piece data within the predetermined range and used for the piece data to be created.   The editing means sequentially selects the segment data to be created in the segment data string of the edited song, and each time one creation target is selected, the segment data of the original music is selected according to a predetermined selection pattern. 2. The music editing apparatus according to claim 1, wherein one piece of data within a predetermined range is selected from a position corresponding to the piece data to be created in the column and used as the piece data to be created. . Computer
A plurality of segment data strings each representing an audio waveform of a phoneme segment; original music storage means for storing a segment data string of the original music serving as a base for editing;
Edited song storage means for storing a segment data string of the edited song;
An editing unit that creates a segment data string of an edited song in the edited song storage unit using the segment data in the segment data sequence of the original song stored in the original song storage unit, When creating one piece of data in a piece data sequence of an edited song, one piece of data within a predetermined range from the position corresponding to the piece data to be created in the piece data row of the original music A song editing program that functions as an editing means for controlling the selection of segment data so that is used.

Images