JP4633022B2 - Music editing device and music editing program. - Google Patents

Description

  The present invention relates to a music content processing technique, and more particularly, to a music editing apparatus, a music editing method, and a music editing program for editing first and second music contents.

  There is known software having a function of editing a new piece of music content by combining or deleting part of two pieces of music content. When editing music content with this software, an audio point of the music content is visually checked to select an editing point, and editing is performed manually using the editing point (see, for example, Non-Patent Document 1).

Moreover, the edit point for combining a part of music content is usually set to the silent part of the music content. When editing points other than silence are set, when two music contents are combined, the amplitude of the audio waveform does not match at the editing point, and annoying noise (frequency not included in the original audio signal) is mixed at the combination point. It is to do.
“GoldWave”, [online], GoldWave Inc., [Search August 7, 2006], <URL: http://www.goldwave.com/>

  However, in the background art described above, the edit point is manually set to the silent part of the music content, so that in addition to the work burden of setting the edit point, the edit points that can be set for one music content are limited. There's a problem. For this reason, the number of editing points is limited, and it has been difficult to improve the degree of freedom in editing music content.

  Furthermore, in order to increase the number of edit points that can be selected in one piece of music content, it is necessary to set an edit point other than the silent part of the music content. There is a problem that the volume becomes unstable.

  In view of the above problems, the present invention provides a music editing device, a music editing method, and a music editing program capable of automatically obtaining a large number of editing points and suppressing noise generated as a result of editing. With the goal.

  In order to achieve the above object, a first feature of the present invention is a music editing device that edits first and second music contents, and calculates a feature amount from each of the first and second music contents. And detecting a positional deviation between a cut point detection unit that detects when the feature amount exceeds a threshold as a cut point, and a cut point detected from the first music content and a cut point detected from the second music content. And a cut point position deviation correction unit that corrects the position deviation, and an edit point storage unit that stores the cut point with the corrected position deviation as an edit point used for editing the first and second music contents. To do. Here, “music content” means content including music (music).

  According to this feature, a cut point detection unit that detects when a feature value exceeds a threshold value, that is, a time point when a change in the feature value of a music is large, and a cut point position deviation correction that corrects the position deviation of the cut point. And an edit point storage unit that stores cut points corrected for misalignment as edit points used for editing the first and second music contents, it is possible to automatically obtain a large number of edit points. Therefore, it is possible to provide a music editing apparatus capable of suppressing noise generated as a result of editing.

  The gist of the second feature is that, in the music editing device according to the first feature, the cut point detection unit calculates the amount of change per unit time of the audio power or frequency as the feature amount. Here, “speech power” means, for example, a value obtained by calculating an amplitude spectrum from a sound waveform by Fourier transform or the like and obtaining a square sum of the amplitude spectrum.

  According to this feature, since the amount of change per unit time of audio power or frequency is calculated from the music content as the feature amount, a preferable feature amount can be obtained for the music content, and an accurate cut point can be detected. it can.

  The third feature is that, in the music editing device according to the second feature, the cut point position deviation correction unit is based on an audio waveform in a certain period before and after the cut point detected from the first music content, and the second music content. The gist is to detect the positional deviation by comparing a speech waveform in a certain period before and after the detected cut point at a time interval shorter than the unit time.

  According to this feature, the cut point position deviation correction unit has a voice waveform in a certain period before and after the cut point detected from the first music content and a certain period before and after the cut point detected from the second music content. Compared with a speech waveform at a time interval (for example, μS order) shorter than the unit time (for example, several tens of milliseconds), the position deviation of the cut point is detected, so that the position deviation of the cut point can be detected with high accuracy. .

  According to a fourth feature, in the music editing device according to any one of the first to third features, an average value of amplitudes of the respective audio waveforms of the first and second music contents is compared, and the first value is determined according to the comparison result. The gist further includes an amplitude correction unit that corrects the amplitude of each audio waveform of the second music content.

  According to this feature, the audio waveform of the first and second music contents can be matched with high accuracy by further including an amplitude correction unit that corrects the amplitude of each audio waveform of the first and second music contents. It becomes possible.

  The fifth feature of the present invention is a music editing method for editing the first and second music contents, wherein the feature quantity is calculated from each of the first and second music contents, and the feature quantity exceeds a threshold value. A step of detecting a time point as a cut point, a step of detecting a positional deviation between the cut point detected from the first music content and a cut point detected from the second music content, and a step of correcting the positional deviation; And storing the corrected cut points as edit points used for editing the first and second music contents.

  A sixth feature of the present invention is a music editing program for editing the first and second music contents, wherein the computer calculates a feature value from each of the first and second music contents, and the feature value is a threshold value. A procedure for detecting a time point exceeding the cut point as a cut point, a procedure for detecting a positional deviation between the cut point detected from the first music content and the cut point detected from the second music content, and a procedure for correcting the positional deviation. The gist of the present invention is to execute a procedure for storing a cut point whose position shift has been corrected as an edit point used for editing the first and second music contents.

  According to the present invention, it is possible to provide a music editing device, a music editing method, and a music editing program capable of automatically obtaining a large number of editing points and suppressing noise generated as a result of editing.

  Next, an embodiment of the present invention will be described with reference to the drawings. In the description of the drawings in the following embodiments, the same or similar parts are denoted by the same or similar reference numerals.

(Hardware configuration example of music editing device)
FIG. 1 is a hardware configuration diagram of a music editing apparatus according to the present embodiment. The music editing device includes a display device 1, an input device 2, an MP3 (MPEG Audio Layer-3) codec 3, a D / A (Digital to Analog) converter 4, a speaker 5, a bus 6, and a DSP (DSP). A digital signal processor (CPU) 7, a central processing unit (CPU) 8, a hard disk drive (HDD) 9, and a memory 10 are provided.

  The display device 1, the input device 2, the MP3 codec 3, the D / A converter 4, the DSP 7, the CPU 8, the HDD 9, and the memory 10 exchange various data via the bus 6.

  The input device 2 includes a tuner that receives a broadcast signal such as a radio broadcast and demodulates an audio signal from the broadcast signal, and an A / D converter that converts the audio signal from the tuner into digital signal data. Alternatively, the input device 2 may be an optical disk device that reads music content from a CD or the like. The input device 2 has means for receiving user input (for example, a keypad, a keyboard, a mouse, or the like).

 The MP3 codec 3 has an encoder function for compressing and encoding digital audio data to obtain encoded audio data, and a decoder function for decoding the encoded audio data.

 The display device 1 displays the audio waveform and edit points of the music content in accordance with the digital audio data decoded by the MP3 codec 3 and the edit points stored in the edit point storage unit 93.

 The DSP 7 analyzes the sound waveform of the music content. CPU8 performs control of the whole music editing apparatus.

 The D / A converter 4 converts the digital audio data decoded by the MP3 codec 3 into an analog signal and supplies the analog signal to the speaker 5.

 The HDD 9 mainly stores encoded audio data. The memory 10 stores programs executed by the DSP 7 and the CPU 8 and is used as a work area during execution of the programs of the DSP 7 and the CPU 8.

(Example of functional configuration of music editing device)
FIG. 2 is a functional block diagram of the music editing apparatus according to the present embodiment. 2 includes a cut point detection unit 71 and an amplitude correction unit 72 in addition to the display device 1, the input device 2, the MP3 codec 3, the D / A converter 4, and the speaker 5 illustrated in FIG. A misalignment correction unit 73, a control unit 80, an edit processing unit 81, a music content storage unit 91, a cut point storage unit 92, an edit point storage unit 93, and a waveform data storage unit 11. Yes. The cut point detection unit 71, the amplitude correction unit 72, the misalignment correction unit 73, the cut point storage unit 92, and the edit point storage unit 93 constitute the edit point setting unit 100.

  The functions of the cut point detection unit 71, the amplitude correction unit 72, and the positional deviation correction unit 73 are realized by the DSP 7 in FIG. The functions of the control unit 80 and the editing processing unit 81 are realized by the CPU 8 in FIG. The music content storage unit 91, the cut point storage unit 92, and the editing point storage unit 93 are configured in the HDD 9 of FIG. The waveform data storage unit 11 is configured in the memory 10 of FIG.

  The control unit 80 acquires two (first and second) music contents that have been compression-coded from the music content storage unit 91 in accordance with, for example, a user input received by the input device 2. Here, it is assumed that the first and second music contents are music contents for the same music.

  The first and second music contents acquired by the control unit 80 are decoded by the MP3 codec 3. The decrypted first and second music contents are supplied to the amplitude correction unit 72.

  The amplitude correction unit 72 compares the average value of the amplitude of each audio waveform of the first and second music contents, and corrects the amplitude of each audio waveform of the first and second music contents according to the comparison result. Specifically, the amplitude correction unit 72 multiplies the amplitude of the music content having a small average value of the audio power among the first and second music contents by x, or the amplitude of the music content having a high average value of the audio power. Multiply by 1 / x.

  In this way, the amplitude error of the first and second music contents is corrected. The first and second music contents whose amplitude error is corrected are supplied to the cut point detection unit 71. In the course of the amplitude correction process described above, the control unit 80 may display the audio waveforms of the first and second music contents on the display device 1.

  The cut point detection unit 71 calculates a feature amount from each of the first and second music contents, and detects a time point when the feature amount exceeds a threshold as a cut point. Here, as the “feature amount”, for example, a change amount per unit time (about several tens of mS) of audio power can be used. The voice power can be obtained by calculating an amplitude spectrum from the voice waveform by Fourier transform or the like and obtaining the square sum of the amplitude spectrum. Details of the cut point detector 71 will be described later.

  In the following, a part of the music content divided by two adjacent cut points is referred to as a “segment”. Each of the first and second music contents is edited (combined) in segment units with the cut point as a contact point.

  The cut point storage unit 92 stores the cut points detected by the cut point detection unit 71. For example, the cut point storage unit 92 stores a cut point identifier for identifying each cut point and a detection time of each cut point (elapsed time from the start time of the music content) in association with each other.

  As shown in FIG. 3, the control unit 80 displays the audio waveforms of the first and second music contents and the cut points stored in the cut point storage unit 92 on the display device 1. Further, the control unit 80, for example, in accordance with a user input received by the input device 2, for each of the first and second music contents, a cut point used for editing, that is, a side to be combined with a cut point on the side to be combined Select the cut point.

  When a cut point used for editing is selected for each of the first and second music contents, the control unit 80 includes audio waveform data in a certain period before and after the cut point used for editing the first music content, Audio waveform data for a certain period before and after the cut point used for editing the second music content is stored in the waveform data storage unit 11.

  The position shift correction unit 73 compares the respective audio waveform data stored in the waveform data storage unit 11 with a time interval (sampling period) shorter than the unit time (about several tens of mS), thereby determining the position of the cut point. Detect deviation. Specifically, as shown in FIG. 3, the positional deviation correction unit 73 fixes one of the two speech waveforms, shifts the other speech waveform for each sampling period, and matches each speech waveform. Further, the positional deviation correction unit 73 calculates the positional deviation of the cut point and corrects the positional deviation when the respective voice waveforms match.

  In this way, by correcting the positional deviation of the cut point in the sampling period of μS order, the cut point on the coupling side and the cut point on the coupling side completely match in the time axis direction. Generation of noise can be avoided when content is combined.

  The edit point storage unit 93 stores the cut point whose position shift has been corrected by the position shift correction unit 73 as an edit point used for editing (combining) the first and second music contents.

  The edit processing unit 81 edits the first and second music contents using the edit points stored in the edit point storage unit 93. Specifically, the edit processing unit 81 uses the edit points stored in the edit point storage unit 93 as contact points to combine the first and second music contents to obtain one music content.

  The control unit 80 compresses and encodes one piece of music content obtained by editing using the MP3 codec 3, and stores the music content after compression coding in the music content storage unit 91. At that time, the control unit 80 may delete each original music content that has been edited (combined) from the music content storage unit 91.

  On the other hand, when the music content stored in the music content storage unit 91 is reproduced, after the music content is decoded by the MP3 codec 3, it is converted into an analog signal by the D / A converter 4 and output from the speaker 5. The

(Configuration example of cut point detector)
Next, a functional block diagram of the cut point detector 71 is shown in FIG. The cut point detection unit 71 includes a power calculation unit 711, a differentiation calculation unit 712, a threshold value comparison unit 713, and a cut point recording processing unit 714.

  As shown in FIG. 5, the power calculation unit 711 calculates audio power from the music content in the time domain.

  The differential operation unit 712 differentiates the sound power as the feature amount in order to detect a change in the sound power. As shown in FIG. 6, the absolute value of the difference in average power in a predetermined period W1 before and after time T is calculated as a differential value. As a result of differentiation, the differential value becomes large during a period in which the power change is large.

  If the differential value (feature value) in the predetermined period calculated by the differential calculation unit 712 is equal to or greater than the predetermined threshold, the threshold comparison unit 713 means that the change in the audio signal is large during that period. As shown, the start point or end point of the period is detected as a cut point. The predetermined threshold value may be changed depending on the type of music (genre, music tone, etc.).

  The cut point recording processing unit 714 records information related to the cut points detected by the threshold comparison unit 713 in the cut point storage unit 92.

  In addition to the configuration of FIG. 4, frequency analysis is performed in the frequency domain, and when the power in a specific frequency band exceeds a threshold and the state continues for a certain period of time, the change in sound is large, It may be configured to detect.

(Example of music editing process flow)
Next, an example of a music recording process flow according to the present embodiment will be described with reference to the flowchart shown in FIG.

 In step S11 of FIG. 7, the edit point setting unit 100 shown in FIG. 2 sets edit points for the first and second music contents.

 In step S12, the edit processing unit 81 edits the first and second music contents using the edit points set in step S11.

(Edit point setting processing flow example)
Next, an example of an edit point setting process flow will be described with reference to the flowchart shown in FIG.

  In step S101 of FIG. 8, the amplitude correction unit 72 compares the average values of the amplitudes of the audio waveforms of the first and second music contents, and compares the audio waveforms of the first and second music contents according to the comparison result. Correct the amplitude.

  In step S102, the cut point detection unit 71 calculates a feature amount from each of the first and second music contents, and detects a time point when the feature amount exceeds a threshold as a cut point.

  In step S103, the control unit 80 is combined with the cut point used for editing, that is, the cut point on the combining side, for each of the first and second music contents, for example, according to the user input received by the input device 2. Select the cut point on the other side.

  In step S104, the waveform data storage unit 11 holds audio waveform data for 2 t seconds from t seconds before the cut point used for editing the first music content. Here, the variable “t” is used to define a certain period before and after the cut point, and can be an arbitrary value (positive number) according to the storage capacity of the waveform data storage unit 11.

  In step S <b> 105, the misalignment correction unit 73 initializes the variable “i” to 0. Here, the variable “i” indicates one sampling period and is used to perform waveform matching for each sampling period.

  In step S106, the waveform data storage unit 11 holds audio waveform data for 2 t seconds from (2t + i) seconds before the cut point used for editing the second music content.

  In step S107, the misalignment correction unit 73 calculates the sum of the absolute values of the differences between the speech waveform data stored in step S104 and the speech waveform data stored in step S106, and the calculation result is a waveform. Store in the data storage unit 11. Thus, in order to determine the degree of coincidence of waveforms, the sum of absolute values of differences can be used.

  In step S108, the positional deviation correction unit 73 determines whether or not the variable “i” is smaller than “2t”. When it is determined that the variable “i” is smaller than “2t”, the process proceeds to step S109. On the other hand, when it is determined that the variable “i” is equal to or greater than “2t”, the process proceeds to step S110.

  In step S109, the positional deviation correction unit 73 increments the variable “i”, and then the process returns to step S106.

  In step S110, the misregistration correction unit 73 searches for a variable “i” that minimizes the sum of the absolute values of the differences calculated for each variable “i” in step S107.

  In step S111, the misregistration correction unit 73 stores the edit point storage unit 93 as an edit point obtained by shifting the cut point of the second music content by (it) seconds.

(Example of music playback processing flow)
Next, an example of a music reproduction processing flow according to the present embodiment will be described with reference to the flowchart shown in FIG.

 In step S401 in FIG. 9, the control unit 80 reads out the compression-encoded music content (encoded audio data) from the music content storage unit 91.

 In step S402, the MP3 codec 3 decodes the encoded audio data read in step S401 to obtain digital audio data.

 In step S403, the D / A converter 4 converts the digital audio data obtained in step S402 into an analog signal to obtain an audio signal.

 In step S404, the speaker 5 outputs the audio signal obtained in step S403.

(Specific example of music editing operation)
Next, a specific example of the music editing operation executed by the editing processing unit 81 will be described with reference to the waveform diagram of FIG.

  FIGS. 10A and 10B show audio waveforms of the same music content recorded at different times. In the speech waveforms shown in FIGS. 10A and 10B, a total of three editing points, editing points 1 to 3, are set.

  The segment divided by the edit points 1 and 2 in FIG. 10A is a segment with little noise and talk. The segment divided by the edit points 2 and 3 in FIG. 10A is a segment with a lot of noise. Here, “talk” means introduction of music by a DJ.

  Further, the segment divided by the editing points 1 and 2 in FIG. 10B is a segment with a lot of talk. The segment divided by the edit points 2 and 3 in FIG. 10B is a segment with little noise.

  Therefore, as shown in FIG. 10 (c), the edit processing unit 81 is divided into segments divided by the edit points 1 and 2 in FIG. 10 (a) and edit points 2 and 3 in FIG. 10 (b). Are combined using the edit point 2 as a contact point. In this way, music content with less noise and talk can be created.

(Action and effect)
As described above in detail, according to the present embodiment, it is possible to provide a music editing apparatus capable of automatically obtaining a large number of editing points and suppressing noise generated as a result of editing.

  In addition, according to the present embodiment, since the amount of change per unit time of audio power or frequency is calculated from the music content as a feature amount, a preferable feature amount can be obtained for the music content, and a cut point with high accuracy is detected. can do.

  Furthermore, according to the present embodiment, the positional deviation correction unit 73 has a sound waveform in a certain period before and after the cut point detected from the first music content and a constant before and after the cut point detected from the second music content. The positional deviation of the cut point is detected by comparing the voice waveform in the period with a time interval (μS order) shorter than the unit time (several tens of milliseconds), so that the positional deviation of the cut point can be detected with high accuracy. .

  Furthermore, according to the present embodiment, by further including the amplitude correction unit 72 that corrects the amplitude of each voice waveform of the first and second music contents, each voice waveform of the first and second music contents can be accurately obtained. It is possible to match.

(Other embodiments)
As mentioned above, although this invention was described by embodiment, it should not be understood that the description and drawing which form a part of this indication limit this invention. From this disclosure, various alternative embodiments, examples and operational techniques will be apparent to those skilled in the art.

 In the above-described embodiment, the case where the amount of change per unit time of audio power is used as the feature amount of the music content has been described. As another embodiment, as shown in FIG. 11, a change amount per unit time of the frequency of the audio signal may be used as the feature amount of the music content.

  Furthermore, in the above-described embodiment, an example of editing two music contents has been described. However, the present invention may be applied when editing three or more music contents.

  In the above-described embodiment, an example in which the HDD is used to store the music content or the like has been described. However, the present invention is not limited to the HDD, and a semiconductor storage device such as a flash memory may be used.

  Moreover, although the case where MP3 is used as the compression encoding method of the music content has been described, other compression encoding methods may be adopted.

  Furthermore, each processing flow described in the above-described embodiment can be implemented as a computer program and executed by a personal computer (PC) or the like.

  Thus, it should be understood that the present invention includes various embodiments and the like not described herein. Therefore, the present invention is limited only by the invention specifying matters in the scope of claims reasonable from this disclosure.

It is a block diagram which shows the hardware structural example of the music editing apparatus which concerns on embodiment of this invention. It is a functional block diagram which shows the function structural example of the music editing apparatus which concerns on embodiment of this invention. It is a figure for demonstrating the function of the position shift correction | amendment part which concerns on embodiment of this invention. It is a functional block diagram which shows the function structural example of the cut point detection part which concerns on embodiment of this invention. It is a wave form diagram for demonstrating operation | movement of the power calculation part which concerns on embodiment of this invention. It is a wave form chart for explaining operation of a differential operation part concerning an embodiment of the present invention. It is a flowchart which shows the example of a music edit process flow which concerns on embodiment of this invention. It is a flowchart which shows the example of an edit point setting process flow concerning embodiment of this invention. It is a flowchart which shows the example of a music reproduction process flow which concerns on embodiment of this invention. It is a figure which shows the specific example of the music edit operation | movement which concerns on embodiment of this invention. It is a wave form chart for explaining cut point detection operation concerning other embodiments.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Display apparatus 2 ... Input device 3 ... MP3 codec 4 ... D / A converter 5 ... Speaker 6 ... Bus 7 ... DSP
8 ... CPU
9 ... HDD
DESCRIPTION OF SYMBOLS 10 ... Memory 11 ... Waveform data storage part 71 ... Cut point detection part 72 ... Amplitude correction part 73 ... Misalignment correction part 80 ... Control part 81 ... Edit processing part 91 ... Music content storage part 92 ... Cut point storage part 93 ... Editing Point storage unit 100 ... editing point setting unit 711 ... power calculation unit 712 ... differentiation operation unit 713 ... threshold value comparison unit 714 ... cut point recording processing unit

Claims (2)

A music editing apparatus that combines a part of the recorded first music content and a part of the second music content that is the same music as the first music content and is recorded separately from the first music content. There,
A cut point detection unit for detecting, as a cut point, a time point when the amount of change per unit time of audio power or frequency exceeds a threshold value from each of the first and second music contents;
A cut point selection unit for selecting a cut point of the first music content and a cut point of the second music content used for the combination;
A cut that compares the audio waveform before and after the cut point of the selected first music content with the audio waveform before and after the cut point of the second music content, detects misalignment of these cut points, and corrects the misalignment A point correction unit;
A coupling unit that couples a part of the first and second music contents using the corrected cut point as a contact;
Cut point selection unit, based on the user input, the music editing apparatus according to claim you to select a cut point of the cut point and the second music of the first music content to be used for binding.
A music editing program that combines a part of the recorded first music content and a part of the second music content that is the same music as the first music content and is recorded separately from the first music content. In the computer, a procedure for detecting, as a cut point, a change amount of audio power or frequency per unit time from each of the first and second music contents exceeds a threshold value;
A procedure for selecting a cut point of the first music content and a cut point of the second music content to be used for combining;
A procedure for comparing the audio waveform before and after the cut point of the selected first music content with the audio waveform before and after the cut point of the second music content, detecting positional deviation of these cut points, and correcting the positional deviation. When,
And a step of combining a part of the first and second music contents using the corrected cut point as a contact point , and
And in the procedure for selecting a cut point, based on the user input, the cut point of the first music content to be used for binding and the cut point of the second music is selected music piece editing program characterized Rukoto.

Images