JP4877811B2 - Specific section extraction device, music recording / playback device, music distribution system - Google Patents

Description

  The present invention relates to an apparatus for extracting a specific section such as a chorus section from music data.

  In recent years, many recording devices have been developed that are equipped with a large-capacity recording medium such as an HDD and can record / reproduce various music contents.

  When searching for music content, the music information included in the music content (such as its header area) can be used as a keyword. However, music content to which music information is not given, such as music content that is aired on FM radio or the like, may be recorded in the recording device. In such a case, it is not possible to search by keyword, so it is necessary to search while reproducing the contents sequentially. Therefore, in order to shorten the search time, a recording apparatus capable of automatically and sequentially reproducing a part of each music content is desired.

For this purpose, a technique for automatically extracting a part from the music content is required. For example, the automatic music division method disclosed in Patent Document 1 can be used. This automatic music division method subdivides music content into short sections, calculates mutual similarities, and extracts points where the similarity changes greatly in adjacent sections as music division points. Thereby, a part of music content can be extracted automatically.
JP 2006-163063 A

  However, even if a part of the music content can be automatically extracted using the method described in Patent Document 1, a part that is not so characteristic (for example, A melody, B melody) may be extracted. is there. Even if such a part is heard, the user may not be able to recognize that it is the music he / she is looking for. Therefore, a recording apparatus is desired that not only can sequentially reproduce a part of each piece of music content, but also can sequentially reproduce the chorus part of each piece of music content.

  The present invention has been made in view of such a problem, and an object of the present invention is to provide a specific section extraction device that can efficiently extract a specific section such as a rust portion from music content.

In order to achieve the above object, the specific section extraction device of the present invention includes a cut point detection unit that detects a time point when the audio signal level of music data or the amount of change thereof is a predetermined value or more, and the audio of the music data. A section before and after the time point at which the signal level or the amount of change in the audio signal level is maximum in the music data is set as a temporary specific section candidate, and similar to the specific section candidate in the music data based on the time information of the cut point It is characterized by having an extraction part which judges whether the section to do exists or not, and when the section similar to the specific section candidate exists , extracts the specific section candidate as a specific section.

  It is considered that the “rust” portion in the music content is likely to exist multiple times in the music content. Utilizing such a property, the above-described specific section extraction device can efficiently extract a specific section such as a rust portion.

  By the way, in the rust portion, the audio signal is often large. Therefore, it is considered that the rust portion can be searched efficiently by searching from the portion where the audio signal is large first, rather than sequentially searching for the presence or absence of similar sections from the beginning of the music.

For this reason, before Symbol extractor, and the front and rear sections of the time variation of the voice signal level or the voice signal level of the music data is maximum in the song as the temporary specific section candidate.

Also, the audio signal level may increase due to noise mixing during content recording. In this case, if the rust portion is simply extracted based on the level of the audio signal level, the noise portion may be erroneously extracted. Therefore, if a specific section candidate is extracted as a specific section after confirming the existence of another section similar to the specific section candidate , a specific section such as a rust portion can be correctly extracted even when noise is mixed. Can do.

Preferably, the extraction unit, when said specific section candidates with similar section not present, identify the front and rear sections of the time variation of the voice signal level or the voice signal level of the music data is the second largest in the music of temporary and section candidates, when the specific section candidates similar to section exists, extracts the specific segment candidate as the specific interval.

  Rust extraction processing is not terminated when there is no section similar to a section with a high audio signal level, but a section with the second highest audio signal level is selected as a rust candidate. Thereby, the probability that a rust part can be detected can be improved. Also, even if the section with the second highest audio signal level is not the original rust, there is a high possibility that the portion where the audio signal is large remains in the user's impression. Thus, such a specific section extracting device is very useful particularly when used for an application in which a part of music content is automatically and sequentially reproduced.

  According to the specific section extraction device of the present invention, a specific section such as a rust portion can be efficiently extracted from the music content.

  Hereinafter, the present invention will be described with reference to the drawings illustrating embodiments thereof. FIG. 1 is a configuration diagram of a recording / reproducing apparatus according to an embodiment of the present invention. This recording / reproducing apparatus records a broadcast signal such as FM radio broadcast on an HDD (Hard Disk Drive) 10. In parallel with this, the music data portion included in the broadcast signal is detected. Further, it has a function of extracting the chorus section in the music data and recording the chorus section data in the HDD 10.

  The tuner unit 1 selects and receives the received broadcast signal such as FM radio broadcast and demodulates it into an audio signal. The A / D converter 2 converts the analog audio signal selected by the tuner unit 1 into a digital signal.

  The MP3 (MPEG Audio Layer-3) codec 3 has an encoder function that encodes digital audio data, generates encoded data that is compressed, and outputs the encoded data as a pair, and a decoder function that decodes the encoded data. Have. The D / A converter 4 converts the digital audio data decoded by the MP3 codec 3 into analog signal data. The analog signal data is input to the speaker 5 through an amplifier omitted in the figure.

  The DSP 7 calculates the sound power obtained by squaring the amplitude value of the sound signal in order to detect the sound signal level based on the sound signal. Further, the DSP 7 calculates the change amount of the sound power in order to detect the change amount of the sound signal level. Further, the DSP 7 detects the timing at which the amount of change in the sound power becomes a predetermined value or more as a cut point. Further, the DSP 7 calculates a feature quantity in the frequency domain only in the vicinity of the cut point, for example, MFCC. Then, the likelihood of the calculated MFCC and the MFCC calculated based on the sample audio signal stored in advance in the external memory 11 is calculated. Here, the likelihood is the likelihood for the music.

  The CPU 8 controls the operation of the recording / reproducing apparatus via the bus 6. In addition, the music section is detected from the broadcast signal using the time information of the cut point detected by the DSP 7. Further, the chorus section is extracted from the detected music data.

  The HDD 10 is a large-capacity recording device that records encoded data and time information via an HDD interface unit 9 that implements an ATA interface, for example. The memory 11 expands the execution program, temporarily stores data generated by the arithmetic processing, and has a function of delaying audio data immediately after AD conversion for a certain period. The MP3 codec 3, DSP 7, CPU 8, HDD interface unit 9, and memory 11 exchange various data with each other via the bus 6.

(Description of processing to detect music data part included in broadcast signal)
This recording / reproducing apparatus finally extracts the chorus section of the music from the broadcast signal. First, the procedure of the process for detecting the music data part from the broadcast signal will be described.

  FIG. 2 is a functional block diagram of the recording / reproducing apparatus of the present embodiment. In this recording / reproducing apparatus, the audio signal selected by the tuner 1 is input to the A / D converter 2 and converted to digital, and then input to the MP3 codec 3 together with the time information, input to MP3 data, and converted to MP3 data. The data is compressed and encoded, and is recorded continuously on the HDD 10 via the HDD interface unit 9 during the recording period as a pair with the time information.

  The digital audio data from the A / D converter 2 is stored in the delay memory 11a that is delayed by the time required for the processing of the DSP 7, and the audio power corresponding to the audio signal level in the audio power calculation unit 71 in the DSP 7, that is, The square value of the amplitude of the audio signal is calculated.

  The cut point detection unit 72 in the DSP 7 detects the timing corresponding to the change in the audio signal level, that is, the timing when the change in the audio power value is greater than the predetermined value as the cut point in order to identify the section corresponding to the music from the audio signal. Then, a detection output is issued. At the same time, the time information at the cut point and the amount of change are stored in the temporary storage memory 11C.

  FIG. 3 is a waveform diagram for explaining the operation of the cut point detection unit 72. FIG. 3A shows the change of the audio power value, and FIG. 3B shows the change of the change amount (differential value). As shown in FIG. 3, the cut point detection unit 72 uses the audio power value calculated by the audio power calculation unit 71 as a cut point at times Tm and Tm + 1 at which the differential value becomes a maximum point greater than a predetermined threshold. Detect as. Then, the detection result is input to the frequency feature amount calculation unit 73.

  In the frequency feature quantity calculation unit 73, the audio data output after being delayed from the delay memory 11a for a predetermined time is synchronized with the output from the cut point detection unit 72 and slightly delayed from the cut point slightly before the cut point. The feature amount of the frequency such as MFCC is temporarily calculated in the minute period at the determined timing and input to the likelihood calculating unit 74.

  In this embodiment, paying attention to the fact that the frequency feature amount of music and spoken voice is different, the frequency feature amount of typical music and the frequency feature amount of spoken voice are stored in advance in the external memory 11b as reference data for comparison. Yes. Accordingly, the likelihood detection unit 74 in the DSP calculates the likelihood between the feature amount calculation output before and after the cut point input from the frequency feature amount calculation unit 73 and the reference data, and the likelihood calculation output is output to the CPU 8. To the cut point determination unit 81.

  In calculating the likelihood, instead of the above-mentioned method for obtaining the likelihood by comparison with the reference data, the likelihood is obtained by substituting the frequency feature quantity into a preset evaluation function to obtain the likelihood. There is also a calculation method, and it is not always necessary to compare with the reference data stored in the external memory 11b.

  Next, the cut point determination unit 81 determines whether the audio signal at the cut point is music or speech based on the likelihood calculation output. In the temporary storage memory 11C in which the time information obtained from the cut point detection unit 72 and the amount of change are stored, the determination result is further stored in association with each other.

  FIG. 4 shows a table of the temporary storage memory 11C in which the determination results as described above are stored in association with each other.

  The time length determination unit 83 has empirically found that the music continues for a predetermined time, for example, 100 seconds or more, and even when the sampling point interval of the voice is less than 100 seconds, the sampling point during that time is determined as music. The interval is not regarded as music, and the interval between sampling points determined to be other than spoken voice, that is, music, is measured, and an interval of 100 seconds or more is determined as a music interval.

  The music section detection unit 82 that receives the determination output obtained from the time length determination unit 83 rewrites the table in the temporary storage memory 11C and changes it to a table for each music (final table).

  FIG. 5 is a diagram showing the final table rewritten in the temporary storage memory 11C. In this final table, T6 was once determined to be a song, but was deleted from the table because it was not considered a song because the interval between sampling points T5 and T7 before and after it was determined to be a speaking voice was short. It is shown that.

  This final table is supplied to the HDD interface unit 9 via the music section detection unit 82 when the recording operation is completed, and is further stored in the HDD 10.

  The final table is recorded in the HDD 10 while leaving the intermediate cut point and the amount of change in addition to the start point and end point of the music, but is used for the purpose of reproducing the chorus part during reproduction.

  The encoded data recorded in the HDD 10 sequentially reads out the encoded data corresponding only to the music section specified in the final table according to the editing / playback operation, and is input to the MP3 codec 3. The MP3 codec 3 decodes the encoded data, is converted into an audio signal by the D / A converter 4, and then is output from the speaker 5. Thereby, it is possible to detect only the music from the audio signal including the conversation and extract the music and reproduce it.

  The maximum point detection unit 84 detects a time when the sound power is high in the music section. Specifically, using the audio power value calculated by the audio power calculating unit 71 and the time information of the music section extracted by the music section detecting unit 82, the time when the audio power is high in the music section is specified. . Then, the time at a plurality of time points when the sound power is high (for example, the time of the best time when the sound signal is large) is recorded in the HDD 10.

  By the above operation, the HDD 10 stores (1) broadcast signal data encoded by the MP3 codec 3, (2) the start time, end time, and cut point time of the music section detected by the music section detection unit 82. Information, (3) Information of the time when the audio power is high among the music sections detected by the music section detection unit 82 is recorded.

  In the above example, only about three cut points are included in one piece of music data (FIGS. 4 and 5). However, this is merely an example and is merely an example. In addition, it is preferable that the number of cut points is larger than this for extracting a chorus section, which will be described later. In practice, a threshold value is set so that about several tens of cut points are detected in one piece of music. Is preferred.

(Explanation of the process to extract the chorus section from the music data part)
Next, a process for extracting a chorus section from the music data portion will be described.

  First, the chorus section detection unit 85 reads information on the cut point time in the music section and information on the time when the audio power in the music section is high from the HDD 10. Then, using these pieces of information, it is examined whether or not there are a plurality of similar melody portions in the music section (based on the knowledge that the rust portion usually appears multiple times in the music). Specifically, the presence or absence of a similar melody section is estimated by examining whether there is a section having a similar combination of cut point intervals. If it is detected that a similar melody section exists, the section is detected as a chorus section.

With reference to FIG. 6, the specific procedure of the process which the chorus area detection part 85 performs is demonstrated. First, time information with the highest audio power is acquired from the HDD 10 during the music section (here, this time is T _max ). Next, time information of a plurality of cut points existing around time T _{max is} acquired from the HDD 10. For example, here, information on five cut points c ₄ , c ₅ , c ₆ , c ₇ , and c ₈ is acquired. Then, the music data between c _{4 and} c ₈ (“section A” in FIG. 6) is first assumed to be a chorus candidate section. The reason why such a section with high audio power is assumed to be a chorus candidate is based on the knowledge that a portion with high audio power is likely to be a chorus part in the music content.

Next, it is checked whether or not a section similar to the section A exists in the music content. Specifically, first, the time interval (interval) of the cut point group in the section A is calculated. Here, I = {i ₁ , i ₂ , i ₃ , i ₄ } (where i _n = T (c _{n + 4} ) −T (c _{n + 3} ), and T (c _n ) is the time of the cut point c _n . Is calculated).

Then, it is checked whether or not a cut point group having a combination of cut point intervals similar to I exists in the music content. For example, in the example of FIG.
D _n = (i ₁ −i _n ) ^ 2 + (i ₂ −i _{n + 1} ) ^ 2 + (i ₃ −i _{n + 2} ) ^ 2
+ (I ₄ −i _{n + 3} ) ^ 2 (n = 5, 6, 7,...)
However, i _n = T (c _{n + 4} ) −T (c _{n + 3} )
Are calculated sequentially. If D _n smaller than the predetermined value exists, it is determined that the section A and the section of the cut points c _{n + 3 to} c _{n + 7} are similar to the section A. In the example of FIG. 6, the section c _{12 to} c ₁₆ (“section B”) is determined to be a section similar to the section A. Thus, when a section (section B) similar to the rust candidate section can be found, the rust candidate section (section A) is detected as a true rust section.

Rust detection by the above procedure is excellent in the following points.
(1): The sound signal is often large in the rust portion. Therefore, rather than sequentially searching for the presence or absence of similar sections from the beginning of the music, the rust portion can be efficiently detected by searching first from the portion where the audio signal is large as described above.
(2): The audio signal level may increase due to noise mixing during content recording. In this case, if a rust portion is simply extracted based on the level of the audio signal level, a portion that is not originally rust may be extracted. Therefore, as described above, it is only necessary to confirm the existence of other sections with similar combinations of cut point intervals, and if the section is identified as a rust portion, the rust portion is correctly detected even when noise is mixed. it can.

In the above example, when there is no D _n that is smaller than the predetermined value, it is determined that there is no section similar to the section A. That is, it is determined that the section A is not a chorus section. In this case, the chorus section detection unit 85 acquires time information having the second largest audio power from the HDD 10 during the music section. Thereafter, the chorus section is detected in the same procedure as described above.

  The chorus section extraction unit 86 extracts the chorus section from the music data decoded by the MP3 codec 3 based on the time information of the chorus section detected by the chorus section detection unit 85. The audio signal of the chorus section extracted by the chorus section extraction unit 86 is output by a speaker or the like. Alternatively, it may be recorded in the HDD 10 as chorus section audio data.

  The chorus section can be extracted from the music data by the procedure as described above.

(Application to music recording and playback devices and music distribution systems)
If the voice data of the chorus section extracted by the procedure as described above is recorded in the HDD, various uses and application examples can be considered.
(1): The user can efficiently search for a desired music piece by sequentially reproducing the chorus parts.
(2): It is possible to realize a music recording / playback apparatus that displays a list of songs in a thumbnail format and clicks on a song icon to reproduce the chorus portion of the song.
(3): A music distribution system capable of purchasing a music selected by clicking an icon or the like can be realized. Normally, when purchasing music using a music distribution system, the user needs to pay a predetermined price. However, if he / she owns chorus partial data, the user should purchase the music at a lower price than usual. Can build a business model.

  In the present embodiment, the functions are distributed between the DSP 7 and the CPU 8, but the present invention is not particularly limited to this, and both functions may be realized only by the CPU 8, and the A / D converter 2, A configuration in which all the software including the functions of the MP3 codec 3 and the D / A converter 4 is processed by the CPU 8 may be employed.

  Further, the present invention is not limited to the above-described embodiment, and it goes without saying that various modifications, substitutions, and the like are possible as long as the description is within the scope of the claims.

  For example, in the above embodiment, a section having a similar combination of cut point intervals is extracted as the specific section. However, a section having a similar cut point time distribution may be extracted as the specific section.

  In the above description, the time point when the audio signal changes greatly is defined as the cut point, but the time point when the size of the audio signal is larger than a predetermined value may be set as the cut point.

  Further, when the music has relatively little intonation, the number of detected cut points may be very small. As a result, it is considered that the presence / absence determination of the similar section based on the cut point base as described above cannot be performed. Specifically, (1) the number of cut points is too small and the similarity determination itself cannot be performed, and (2) the number of cut points as the number of samples is small, so that the rust section may be erroneously detected. The two are considered. In such a case, a predetermined threshold value for recognition as a cut point may be set low.

  That is, (1) when the number of cut points in the music data is smaller than a predetermined number, the threshold value for identifying as a cut point is lowered, and (2) when the chorus section cannot be detected, the second largest Instead of setting the preceding and following sections of the audio power as the chorus section candidates, a process of redetecting cut points (lowering the cut point threshold) may be performed.

It is a block diagram of the recording / reproducing apparatus which concerns on embodiment of this invention. It is a functional block diagram of the recording / reproducing apparatus which concerns on this Embodiment. 6 is a waveform diagram for explaining the operation of the cut point detection unit 72. FIG. It is a table of temporary storage memory 11C. It is a figure which shows the last table rewritten by 11 C of temporary storage memories. It is a figure for demonstrating the process which specifies the area where the combination of the detected cut point interval is similar.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Tuner part 2 A / D converter 3 MP3 codec 4 D / A converter 5 Speaker 6 Bus 7 DSP
8 CPU
9 HDD interface unit 10 HDD
11 memory

Claims (5)

A cut point detection unit for detecting a time point when the audio signal level of the music data or the amount of change thereof is equal to or greater than a predetermined value;
The period before and after the time point when the change amount of the audio signal level or the audio signal level of the music data is the maximum in the music data is a temporary specific section candidate,
Based on the time information of the cut point, it is determined whether or not there is a section similar to the specific section candidate in the music data, and when there is a section similar to the specific section candidate , the specific section candidate is A specific section extraction apparatus including an extraction unit that extracts a specific section. The extraction unit is present when the combination of cut point intervals similar to the combination of cut point intervals existing in the specific section candidate exists in the music data, or exists in the specific section candidate. when the time of the distribution of the cut points similar to the distribution of the time of the cut point is present in the music data, and a section similar to the specific section candidate in the music data is present, the specific section candidate The specific section extraction device according to claim 1 , wherein the specific section is extracted as a specific section. The extraction unit includes:
When there is no section similar to the specific section candidate , the section before and after the time when the audio signal level of the music data or the change amount of the audio signal level is the second largest in the music is set as a temporary specific section candidate,
When sections similar to the specific segment candidate is present, the specific section extracting device according to claim 1 or 2 for extracting the specific section candidate as a specific section. The specific section extraction device according to any one of claims 1 to 3,
A music recording / reproducing apparatus comprising: a recording unit that records music data related to a specific section extracted by the extraction unit as specific section data. The recording unit is
A display unit that displays a list of music information related to the specific section data to be recorded,
A selection unit for selecting a song from the song list displayed on the display unit;
The music recording / reproducing apparatus according to claim 4 , further comprising a reproducing unit that reproduces specific section data related to the music selected by the selection unit.