TWI516098B - Record the signal detection method of the media - Google Patents

Description

Signal detection method for recording media

The invention relates to a method for detecting signals in a recording medium, in particular to a method for automatically detecting similar fragments by using a signal processing and comparison technique to compare the recorded signals with the signals provided by the content provider.

With the advancement of technology, multimedia information has become the focus of our lives. On the topic of multimedia search, how to respond quickly to the needs of users has always been a hot research topic, especially in the era of information explosion, the amount of information is even more The speed of multiples grows, and it is time consuming and laborious to search by manual means. Therefore, we hope to use the data indexing technology to extract the connotation features similar to the user's interest from the original disorganized data, so as to achieve fast and effective search and retrieval.

At present, the retrieval technology of multimedia materials is mainly based on the text information of multimedia itself. In the prior patent technology, a similar concept is mentioned: Taiwan DigitalInn Publication No. 200307874, which is a method and system, which can be used. The portable device uploads the audio file to the server, and uses the audio fingerprint to identify the content of the audio file and compares the same audio file in the search database. However, in this patent, the audio fingerprint is limited to the time and region when a song is broadcasted. The surrounding text information does not include the audio information extracted by the music itself.

In addition, there are also methods for using audio information in the literature, such as Microsoft Taiwan Patent No.: I329455 is a system and method for identifying and extracting repeated sounds or video objects from multimedia streams, using self-correlation coefficients as the identification criteria. However, the patent does not include an indexing technique for accelerated search and is used for Musical features such as BPM and Bark Spectra should not be able to effectively identify music clips. Another example is HP patent US Pat. No. 6,995,309, which is a system and method for music recognition, recording a music sample, generating a feature vector of the sample, and calculating a feature difference together with a music feature vector in the music library, if the song matching rule is met. , providing song information to the user. This patent focuses on its matching comparison method, using FFT overlap access convolution and angle cosine formula; as for feature extraction method is not within the scope of patent declaration; Dolby patent US patent number US20100205174, is a use of multiple search combinations to improve The technology of audio/video fingerprint search for correctness.

For the audio/video clip, get its fingerprint features and search the database for fingerprints to get a possible match. The method for measuring the difference between fingerprints includes Hamming distance, bit error rate, Lp norm, L2 distance, autocorrelation coefficient, etc. If there is a match, then the search success message is returned to the user, if not, Will return the search results are not in the database. The shortcomings of the above two patents are that there are many mathematical operations, a large amount of calculation, and a slow return speed. In addition, GraceNote patent US 7,549,051 B2 establishes an index and pattern match based on the first-order change of the Time-Frequency components of the signal; the Shazam patent US 2009/0265174A9 is based on the signal Time-Frequency landmarks and generates the frequency peak pair invariant/ Variant two kinds of hash values, first match when searching The invariant pattern finds all possible frequency peaks and uses the histogram statistics of relative time offsets to find music with a histogram peak.

Both of the above patents can only be used for music files and cannot handle multimedia movies.

It can be seen that there are still many shortcomings in the above-mentioned methods of use, which is not a good design, but needs to be improved.

In view of the shortcomings derived from the above-mentioned conventional methods, the inventors of the present invention have succeeded in research and development and completed the signal detection method of the recording medium of the present invention after painstaking research and development.

With the rapid development of compression methods, digital content has become a part of life. In such an environment, it is often the case that we are interested in some content but cannot use traditional keyword search to find out the content. Information. The use of recording media search can achieve the goal of searching for the digital content in this case. For example, when we hear a music that is very interesting, but we don't know any information about the music, we can use the recording device to record the music clip, and then use the search system to analyze the feature value of the music to find out The low-order eigenvalues are then used to find the music that is most likely to contain the music segment in the music database, so that even if we can't place keywords on the music, the system can follow the music itself. The feature then completes the search work.

The object of the present invention is to provide a signal for recording media The detection method can be applied to mobile devices to find similar songs or movies through recording and recording; it can also be applied to electronic storage devices to automatically analyze and mark and organize digital content.

The recording medium signal detecting method for achieving the above object aims to extract, group and index the signal content information provided by the content provider, and when the user wants to query a certain media signal, the recording and analyzing media device can be recorded and analyzed. Signal content, perform feature extraction and find the distance between each feature and the group center of the content provider, take the group with the smallest distance as the representative, and then use the index file to find the content position corresponding to the representative, and calculate the recorded content. The content of the signal is similar to the content near that location to determine the most similar segment of the signal content.

In order to enable the office to further understand the technical content of the present invention, a preferred embodiment will be described below.

The invention relates to a signal detecting method for recording media, which performs analysis and processing by recording multimedia signal content, and searches for similar multimedia segments, which at least comprises a connected multimedia index establishing method and a recording media index matching method.

The step of the multimedia index establishing method may further include:

a. Time division, N multimedia files, numbered 1~N, are divided into multimedia segments of length several seconds in chronological order, each segment is named as the multimedia file number plus the bottom line and a time segment number, this serial number Equivalent to the beginning of the fragment in the original file In the case is the first few seconds.

b. Feature extraction, step a. If the multimedia segments are video, extract 960 dimensions of scene-oriented features (GIST); if it is audio, extract 13-dimensional Mel frequency cepstral features (MFCC).

c. feature coding, the feature extracted by each segment in step b., the feature extracted from the latter segment, and the difference of each dimension is binary coded. If the difference between the two is greater than 0, the code is 1; Encoded to 0, so if the multimedia clip is video, each clip will have 960 dimensions 0 or 1. If the multimedia clip is audio, each clip has 13 dimensions 0 or 1, which is called high dimension. Encoding or feature encoding.

d. Establish an index.

However, in the above step d, the indexing is to establish a multimedia index, that is, according to the feature coding, the high-dimensional codes are grouped by using a grouping algorithm and the group center is obtained, and the multimedia file corresponding to each high-dimensional code is encoded. The number and time segment sequence number are recorded in the index of the group center closest to the binary code.

Further, the clustering algorithm is to use an unsupervised grouping method, and the number of given group centers is the root number of the total number of multimedia.

Moreover, the calculation of the distance is based on Hamming distance or Euclidean distance.

The method for detecting a signal of a recording medium of the present invention, wherein the recording medium index comparison method may further include:

a. Time division, the multimedia files are divided into multimedia segments of several seconds in chronological order, each segment is named as one time Inter-segment number, which is equal to the first few seconds of the beginning of the fragment in the original file;

b. Feature extraction, if the multimedia segment of step a. is video, extract 960 dimensions of scene-oriented features (GIST); if it is audio, extract 13-dimensional Mel frequency cepstral feature (MFCC);

c. feature coding, the feature extracted from each segment in b., and the feature extracted from the latter segment, the difference between each dimension is binary coded, if the difference between the two is greater than 0, the code is 1; Encoded to 0, so if the multimedia clip is video, each clip will have 960 dimensions 0 or 1. If the multimedia clip is audio, each clip has 13 dimensions 0 or 1, which is called high dimension. Encoding or feature encoding.

d. Index comparison.

Wherein, in the step d, the index comparison encodes the feature of each segment, and finds the group center closest to the high-dimensional coding distance one by one compared with the group centers, and then in the group center index The corresponding multimedia file name and time segment are included, and the distance between the recorded media content feature and the feature of the time segment is separately determined, and the segment with the smallest distance is the most similar multimedia segment.

However, the index of the step d is calculated based on the Hamming distance or the Euclidean distance.

The invention uses the signal detection of music as an example, and the signal of the music can be an mp3 or wav file format.

Referring to FIG. 1 , when the content provider wants to establish an index of its music content, the following four steps can be performed in sequence using the multimedia index creation method proposed by the present invention:

a. Cut all the music into 38 clips per second.

b. Each segment is extracted by the Mel's Guided Spectrum Coefficient (MFCC) method. The steps of a. and b. are shown in Figure 3.

c. Subtract the Mel spectral coefficient vector of each segment from the latter vector. If the value in the difference vector is greater than 0, it is set to 1, and the rest is set to 0. The result is shown in the first column of Figure 5. .

d. The difference vector established by c. is divided into 10000 groups by K-means algorithm, as shown in Fig. 4, and the group center and index files are established. The index file format is shown in the second and third columns of Figure 5.

Referring to FIG. 2, when the user wants to query the index of the music content, the following four steps can be performed in sequence by using the signal detection method of the recording medium proposed by the present invention:

e. Cut the query music into 38 segments per second.

f. Each segment is extracted by the Mel's Guided Spectrum Coefficient (MFCC) method. The steps of e. and f. are shown in Figure 3.

g. Subtract the Mel spectral coefficient vector of each segment from the latter vector. If the value in the difference vector is greater than 0, it is set to 1, and the rest is set to 0. The result is shown in the first column of Figure 5. .

h. Compare the code created by each g. with the index file created by d. Find all similar segments in the database, compare the segments of the query music and the code before and after the segment. The coding, plus the total, calculates the Hamming Distance, and the music with the smallest Hamming distance in the last database is the desired.

The invention provides a signal detecting method for a recording medium, which has the following advantages when compared with other conventional technologies:

1. The present invention automatically processes multimedia content and produces a multimedia index.

2. The invention can automatically find corresponding multimedia content segments by recording media signals.

3. The invention can automatically analyze the digital content in the electronic storage device for automatic marking and sorting.

4. The present invention utilizes coding and sampling techniques to find results relatively quickly.

5. The present invention utilizes grouping and fault tolerance techniques to effectively combat interference or noise when recording media signals.

The detailed description of the present invention is intended to be illustrative of the preferred embodiments of the present invention, and is not intended to limit the scope of the invention. The patent scope of this case.

To sum up, this case is not only innovative in terms of technical thinking, but also has many of the above-mentioned functions that are not in the traditional methods of the past. It has fully complied with the statutory invention patent requirements of novelty and progressiveness, and applied for it according to law. Approved this invention patent application, in order to invent invention, to the sense of virtue.

100‧‧‧Time Division

200‧‧‧ Feature extraction

300‧‧‧Character code

400‧‧‧ Indexing

500‧‧‧ index comparison

Please refer to the detailed description of the present invention and the accompanying drawings, and the technical contents of the present invention and the functions thereof can be further understood. The related drawings are: FIG. 1 is a flowchart of a method for establishing a multimedia content index according to the present invention; The flowchart of the method for detecting a recording medium signal of the present invention; FIG. 3 is a schematic diagram of a method for establishing a multimedia content index and a method for detecting a timing of a recording medium signal, and a method for extracting features; FIG. 4 is a method for establishing a multimedia content index and recording a media signal. An example of feature encoding of a measurement method; and FIG. 5 is an example of indexing a method for establishing a multimedia content index.

100‧‧‧Time Division

200‧‧‧ Feature extraction

300‧‧‧Character code

400‧‧‧ Indexing

500‧‧‧ index comparison

Claims (7)

A signal detecting method for recording media, which performs analysis and processing by recording multimedia signal content, searches for similar multimedia segments, and at least includes a connected multimedia index establishing method and a recording media index matching method, wherein the multimedia index establishing method The steps further include: a. timing division, dividing N multimedia files, numbered 1~N, into chronologically divided into multimedia segments of several seconds in length, each segment being named as the multimedia file number plus the bottom line and a time Fragment serial number, the serial number is equal to the first few seconds in the original file; b. feature extraction, in step a. in the multimedia segment, if the video segment is video, extract 960 dimensions of the scene-oriented feature (GIST); For audio, the 13-dimensional Mel frequency cepstral feature (MFCC) is extracted; c. feature coding, the feature extracted by each segment in step b., and the feature extracted from the latter segment, the difference between each dimension is performed. Meta-encoding, if the difference between the two is greater than 0, the code is 1; otherwise, the code is 0, so if the multimedia segment is video, each segment will There will be 960 dimensions 0 or 1. If the multimedia clip is audio, each clip has 13 dimensions 0 or 1, hereinafter referred to as high dimensional encoding or feature encoding; and d. indexing. The method for detecting a recording medium according to the first aspect of the patent application, wherein the step d is to establish a multimedia index, that is, according to the feature encoding, the high-dimensional editing is performed. The code is grouped by the grouping algorithm and the group center is obtained, and the multimedia file number and the time segment number corresponding to each high-dimensional code are recorded in the index of the group center closest to the binary code. The method for detecting a recording medium according to the second aspect of the patent application, wherein the grouping algorithm is to use an unsupervised grouping method, and the number of the given group centers is a total number of multimedia numbers. The method for detecting a signal of a recording medium according to claim 2, wherein the calculation of the distance is based on a Hamming distance or an Euclidean distance. The method for detecting a recording medium according to the first aspect of the invention, wherein the recording medium index comparison method further comprises: a. timing division, dividing the multimedia file into a length of several seconds in time sequence. The multimedia segment, each segment is named as a time segment number, which is equal to the first few seconds in the original file; b. feature extraction, if the multimedia segment of step a. is video, then 960 dimensions are extracted. Scene-oriented feature (GIST); if it is audio, extract 13-dimensional Mel frequency cepstral feature (MFCC); c. feature coding, extract the feature extracted from each segment in b. Feature, the difference of each dimension is binary coded, if the difference between the two is greater than 0, the code is 1; if the code is 0, if the multimedia segment is video, then each The segment will have 960 dimensions 0 or 1. If the multimedia segment is audio, each segment has 13 dimensions 0 or 1, hereinafter referred to as high-dimensional coding or feature coding; d. index alignment. The method for detecting a recording medium according to claim 5, wherein the step d is to encode the feature of each segment, and compare with the group centers one by one to find the high dimension. The group center with the closest coding distance, and then the distance between the recorded media content feature and the feature of the time segment is individually determined for the corresponding multimedia file name and time segment included in the group center index, and the segment with the smallest distance is the most similar. Multimedia clips. The method for detecting a signal of a recording medium according to claim 6, wherein the calculation of the index by the step d is based on a Hamming distance or an Euclidean distance.