JP2012194653A - Data processing device, data processing system, data processing method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To quickly search for a musical piece by reducing a data amount of meta-data used for a music search.SOLUTION: A data processing device comprises: first and second measurement units 102A and 102B for measuring a fundamental frequency f(p) of audio data M1 and M2 of a musical piece at preset time intervals t; first and second calculation units 103A and 103B for, by using the fundamental frequency f(p), calculating a deviation qof the fundamental frequency f(p) from a reference frequency of standard sound as sound corresponding to the fundamental frequency f(p), respectively; and first and second meta-data generation units 104A and 104B for, by using the deviation q, generating meta-data MET1(t,q) and MET2(t,q) concerning features of the musical piece.

Description

  The present invention relates to a data processing device, a data processing system, a data processing method, and a program.

  In recent years, with the widespread use of personal computers (hereinafter referred to as “PCs”), multi-function mobile phones, and the like, most of music is converted into digital data. In that case, in order to facilitate the search and management of music, metadata describing the attributes of audio data such as music name and composer name is often generated.

  In addition, with the spread of electronic commerce using a network, a huge number of music pieces are provided by content providers. For example, when a personal user searches for a desired piece of music using a PC, some keyword such as a song name or a composer name is required. If these keywords are not known, it is difficult to find a desired music piece.

  In order to solve this problem, Patent Document 1 discloses that audio data is subjected to signal processing such as fast Fourier transform at predetermined time intervals, and feature values of pitches are extracted as metadata, so that sound (melody) is used as a keyword. A technique capable of searching for music is disclosed. This signal processing method is disclosed in cited document 2.

JP 2008-192102 A JP 2006-201614 A

  When searching for a desired piece of music from among the same and different recordings using sound as a keyword, the amount of metadata used for the search increases. Inevitably, the amount of calculation when matching the metadata of the search target and the search target increases, and the time required for searching for the music also increases.

  An object of the present invention is to provide a data processing device, a data processing system, a data processing method, and a program capable of reducing the amount of metadata used for searching and quickly searching for music.

  The data processing apparatus of the present invention uses a measurement unit that measures a fundamental frequency of audio data of a music for each predetermined time interval, and a reference frequency using the fundamental frequency obtained for each measurement by the measurement unit. A calculation unit that calculates a deviation of the fundamental frequency with respect to a reference frequency as a sound corresponding to the fundamental frequency, and metadata that generates metadata about the characteristics of the music using the deviation calculated by the calculation unit, respectively. And a generation unit.

  The data processing method of the present invention serves as a reference by using a measurement step for measuring the fundamental frequency of audio data of music at every predetermined time interval and the fundamental frequency obtained for each measurement in the measurement step. A calculation step for calculating a deviation of the fundamental frequency with respect to a reference frequency of the sound as a sound corresponding to the fundamental frequency, and a meta for generating metadata relating to the feature of the music using the deviation calculated in the calculation step. And a data generation step.

  The program of the present invention uses a measurement procedure for measuring a fundamental frequency of audio data of a music for each predetermined time interval, and a fundamental frequency obtained for each measurement in the measurement procedure. A calculation procedure for calculating a deviation of the fundamental frequency with respect to a reference frequency as a sound corresponding to the fundamental frequency, and a metadata generation for generating metadata relating to the feature of the music using the deviation calculated in the calculation procedure. And causing the computer to execute the procedure.

  ADVANTAGE OF THE INVENTION According to this invention, the data amount of the metadata used for a search can be reduced, and a music can be searched quickly.

FIG. 1 is a schematic block diagram showing a configuration example of a data processing system A according to an embodiment of the present invention. FIG. 2 is a block diagram showing a hardware configuration example of the server apparatus 1 according to the embodiment of the present invention. FIG. 3 is a block diagram showing functions of the server device 1 shown in FIG. FIG. 4 is a diagram for explaining calculation processing according to the embodiment of the present invention. FIG. 5 is a diagram for explaining a configuration of metadata obtained by the metadata generation processing according to the embodiment of the present invention. FIG. 6 is a sequence diagram showing an operation example of the data processing system A according to the embodiment of the present invention.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[Configuration example of data processing system A]
FIG. 1 is a schematic block diagram showing a configuration example of a data processing system A according to an embodiment of the present invention. As shown in FIG. 1, the data processing system A includes a server device 1, a music server device 2, a client terminal device 3, and a network 4 as data processing devices.

  In the data processing system A, the content provider who owns the server device 1 and the music server device 2 desires the individual user to the owner of the client terminal device 3 (hereinafter referred to as “individual user”) via the network 4. This system provides music.

  When providing the music, the data processing system A searches for the music (music data M1) desired by the individual user from the thousands and tens of thousands of music stored in the music server device 2. At that time, if the keyword of the word such as the music title is not known, the search becomes difficult. Therefore, the data processing system A can search not only words but also sounds as keywords by uploading the audio data M2 stored in the client terminal device 3 to the server device 1. In this embodiment, a case where sound is searched for as a keyword is taken as an example.

  In order to facilitate the search for music, the data processing system A has a music search mode and a similar search mode. Here, these two modes will be described.

(Music search mode)
The music search mode is a mode for searching for music desired by the individual user from among a number of music stored in the music server device 2 using the audio data M2 of the individual user.

(Similar search mode)
The similar search mode is a mode for searching for music similar to the melody of the audio data M2.

  For example, classical music is played and recorded by performers around the world. Like classical music, there are innumerable songs with the same music title but different performances and different recording environments. The similarity search mode is suitable for searching for music of the same recording.

  The server device 1 includes a computer that performs arithmetic processing according to a program. The server device 1 is connected to the network 4 and communicates with the music server device 2 and the client terminal device 3. The server device 1 performs the following processing inside the server device 1 in accordance with a request from the client terminal device 3.

(Music search process)
When the request for prompting the execution of the music search mode is received from the client terminal device 3, the server device 1 searches the music data M 1 from the music server device 2 on the network 4 and transmits the search result to the client terminal device 3. To do.

(Similarity search process)
When the request for prompting the execution of the similar search mode is received from the client terminal device 3, the server device 1 searches for music data similar to the audio data M <b> 2 (referred to as music data M <b> 1), and sends the search result to the client terminal device 3. Send.

  The music server device 2 stores a plurality of music data having various attributes, including music data M1 to be searched. The music server device 2 is connected to the network 4 and communicates with the server device 1 and the client terminal device 3.

  The client terminal device 3 is an electronic device such as a PC, a digital audio device, a PDA (Personal Data Assistance), and a multi-function / high-function mobile phone. The client terminal device 3 basically has the following functions.

  First, the client terminal device 3 can handle audio data. Secondly, the client terminal device 3 is connected to the network 4 regardless of wired or wireless, and communicates with the server device 1 and the music server device 2. Third, the client terminal device 3 displays characters and images on the display screen.

  In the present embodiment, a PC is taken as an example of the client terminal device 3 unless otherwise specified. The configuration of the client terminal device 3 is not particularly limited as long as it is an electronic device capable of handling audio data, connecting to the network 4, and displaying characters and images.

  Hereinafter, it is assumed that the music data M1 and the audio data M2 are digital audio data.

  The network 4 is the Internet that can use, for example, TCP / IP (Transmission Control Protocol / Internet Protocol) as a communication protocol.

[Hardware configuration example of server device 1]
FIG. 2 is a block diagram showing a hardware configuration example of the server apparatus 1 according to the embodiment of the present invention. As illustrated in FIG. 2, the server device 1 includes a CPU 10, a first HDD 11, a second HDD 12, a ROM 13, a RAM 14, an I / O (Input / Output) port 15, and an internal bus 16. Each component of the server device 1 is connected to the internal bus 16.

  The CPU 10 is, for example, a central processing unit. The CPU 10 performs music search processing and similarity search processing while exchanging data with the first and second HDDs 11 and 12 and the RAM 14 according to the program 131 stored in the ROM 13 via the internal bus 16.

  The first HDD 11 is a storage device, for example, a hard disk. Similarly, the second HDD 12 is a storage device, for example, a hard disk. The first HDD 11 mainly stores metadata used for music search processing and similarity search processing.

  The second HDD 12 mainly stores an index file used for music search processing and similarity search processing.

  The ROM 13 is a non-volatile storage device. The ROM 13 stores a program 131 necessary for the CPU 10 to execute a music search process and a similar search process.

  The RAM 14 is, for example, a DRAM (Dynamic Random Access Memory). The RAM 14 stores temporary data used by the CPU 10 during arithmetic processing.

  The I / O port 15 is a port for connecting an external device, and exchanges data between the external device and the CPU 10 via the internal bus 16. Examples of external devices that can be connected to the I / O port 15 include a network card and a keyboard. In the present embodiment, the server device 1 communicates with the music server device 2 and the client terminal device 3 on the network 4 via the I / O port 15 to which the network card is connected.

  When reading the program 131 from the ROM 13, the CPU 10 performs processing according to the processing procedure of the program 131 while accessing the first HDD 11, the second HDD 12, and the RAM 14. Thereby, each function which the server apparatus 1 has is implement | achieved (refer FIG. 3).

  The hardware configuration shown in FIG. 2 is an example, and various modifications can be made. For example, the program 131 can be stored in the ROM 13, the first HDD 11, or the second HDD 12. In addition, for example, various data used in the present embodiment can be stored in one hard disk.

[Functional Block Diagram of Server Device 1]
FIG. 3 is a block diagram showing functions of the server device 1 shown in FIG. As shown in FIG. 3, the server device 1 includes a first processing system 10A, a second processing system 10B, and a music database DB. Each process of the first and second processing systems 10A and 10B is executed by software by the CPU 10 (see FIG. 2).

  First, an outline of the first and second processing systems 10A and 10B will be described.

(First treatment system 10A)
The first processing system 10A includes a first file format conversion unit 101A, a first measurement unit 102A, a first calculation unit 103A, a first metadata generation unit 104A, a query generation unit 105A, a query execution unit 106, and a display processing unit 107. Have. Note that the query generation unit 105A and the query execution unit 106 are examples of a collation unit.

  The purpose of the first processing system 10A is to collate the audio data M2 of the individual user with the music database DB created by the second processing system 10B in order to perform music search processing and similarity search processing. The basic processing of the first processing system 10A is as follows.

(Measurement process)
First, the first processing system 10A inputs the audio data M2 from the client terminal device 3, and measures the fundamental frequency f (p) of the audio data M2 at predetermined time intervals. Here, the fundamental frequency is a physical quantity that determines the pitch (pitch) of a certain sound (for example, “A” in the name of a note (hereinafter referred to as American name)), and the frequency of the sound. It is the lowest frequency among the components. In the present embodiment, the time interval for measuring the fundamental frequency f (p) is approximately 100 ms.

(Calculation process)
Second, the first processing system 10A calculates the deviation of the fundamental frequency f (p) from the reference frequency f (A) of the reference sound (reference sound) in units of semitones, and the sound corresponding to this deviation is calculated. calculate. In other words, the first processing system 10A calculates a sound corresponding to the deviation of the fundamental frequency f (p) from the reference frequency f (A). Here, the reference sound is a sound corresponding to “A” when expressed by a pitch name (hereinafter, US name). The reference frequency f (A) of the sound is desirably 440 Hz.

  It is well known that music is composed of fundamental frequency components and various harmonic components. The fundamental frequency f (p) was measured by measuring the fundamental frequency f (p) and calculating how much the fundamental frequency f (p) is deviated from the reference frequency f (A) by the calculation process. The sound at the time (for example, “B” in the pitch name) is known. Details thereof will be described later.

(Metadata generation process)
Thirdly, the first processing system 10A generates metadata representing the characteristics of the audio data M2, specifically, the melody, based on the sound obtained by the calculation process.

(Music search process / similarity search process)
Fourth, the first processing system 10A uses the metadata to perform a search or similarity search of the music data M1 from the music database DB according to a processing request from the client terminal device 3, and transmits the result to the client terminal device 3. To do.

(Second processing system 10B)
The second processing system 10B includes a second file format conversion unit 101B, a second measurement unit 102B, a second calculation unit 103B, a second metadata generation unit 104B, a crawler unit 108, and a music database generation unit 109.

  The purpose of the second processing system 10B is to create a music database DB. The music database DB records information of music data in the music server device 2 including the music data M1 in order to perform music search processing and similarity search processing. The basic processing of the second processing system 10B is as follows.

(Collection processing)
First, the second processing system 10 </ b> B circulates the directory of the music server device 2 and collects various music data stored in the music server device 2.

(Database creation process)
Second, the second processing system 10B performs a measurement process, a calculation process, and a metadata generation process to create a music database DB.

  Hereinafter, the calculation process and the metadata generation process, which are features of the present embodiment, will be described in order.

[Sound calculation method in calculation processing]
FIG. 4 is a diagram for explaining calculation processing according to the embodiment of the present invention. In this embodiment, a chromatic scale is used as a musical scale expression method. The semitone in the twelve equal temperament is equal to one octave divided into 12 equal parts. One octave uses the notation of the American name, in order from the lowest pitch: “C”, “D ♭ (flat)”, “D”, “E ♭ (flat)”, “E”, “F ”,“ F # (sharp) ”,“ G ”,“ A ♭ (flat) ”,“ A ”,“ B ♭ (flat) ”,“ B ”.

  The frequency ratio of two adjacent semitones is expressed by equation (1).

  In the present embodiment, as described above, “A” is the reference sound, and the reference frequency is f (A) = 440 Hz. For example, the fundamental frequency f (B ♭) of the “B ♭” sound whose pitch is higher than the reference sound “A” by a semitone is expressed by the equation (2) according to the relationship of the equation (1).

From the equation (2), the frequency f (B ♭) is expressed by the equation (3).

  For example, the fundamental frequency f (B) of the “B” sound whose pitch is higher by a semitone than the “B 半” sound is expressed by the equation (4) by the same discussion as the calculation of the fundamental frequency f (B ♭).

  That is, the frequency f (B ♭) is expressed by the expression (5) when the expression (3) is used.

  FIG. 4 shows the fundamental frequency f (p) corresponding to each sound from the “C” sound to the “C” sound one octave higher than that. The variable p represents any sound from “C” to “B”. The higher the sound, the higher the fundamental frequency f (p).

  Ratio (frequency ratio) between the reference frequency f (A) of the reference sound “A” and the basic frequency f (B) of the “B” sound that is higher than the reference sound “A” and is shifted by two semitones from now on Is expressed by equation (6).

  FIG. 4 shows the ratio between the reference frequency f (A) and the fundamental frequency f (p) of each sound when the reference sound “A” is 1.

  Referring to the equation (6), the ratio between the reference frequency f (A) of the reference sound “A” and the fundamental frequency f (p) of the p-tone shifted by q semitones from the reference sound “A” is It can be expressed by the formula (7).

  In the equation (7), the variable q takes an integer. Hereinafter, the variable q is referred to as “the number of deviations”. When the equation (7) is modified, the relational equation (8) is obtained.

In equation (8), when taking the logarithm with the base being 2 ^1/12 , the deviation number q is expressed by equation (9).

  However, the deviation number q is a value rounded off so as to be an integer. In equation (9), since the reference frequency f (A) of the reference sound “A” is known, if the fundamental frequency f (p) is known, the number of deviations q can be obtained. Therefore, it can be understood how many halftones are deviated from the reference sound “A”.

  FIG. 4 shows the correspondence between the number of deviations q and each sound. When the fundamental frequency measured in the measurement process is f (p) = 466.16 Hz, the number of deviations is q = 1. This calculation result represents “B ♭” that is shifted from the reference sound “A” by the number of shifts q = 1 in semitones. Note that the pitch name is not affected by an octave and is given a certain notation (“C”, “D ♭”,... “B”), but in this embodiment, the pitch itself can be expressed. .

  Thus, by calculating how much the fundamental frequency f (p) is deviated from the reference frequency f (A), it can be understood which sound the measured fundamental frequency f (p) corresponds to.

[Metadata configuration example]
FIG. 5 is a diagram for explaining a configuration of metadata obtained by the metadata generation processing according to the embodiment of the present invention. In the measurement process by the first processing system 10A, the fundamental frequency f (p) of the audio data M2 is measured approximately every 100 ms from the start of the music. Since the measurement processing by the second processing system 10B is the same as that of the first processing system 10A, here, the processing of the first processing system 10A will be described as an example.

Since music is often composed of chords, a plurality of fundamental frequencies f (p) may be measured at one time. In the example shown in FIG. 5, two basic frequencies f (p ₁ ) = 293.7 Hz and f (p ₂ ) = 349.2 Hz are measured at time t _{1 when} 100 ms has elapsed from the start of the music.

As a result of the calculation processing, if the number of deviations at the fundamental frequency f (p ₁ ) is q ₁ = −7 at time t ₁ , it can be seen that the sound corresponding to the number of deviations q ₁ is “D”. Similarly, it can be seen that the sound corresponding to the number of deviations q ₂ at the fundamental frequency f (p ₂ ) is “F”.

In the metadata generation process, the first processing system 10A generates metadata in which the number of deviations q _n is described in chronological order together with the elapsed time t _n of the fundamental frequency f (p). The metadata is described in the following syntax 1 using the elapsed time t _n and the number of deviations q _n as parameters.

(Syntax 1)
(T _n ), q ₁ , q ₂ ,..., Q _n ,.

The variable n takes a positive integer (1, 2, 3,...). The elapsed time t _n is an elapsed time from the start of the music and is described in parentheses “()”. After the description of the elapsed time t _n, the number of deviations q ₁ is described using “, (comma)”. When the number of deviations q _n is described after the description of the number of deviations q _n−1 , the latter number of deviations q _n is described by “,” after the former description. After the last shift number q _n description, indicating the end of the metadata ", (comma)" is described.

For example, in the measurement at the elapsed time t ₁ = 100 ms, since the number of deviations q ₁ = −7, q ₂ = −4, metadata “(100), −7, −4,...” Is generated. If the number of deviations q _{n is associated} with the pitch name (specifically, the pitch) in advance, the music data at the elapsed time t _n = 100 ms is “D” sound and “F” sound only by referring to the metadata. It can be seen that it is constituted by.

In a music piece of about 15 minutes, the number of bytes of metadata obtained for each elapsed time t _n is about several bytes to 20 bytes, and on average about 15 bytes. Assuming that 15 bytes of metadata are obtained every 100 ms, the total number of bytes of metadata obtained from this music is approximately 15 bytes × (900000 ms / 100 ms) = 135 KB.

As described above, since the melody of the audio data M2 is expressed using the elapsed time t _n and the number of deviations q _n , the metadata data amount is only about 135 KB even for a music piece of about 15 minutes. The server device 1 can generate lightweight metadata while accurately representing the characteristics of the music.

(Each component of the first processing system 10A)
Next, each component of the first processing system 10A will be described with reference to FIG.

  The first file format conversion unit 101A inputs audio data M2 from the client terminal device 3 in time series via the network 4 (not shown), converts this into a predetermined file format, and converts the converted audio data M2c to the first 1 output to the measurement unit 102A. This is performed to unify the file format to be processed. In the present embodiment, for example, when the audio data M2 is compressed in the MP3 (MPEG Audio Layer3) file format, the first file format conversion unit 101A converts the audio data M2 into the MP4 file format.

The first measurement unit 102A takes the audio data M2c from the first file format conversion unit 101A in time series, performs measurement processing approximately every 100 ms, and outputs the obtained fundamental frequency f (p) to the first calculation unit 103A. To do. During the measurement process, the first measurement unit 102A converts the audio data M2c into frequency spectrum data using, for example, fast Fourier transform (FFT), and calculates the fundamental frequency f (p) about every 100 ms. Further, the first measurement unit 102A outputs the measurement time when measuring the fundamental frequency f (p) as the elapsed time t _n from the music start to the first meta-data generating unit 104A. In addition to the above, the fundamental frequency f (p) can be calculated using a harmonic clustering method, a comb filter method, or the like.

The first calculation unit 103A receives the fundamental frequency f (p) from the first measurement unit 102A, performs calculation processing, and outputs the calculated number of deviations q _n to the first metadata generation unit 104A. During the calculation process, the first calculation unit 103A substitutes the reference frequency f (A) of the reference sound “A” and the fundamental frequency f (p) measured by the first measurement unit 102A into the equation (9). Then, the shift number q _n is calculated as an integer.

The first metadata generation unit 104A inputs the elapsed time t _n from the first measurement unit 102A, and inputs the number of deviations q _n from the first calculation unit 103A, and performs metadata generation processing approximately every 100 ms to generate The meta data MET2 (t _n , q _n ) is output to the query generation unit 105A. During the metadata generation process, the first metadata generation unit 104A, as shown in the syntax 1, the metadata having the description “(t _n ), q ₁ , q ₂ ,..., Q _n ,”. Is generated.

In the present embodiment, in order to clarify the flow of data, the elapsed time t _n is input from the first measurement unit 102A to the first metadata generation unit 104A, but the first metadata generation unit 104A There is no problem even if the elapsed time t _n is directly grasped. The same applies to the second processing system 10B.

The query generation unit 105A inputs the metadata MET2 (t _n , q _n ) from the first metadata generation unit 104A, generates a query using the metadata MET2 (t _n , q _n ), and generates the generated query QUE Is output to the query execution unit 106. At that time, the query generation unit 105 </ b _{> A} also outputs metadata MET <b> 2 (t _n , q _n ) to the query execution unit 106.

  The query QUE is data necessary for executing the music search process or the similar search process, and describes a processing request for the music database DB. The processing request includes execution of the music search mode or the similar search mode received by the server device 1 from the client terminal device 3.

  The query execution unit 106 inputs the query QUE from the query generation unit 105A, and performs the following processing while accessing the music database DB according to the query QUE.

(When performing music search mode)
When executing the music search mode, the query execution unit 106 checks the metadata MET2 (t _n , q _n ) with the index file 121 stored in the second HDD 12, and displays the check result as the music search result ANS as the display processing unit 107. Output to.

For example, as illustrated in FIG. 5, the index file 121 records metadata MET1 (t _n , q _n ) generated by the second processing system 10B in chronological order. In the index file 121, metadata other than the music data M1 is also recorded.

In matching, the query execution unit 106 compares the audio metadata MET2 _{(t n, q} n) of data M2 metadata MET1 _{(t n, q} n) which is described with the index file 121 and.

  If the two match as a result of the comparison, the query execution unit 106 determines that the music data M1 is in the music server device 2, and stores that the music data M1 is found in the first HDD 11 as a music search result ANS. The attribute data D is output to the display processing unit 107. On the other hand, if the two do not match, the query execution unit 106 determines that the music data M1 is not in the music server device 2, and outputs that the music data M1 was not found to the display processing unit 107 as a music search result ANS.

(When similar search mode is executed)
When executing the similar search mode, the query execution unit 106 checks the metadata MET2 (t _n , q _n ) with the index file 121 and displays the check result as the search result ANS, as in the music search mode. It outputs to 107. However, the following points are different from those in the music search mode.

In matching, the query execution unit 106, the metadata MET2 _{(t n, q} n) of the audio data M2 is whether similar metadata MET1 described in the index file 121 _{(t n, q} n) Judging. In the present embodiment, when the former metadata MET2 (t _n , q _n ) matches the latter metadata MET 1 (t _n , q _n ) at a certain rate (for example, 80%), the query execution unit 106 Judge that both are similar. The ratio representing the degree of similarity may be suitably set by the owner of the server device 1 or an individual user.

  As a result of the comparison, if the two are similar, the query execution unit 106 estimates that the audio data M2 is a song similar to the song data M1, and displays the fact as a similarity search result ANS together with the attribute data D and the display processing unit 107. Output to. On the other hand, if the two are not similar, the query execution unit 106 determines that there is no music similar to the audio data M2 in the music server device 2, and outputs that fact to the display processing unit 107 as a similar search result ANS.

  The display processing unit 107 inputs the attribute data D from the query execution unit 106 together with the music search result ANS or the similar search result ANS, performs display processing for displaying both on the client terminal device 3, and displays the processing result R as the processing result R. The data is transmitted to the client terminal device 3 via the network 4. The display process is a process for determining the layout of the display screen, for example.

  In any mode, when a search target is found, the client terminal device 3 displays the attribute data D of the music data M1 on the display screen together with that fact.

(Each component of the second processing system 10B)
Each component of the second processing system 10B will be described with reference to FIG.

  As a collection process, the crawler unit 108 circulates a directory of the music server device 2 on the network 4 and collects cache data of content data stored in the music server device 2. The cache data includes various music data including the music data M1 and its attribute data. As described above, the attribute data is data such as a song name. The crawler unit 108 outputs the music data M1 of the cache data to the second file format conversion unit 101B, and outputs the attribute data D to the first HDD 11.

  The second file format conversion unit 101B receives the music data M1 from the crawler unit 108, performs the same processing as the first file format conversion unit 101A, and converts the music data M1c converted into a predetermined file format to the second measurement unit 102B. Output to.

The second measurement unit 102B takes the music data M1c from the second file format conversion unit 101B in time series, performs the same measurement process as the first measurement unit 102A, and uses the fundamental frequency f (p) obtained by this process. output to the second calculation unit 103B, and outputs the elapsed time _{t n} to the second metadata generating unit 104B.

The second calculation unit 103B receives the fundamental frequency f (p) from the second measurement unit 102B, performs a calculation process similar to that of the first calculation unit 103A, and calculates the calculated shift number q _n to the second metadata generation unit 104B. Output to.

The second metadata generation unit 104B inputs the elapsed time t _n from the second measurement unit 102B, and inputs the number of deviations q _n from the second calculation unit 103B. The same metadata as the first metadata generation unit 104A A generation process is performed, and the generated metadata MET ₁ (t _n , q _n ) is output to the first HDD 11.

The music database generation unit 109 accesses the music database DB and creates an index file 121 using the metadata MET1 (t _n , q _n ) stored in the first HDD 11.

(Music database DB)
The music database DB will be described. The music database DB is composed of various music data in the music server device 2 and an index file 121 created based on metadata of these music data.

The first HDD 11 stores metadata MET1 (t _n , q _n ) input from the second metadata generation unit 104B and attribute data D input from the crawler unit 108. The second HDD 12 stores an index file 121.

  When the attribute data D is not input from the crawler unit 108 to the first HDD 11 for some reason, such as when the crawler unit 108 fails in the collection process, for example, the attribute data D is manually input by the owner of the server device 1. Direct input can be made to the first HDD 11. Furthermore, the attribute data D can be changed manually.

[Operation example of data processing system A]
FIG. 6 is a sequence diagram showing an operation example of the data processing system A according to the embodiment of the present invention.

  Here, the case where the individual user who is the owner of the client terminal device 3 searches the music server device 2 on the network 4 for the same music as the audio data M2 is taken as an example. Even in the similar search process, the operation of the data processing system A is the same.

  When a request for urging execution of the music search mode is received from the client terminal device 3, the server device 1 performs the following processing when the request is permitted. First, the flow of processing by the first processing system 10A shown in steps S1 to S5 will be described.

  When the audio data M2 is input from the client terminal device 3 in time series, the first file format conversion unit 101A converts the audio data M2 into a predetermined file format (step S1).

  After the conversion of the file format, the first measurement unit 102A detects the start portion of the music in the audio data M2c. Then, the first measurement unit 102A performs a measurement process approximately every 100 ms to obtain a fundamental frequency f (p) (step S2).

After the measurement process, the first calculation unit 103A performs a calculation process using the fundamental frequency f (p) to obtain the number of deviations q _n (step S3).

After the calculation process, the first metadata generation unit 104A performs a metadata generation process about every 100 ms using the elapsed time t _n from the start of the music and the number of deviations q _n, and generates metadata MET2 (t _n , q _n ) is obtained (step S4).

The metadata generation process / query generation unit 105A generates a query using the metadata MET2 (t _n , q _n ) to obtain a query QUE (step S5).

  Next, the process flow of the second processing system 10B shown in steps S6 to S11 will be described.

  The crawler unit 108 collects cache data of content data stored in the music server device 2 and obtains music data M1 and attribute data D (step S6).

  After the collection process, the second file format conversion unit 101B converts the music data M1 input in time series from the crawler unit 108 into a predetermined file format (step S7).

  After the file format conversion, the second measurement unit 102B performs a measurement process to obtain the fundamental frequency f (p) (step S8).

After the measurement process, the second calculation unit 103B performs a calculation process using the fundamental frequency f (p) to obtain the number of deviations q _n (step S9).

After the calculation process, the second metadata generation unit 104B performs the metadata generation process about every 100 ms using the elapsed time t _n from the start of the music and the number of deviations q _n, and generates metadata MET1 (t _n , q _n ) is obtained (step S10).

Metadata MET1 _{(t n, q} n) after is generated, the music database generation unit 109, the metadata MET1 _{(t n, q} n) is used to create an index file 121 (step S11).

  The music database DB is created by the processing of steps S6 to S11 described above. This series of processes is preferably completed before the process of step S12 is started. Further, this series of processes may be executed periodically (for example, every 24 hours).

  Next, the flow of processing by the first processing system 10A shown in steps S12 and S13 will be described.

The query execution unit 106 collates the metadata MET2 (t _n , q _n ) of the audio data M2 with the index file 121 (step S12).

  When the display processing unit 107 obtains the search result, the display processing unit 107 performs display processing based on the search result, and transmits the processing result R to the client terminal device 3 (step S13).

  When the client terminal device 3 receives the processing result R from the server device 1, the client terminal device 3 displays a search result indicating whether or not the music data M1 has been found on the display screen.

  Note that the processing of steps S1 to S13 is described as a processing procedure in the program 131 (see FIG. 2).

As described above, according to the present embodiment, the server device 1 generates metadata using the elapsed time t _n and the number of deviations q _n , so that the following remarkable effects can be obtained.

  As described above, the metadata is about 135 KB even for a music piece of about 15 minutes and is very lightweight. Therefore, the amount of calculation when the query execution unit 106 collates the metadata can be significantly reduced. This leads to a reduction in the use capacity of the database and a reduction in the time required for the music search process and the similar search process.

  Furthermore, according to the present embodiment, the following remarkable effects can be obtained.

  You can search accurately regardless of language (Japanese, English, etc.). Like classical music, music with the same recordings often have the same attributes, such as music name, and are described in a variety of languages. In the first place, the description method of attributes such as music titles is not unified, and the attributes can be changed by a third party. Therefore, it is difficult to search for a song from songs of the same recording using words such as song names. However, in the present embodiment, since no word keyword is described in the metadata, the search can be performed using only the melody.

  When the search target is found, the search target attribute is transmitted to the client terminal device 3 together with the search result. Therefore, even if the individual user does not know the name of the music to be searched, if the search target is found, the individual user can immediately know the name of the music.

  In the calculation process, attention is paid to the fundamental frequency f (p), which calculates the deviation from the reference frequency f (A). It is possible to generate metadata that accurately represents.

  In addition, if the data processing system A is used, it becomes possible to search for illegal music such that attribute data is intentionally rewritten by a third party and uploaded to another server or the like.

(Modification of the embodiment)
A modification of the embodiment will be described. This modification relates to a search for music with a key change, such as shifting the pitch from “C” sound to “E” sound. Hereinafter, differences from the embodiment will be described.

In a similar search mode, the query execution unit 106, the metadata MET2 _{(t n, q} n) when it is determined whether or not similar to the metadata _{MET1 (t n, q n)} , the following key change music search Process.

Specifically, the query execution unit 106, in all of the elapsed time t _n, if the shift number q _n between the two is offset a predetermined number q _w, a key change between the audio data M2 and the music data M1 Judge. The certain number q _w can be suitably set, for example, q _w = 3. The query execution unit 106 outputs to the display processing unit 107 together with the attribute data D as a similarity search result ANS that there is a key change. The processing when the shift number q _n between the two do not deviate a predetermined number q _w is the same as the embodiment.

  The key change music search process described above can be added to the music search mode and the similar search mode as the key change music search mode.

  According to this modification, it is possible to easily search for a musical piece or an original musical piece performed by changing the musical instrument. Of course, the effects of the embodiment can be obtained.

  The present invention can be variously modified without departing from the gist thereof. For example, the server device 1 and the music server device 2 are connected to a home LAN (Local Area Network), and the music search mode, the similar search mode, and the key change music search mode are executed in the home using the client terminal device 3. be able to.

  For example, the functions of the music server device 2 can be incorporated into the server device 1 and both can be made into one server device.

DESCRIPTION OF SYMBOLS 1 ... Server apparatus 2 ... Music server apparatus 3 ... Client terminal apparatus 4 ... Network 10 ... CPU
10A ... 1st processing system 10B ... 2nd processing system 11 ... 1st HDD
12 ... Second HDD
13 ... ROM
14 ... RAM
DESCRIPTION OF SYMBOLS 15 ... I / O port 16 ... Internal bus 101A ... 1st file format conversion part 101B ... 2nd file format conversion part 102A ... 1st measurement part 102B ... 2nd measurement part 103A ... 1st calculation part 103B ... 2nd calculation part 104A ... First metadata generation unit 104B ... Second metadata generation unit 105A ... Query generation unit 106 ... Query execution unit 107 ... Display processing unit 108 ... Crawler unit 109 ... Music database generation unit 121 ... Index file 131 ... Program

Claims (8)

A measurement unit that measures the fundamental frequency of the audio data of the music at predetermined time intervals;
A calculation unit that calculates a deviation of the fundamental frequency from a reference frequency of a reference sound as a sound corresponding to the fundamental frequency, using the fundamental frequency obtained for each measurement of the measurement unit;
A metadata generation unit that generates metadata related to the characteristics of the music using the deviations calculated by the calculation unit;
A data processing apparatus. The metadata generation unit
Associating each deviation with the elapsed time from the start of the music, and generating the metadata,
The data processing apparatus according to claim 1. The calculation unit includes:
When the reference frequency is denoted by f (A), the fundamental frequency is denoted by f (p), and the deviation of the fundamental frequency (p) with respect to the reference frequency f (A) is denoted by q, the deviation q Is calculated using the following formula:
The data processing apparatus according to claim 1 or 2.

A collation unit that collates the metadata generated by the metadata generation unit with a database in which attributes of a plurality of music pieces including the music piece are registered;
The collation unit
Outputting the attribute of the music together with the matching result;
The data processing apparatus according to any one of claims 1 to 3. The collation unit
If the metadata matches the metadata in the database at a certain rate, it is determined that the metadata is similar to the metadata in the database, and the attribute of the music is output together with a matching result.
The data processing apparatus according to claim 4. A data processing device according to any one of claims 1 to 5;
A terminal device for transmitting audio data to the data processing device;
A data processing system. A measurement step for measuring the fundamental frequency of the audio data of the music at predetermined time intervals;
Using the fundamental frequency obtained for each measurement in the measurement step, a calculation step for calculating a deviation of the fundamental frequency relative to a reference frequency of a reference sound as a sound corresponding to the fundamental frequency;
A metadata generation step for generating metadata relating to the characteristics of the music using the deviations calculated in the calculation step;
A data processing method. A measurement procedure for measuring the fundamental frequency of the audio data of a song at predetermined time intervals;
Using the fundamental frequency obtained for each measurement in the measurement procedure, a calculation procedure for calculating a deviation of the fundamental frequency with respect to a reference frequency of a reference sound as a sound corresponding to the fundamental frequency,
A metadata generation procedure for generating metadata related to the characteristics of the music using the deviations calculated in the calculation procedure,
A program that causes a computer to execute.

Images