JP2009276776A - Music piece identification device and its method, music piece identification and distribution device and its method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To be able to easily identify a music piece name of a music piece by use of one segment of the music piece and also to distribute the music piece. <P>SOLUTION: Music segment data 41 for one part of a music piece for a few second is transmitted to a power spectrum analysis part 43 and converted into a time frequency distribution. A feature matrix generation part 44 executes resampling of the time frequency distribution data from the power spectrum analysis part 43 at a prescribed frequency and time interval, and generates a music segment feature matrix (feature vector) S<SB>fu</SB>. The data of the music segment feature matrix S<SB>fu</SB>is transmitted to a matching part 45 and compared with a plurality of music feature matrixes A<SB>ft</SB>obtained from a database 35 to calculate the degree of similarity, so as to identify a music piece name to be a candidate out of music pieces registered already. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a music identification apparatus and method for electronic music distribution, for example, a music identification distribution apparatus and method, and in particular, a music identification apparatus and method for music distribution using a mobile phone, and a music identification distribution apparatus. And a method.

  In recent years, a huge number of new songs have been announced, and it is impossible to know all of them, including various songs created in the past. Also, these various songs are often played in television and radio programs, commercial messages, and on the streets, so people listen to various songs regardless of their intentions. There are many opportunities to make.

  As described above, when trying to obtain a favorite music from among the music overflowing in the surrounding area, the purchaser generally takes a procedure such as the following order (1) to (4).

  (1) Listen to music.

  (2) Check the song name.

  (3) Go to a music store.

  (4) Designate a song name and purchase a song.

  Patent documents 1-6 are known as conventional technology.

Japanese Patent Application Laid-Open No. 07-121556 Japanese Patent Laid-Open No. 02-033200 JP 09-293083 A JP 2000-172893 A JP 2000-187671 A Japanese Patent Laid-Open No. 06-069999

  However, in many cases, a great deal of labor is required for the investigation of the music name in (2) above. In addition, although you may know the song name at the same time as knowing the song, for example, if you listen to only a part of the song, a song you hear in a broadcast program or commercial message, or a song you hear when you go out, that song In many cases, the name cannot be identified, and eventually it cannot be obtained. On the other hand, even if the name of the music is known as a result of the survey, in order to actually obtain the music, it is necessary to go to a store or use electronic music distribution. When using electronic music distribution, it is not necessary to go to the store, but operate a personal computer, connect to the Internet, register the payment method to the distributor, download music, etc. Multi-step action is required.

  Therefore, the present invention has been made in view of such circumstances, and for example, even if a piece of music is being played, played back, broadcasted, or a partial fragment thereof, the piece of music can be easily identified and obtained (for example, It is an object of the present invention to provide a music identification method and apparatus, a music identification distribution apparatus and method, which can be purchased), and can also be used for payment of the music, reproduction of music, and the like.

  In order to solve the above-described problems, a music identification device according to the present invention includes a music fragment feature amount extraction unit that extracts music fragment feature amounts for identifying candidate song names from music fragment data of music data, And identifying means for identifying a candidate song name from already registered songs based on the music fragment feature value extracted by the music fragment feature value extracting means.

  The music identification method according to the present invention includes a music fragment feature extraction step for extracting a music fragment feature for identifying a candidate song name from music fragment data of music data, and the music fragment feature extraction step. And an identification step of identifying candidate song names from among the already registered songs based on the music fragment feature values extracted by the above.

  In addition, the music identification and distribution device according to the present invention includes a recording means for recording music data, a playback means for playing music from the music data, and a candidate from music fragment data of the music data via a mobile phone operation system. A music fragment feature quantity extracting means for extracting a music fragment feature quantity for identifying a song name, and the music piece feature quantity extracted by the music fragment feature quantity extraction means Among the above, an identification means for identifying a candidate music name, and a transmission means for transmitting the identified music based on the identification means.

  In addition, the music identification and distribution method according to the present invention includes a recording step of recording music data, a playback step of playing music from the music data, and candidates from music fragment data of the music data via a mobile phone operation system. A music fragment feature extraction step for extracting a music fragment feature for identifying the music name to be and a music fragment feature extracted in the music fragment feature extraction step, Among them, the method includes an identification step for identifying a candidate song name, and a transmission step for transmitting the song identified by the identification step.

  According to the present invention, for example, a part of a piece of music that is played, played, broadcast, etc., by identifying candidate music names from among already registered music pieces based on the music fragment feature amount Even if it is a fragment, the music can be easily identified. In addition, the identified music can be distributed via the mobile phone operation system.

It is a block diagram which shows schematic structure of the transmission part of the mobile telephone terminal which concerns on embodiment of this invention. It is a block diagram which shows schematic structure of the receiving part of the mobile telephone terminal which concerns on embodiment of this invention. It is a block diagram which shows schematic structure of the system for registering a music in a database. It is a figure used for description of a feature matrix and a mask pattern. It is a block diagram which shows schematic structure of the music identification system for identifying a music from a music fragment. It is a flowchart which shows the flow of a matching method. 1 is a system configuration diagram showing an overall configuration of an electronic music identification distribution system. It is a flowchart which shows the flow of a music registration method. It is a flowchart which shows the flow of a music identification method. It is a flowchart which shows the flow of the music trial listening method. It is a flowchart which shows the flow of the music purchase method.

  Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings.

  First, a portable terminal will be described as the first invention, then a music identification method will be described as the second invention, and an electronic music identification distribution service system will be described as the third invention.

  FIG. 1 shows an overview of a transmission unit of a mobile phone terminal as an embodiment to which the mobile terminal of the first invention is applied, and FIG. 2 shows an overview of a reception unit.

  The transmission unit of the mobile phone terminal of the present embodiment has substantially the same configuration as that of a general mobile phone device. As shown in FIG. 1, the microphone 1 as an acoustoelectric conversion means, “0” to “9” 10 keys, a call start key, a call end key, various keys such as a power on / off key, and selection means for selecting various menus, icons, applications, and the like (including a jog dial having a push switch function, for example) A key input unit 2; an A / D converter 4 that converts an analog sound signal that has been subjected to acoustoelectric conversion by the microphone 1 into a digital sound signal; and a voice code that encodes a digital sound signal (for example, a voice signal during a call) 6, a multiplexer 8 that multiplexes signals from the key input unit 2 and the speech encoder 6, and a modulation that modulates a high frequency signal for transmission It is equipped with a 9, and the like. Since these components are the same as those mounted on the conventional mobile phone device, detailed description of these operations is omitted here.

  In the transmission unit of the mobile phone terminal according to the present embodiment, unique parts different from general mobile phone devices are a recording button 3, a switch 5, and a music data processing unit 7, which will be described in detail below. To do.

  The recording button 3 is, for example, an ON / OFF button, and the ON / OFF signal is a directivity switching control signal for the microphone 1 and a switching control signal for the switch 5.

  When the recording button 3 is turned off (OFF), the directivity of the microphone 1 is set to a short distance for telephone use, and the switch 5 is set to the voice encoder 6 side. Therefore, when the recording button 3 is turned off, the acoustic signal input through the microphone 1 (in this case, the voice signal at the time of a call) is digitized by the A / D converter 4, and the voice code The data is encoded by the encoder 6 to be used as voice data (digital bit stream data) used for telephone call transmission, and is further modulated to a high frequency signal by the modulator 9 and then transmitted to the operation system of the mobile phone company. The Also, during this call, for example, when the key input unit 2 is operated by an operator and a number or an operation command is input from the key input unit 2, the key input signal is sent to the audio data by the multiplexer 6. And further modulated to a high-frequency signal by the modulator 9, and then transmitted to the operation system of the mobile phone company. The above is exactly the same as the basic operation of a conventional mobile phone.

  On the other hand, when the recording button 3 is turned on (ON), the directivity of the microphone 1 is set to a long distance for music recording, and the switch 5 at this time is set to the music data processing unit 7 side. As a result, the acoustic signal (in this case, a music signal) input through the microphone 1 is digitized by the A / D converter 4 and compression encoded or feature extracted by the music data processing unit 7 as described later. After that, it is converted into a bit stream, sent to the temporary storage unit 10, and accumulated for at least a predetermined time (for example, several seconds). A bit stream related to music data for a predetermined time accumulated in the temporary storage unit 10 is read out in response to a read instruction signal from the key input unit 2 automatically or by an operator's operation and sent to the multiplexer 8. It is done. The multiplexer 8 receives a request signal, which will be described later, output from the key input unit 2 according to the operation of the operator, and receives a bit stream related to the music data for the predetermined time and a signal from the key input unit 2. Are multiplexed and sent to the modulator 9. The modulator 9 modulates the high frequency signal and transmits it to the mobile phone operation system in the same manner as in the above call.

  The directivity of the microphone 1 may be adjusted by changing the directivity of one microphone or by switching between a plurality of microphones having different directivities. The temporary storage unit 10 may be newly provided as a large-capacity memory different from a memory included in a general mobile phone device, or when the memory provided in the general mobile phone device is a large-capacity memory May use the memory as it is. The temporary storage unit 10 can be shared with the memory 18 of the receiving unit described later, or may be removable like a semiconductor memory card. When the removable device such as a semiconductor memory card is used as the temporary storage unit 10, it is acquired not only by the music captured by the microphone 1 but also by various devices such as another mobile phone terminal and a portable information processing device. As for the music, the music name can be identified and the music can be purchased as described later.

  The music data processing unit 7 uses the music name of the music name from the short-time music data (hereinafter referred to as a music fragment or music clip) of the predetermined time stored in the temporary storage unit 10 (hereinafter referred to as recording as appropriate). Coded data required for identification is generated, or features to be described later required for music name identification are extracted.

  In other words, in the present embodiment, the music data processing unit 7 can take any of the following first and second configurations.

  The configuration of the first music data processing unit 7 is a configuration in which an audio compression encoder such as so-called MPEG audio is used as it is. In this case, the music piece recorded in the temporary storage unit 10 is compressed and encoded as it is by the audio compression encoder and transmitted to the music identification system described later. Conversion to a later-described special amount necessary for name identification is performed. According to this example, since only a general encoder needs to be installed in the mobile phone terminal of the present embodiment, the cost of the mobile phone terminal can be reduced. At the same time, in this example, feature extraction to be described later for music name identification is performed on the music identification system side, so a highly accurate identification method that requires a large amount of computation for the feature extraction can be introduced. In addition, songs that cannot be automatically identified can be heard and confirmed by people on the music identification system side, and when a new identification algorithm is developed, the old identification algorithm is replaced with the new identification algorithm. There are advantages such as easy replacement.

  The configuration of the second music data processing unit 7 is a configuration in which the music data processing unit 7 performs feature extraction using a feature extraction method described later, and encodes the feature amount information (for example, feature vectors). In the case of this example, the feature amount information obtained by the music data processing unit 7 has a much smaller data amount than the compression-coded music data as in the first configuration. Therefore, there is an advantage that the storage capacity (memory capacity) of the temporary storage unit 10, the transmission capacity when transmitting the data, the transmission time, and the like can be saved.

  Next, the receiving unit of the mobile phone terminal according to the present embodiment shown in FIG. 2 has a configuration substantially similar to that of the conventional mobile phone device, and demodulates the high frequency signal received via the antenna. A demultiplexer 12 for separating the multiplexed data, a speech decoder 15 for decoding the speech data for speech separated by the demultiplexer 12, and the speech digital speech data obtained by the above decoding. A D / A converter 16 for converting into an analog voice signal, a speaker 17 for emitting a speech analog voice signal from the D / A converter 16, and numbers and characters included in the control data separated by the demultiplexer 12 A display processing unit 13 that generates a display signal from data, a delay 14 including a liquid crystal display device, and the like are provided. Since these components are the same as those mounted on the conventional mobile phone device, detailed description of these operations is omitted here.

  On the other hand, in the receiving unit of the mobile phone terminal of the present embodiment, a unique part different from a general mobile phone device includes a memory 18, an audio decoder 19, a D / A converter as a configuration of a music signal processing system. 20 and the speaker 21 are provided, and their detailed operations will be described below.

  In the receiving unit shown in FIG. 2, when the received data demodulated by the demodulator 11 includes music data according to the present embodiment, which is different from voice data for speech, the demultiplexer 12 Separates the music data from the received data and sends it to the memory 18. The memory 18 temporarily stores the music data. Note that the memory 18 may be a built-in mobile phone terminal or a removable one such as a semiconductor memory card. If a removable memory such as a semiconductor memory card is used as the memory 18, the acquired music data can be conveniently reproduced on other devices.

  After all or sufficient data of the music is received, the music data is read from the memory 18, sent to the audio decoder 19, decoded, and further passed through the D / A converter 20 to the speaker 21 or headphones. Is output as sound. The D / A converter 20 and the speaker 21 may be shared with the D / A converter 16 and the speaker 17 of the audio signal processing system in order to reduce the number of components. Since the required sound quality is generally different from that of a music signal (for example, high quality for music), it is desirable to provide them separately according to the required sound quality.

  Next, a method for identifying a song name as the second invention will be described below.

  The music identification method according to the present invention can be broadly divided into a music registration process, which is a previous process of music identification, and an actual music identification process using the registered music database, as described below. This is done in two streams.

  FIG. 3 shows a system configuration for registration processing of the music in the database in the music identification method according to the present invention.

  In FIG. 3, a plurality of pieces of music data 31 distributed via, for example, a so-called compact disc (CD) or a network is sent to the power spectrum analysis unit 33. The power spectrum analysis unit 33 converts each of the plurality of input music data 31 into a time frequency distribution, and sends the obtained time frequency distribution data to the feature matrix generation unit 34.

The feature matrix generation unit 34 resamples the time-frequency distribution data corresponding to each song at a predetermined frequency and time interval, thereby generating a song feature matrix (feature vector) A _ft corresponding to each song. However, as shown in FIG. 4A, “f” of the music feature matrix A _ft represents an index (row) in the frequency direction of the matrix, and “t” represents an index (column) in the time direction. The data of the music feature matrix A _ft corresponding to each music is associated with the attribute data 32 of each music such as the music name, performer name, music ID number of each music distributed together with the music data 31. Are registered in the database 35.

  Next, FIG. 5 shows a system configuration for music identification using the registered music database.

In FIG. 5, a piece of music (the music fragment) data 41 of, for example, about several seconds obtained by the mobile phone terminal according to the first invention described above or any other method has the same power as in FIG. It is sent to the spectrum analysis unit 43 and converted into a time frequency distribution. Also, the feature matrix generation unit 44 similar to that in FIG. 3 resamples the time frequency distribution data from the power spectrum analysis unit 43 at a predetermined frequency and time interval to generate a music fragment feature matrix (feature vector) _Sfu . To do. However, as shown in FIG. 4B, “f” of the music fragment feature matrix S _fu is an index (row) in the frequency direction of the same matrix as described above, and “u” is an index (column) in the time direction. To express. The data of the music fragment feature matrix _Sfu is sent to the matching unit 45. Note that when the mobile phone terminal performs feature value extraction, the same configuration as the power spectrum analysis unit 43 and the feature matrix generation unit 44 in FIG. It is provided in the voice data processing unit 7 of the telephone terminal. On the other hand, when the mobile phone terminal only performs compression encoding of music data and does not extract features, the mobile phone terminal stores the music fragment data 41 supplied to the power spectrum analysis unit 43 of FIG. Thus, data obtained by decompressing and decoding data that has been compression-encoded and transmitted is used.

The matching unit 45 includes a song feature matrix A _ft for a plurality of songs registered in the database 35 and attribute data 32 of each song such as a corresponding song name, performer name, and song ID number. Data 42 is also input. The matching unit 45 uses a plurality of music feature matrices A _ft, that is, music feature vectors, obtained from the database 35, and the music fragment feature matrix S _fu that is supplied from the feature matrix generation unit 44, that is, music fragment feature vectors. Then, matching (vector matching) is performed by a method described later, the degree of similarity between these vectors is calculated, and when the degree of similarity between the music fragment feature vector and the music feature vector exceeds a predetermined threshold, the music feature For the music corresponding to the vector, the degree of similarity Q, the time when the degree of similarity becomes maximum (maximum degree of similarity) T, and data 46 consisting of attribute data 32 are output.

  Here, the matching in the matching unit 45 is performed as follows using the cross-correlation.

As shown in FIG. 4, the music feature matrix A _ft is a feature matrix for all of a plurality of music pieces, and the music fragment feature matrix S _fu is a (deteriorated) feature matrix of a part of a music piece. In principle, the correlation value at the time when the cross-correlation between these matrices becomes maximum may be used as the similarity, but since noise is usually added to the music fragment feature matrix _Sfu , the following operation is performed. To improve noise immunity.

First, the music fragment feature matrix S _fu is converted by the following equation (1).

S ′ _fu = M _fu (S _fu −B _fu ) (1)
However, B _fu in the equation is a constant matrix for reducing stationary noise components, and is created by, for example, the minimum value in each frequency component. M _fu in the equation is a matrix that masks a part of the music fragment feature matrix S _fu (the part indicated by m in each figure) as shown in FIGS. 4C to 4E. . The mask matrix M _fu shown in FIG. 4C is a matrix for masking low frequency components. According to this, when low frequency noise such as traffic noise is strong, the low frequency noise is converted into music fragment features. This is effective for removing from the matrix _Sfu . Also, the mask matrix M _fu shown in FIG. 4D is a matrix for performing time masking. By using this time mask, for example, only the time when the music component is strong is extracted from the music fragment feature matrix S _fu. Matching processing can be realized. Also, a mask matrix M _fu shown in FIG. 4E is a matrix for masking speech components, and a frequency range (for example, 100 Hz to 1 kHz) containing the most speech components is removed from the music fragment feature matrix S _fu. Thus, for example, when there is audio mixing, the audio component can be removed. Various other mask patterns are conceivable, but stable matching with high noise resistance is performed by switching these masks and selecting the one with the highest similarity.

  Next, using the transformation of the above equation (1), the similarity at each time is calculated by the following equation (2).

  Further, the similarity Q and similarity maximum time T between the music piece and the music fragment are obtained by the maximum similarity among the times and the time as shown in the equations (3) and (4).

Q = max _t R (t) (3)
T = argmax _t R (t) (4)
The matching unit 45 implements the matching method described above in the flow shown in FIG.

  In FIG. 6, the matching unit 45 is prepared in advance with various mask patterns that may be necessary, such as the low-frequency component mask, the time mask, and the voice component mask described above. As a result, one of the mask patterns is selected.

  Next, in step S52, the matching unit 45 determines whether or not all mask patterns have been processed. If it is determined that all mask patterns have not yet been processed, the process proceeds to step S53. If it is determined that the processing of all the prepared mask patterns has been completed, the process proceeds to step S55.

  If it is determined in step S52 that the processing of all the mask patterns has not been completed yet and the process proceeds to step S53, the matching unit 45 calculates a correlation function R (t) using the mask pattern selected in step S51. In the next step S54, the similarity Q and the maximum similarity time T are calculated, and the obtained values are stored. Thereafter, the process returns to step S51, and a mask pattern is selected again.

  On the other hand, in step S52, it is determined that the processing of all the prepared mask patterns has been completed, and when the processing proceeds to step S55, the matching unit 45 inputs the music piece having the largest similarity score Q. The degree of similarity to the music fragment that was made.

  Next, an electronic music identification distribution service method and system thereof according to a third invention using the mobile phone terminal according to the first invention and the music identification system according to the second invention will be described below. . The advantage of this system is that the music is investigated, purchased, played back, paid, etc., from a single mobile terminal (the mobile phone terminal of the present embodiment).

  FIG. 7 shows a system configuration of an embodiment to which the electronic music identification distribution service according to the third aspect of the invention is applied, and an operation method of the system will be described with reference to FIGS. In addition, in the instruction codes S71, S72, S75, S76, S78, S79, S82 to S85, and S90 to S99 in FIG. 7, the processing of the steps represented by the corresponding instruction codes in FIGS. It is shown that. FIG. 8 shows a flow of processing at the music registration stage performed in advance, FIG. 9 shows a flow of processing at the music identification stage, FIG. 10 shows a flow of processing at the music trial listening stage, and FIG. The flow of processing at the music purchase stage is shown.

  In the music registration stage shown in FIG. 8, first, in step S71, the music distribution system 64 on the music seller side transmits the data (the plurality of music data and attribute data thereof) necessary for music identification to the music identification person on the music identification side. The music identification system 63, which has been sent to the system 63 and received the data, obtains a feature matrix from the music data for music identification as described above and registers it in the database in association with the attribute data. In step S72, the music distribution system 64 sends data necessary for the trial listening of the music (a plurality of music data for trial listening and IDs of the music data for each trial listening) to the music identification system 63, and the music identification system 63 In 63, the music preview data is registered in the database. Note that the processing of steps S71 and S72 may be performed simultaneously, or the processing of step S72 may be performed after the processing of step S71.

  Next, in the music identification stage shown in FIG. 9, first, as step S73, when the recording button 3 is turned on by the operator of the music purchaser, the mobile phone terminal 61 on the music purchaser side is turned on. For example, a part (music fragment) of a piece of music played in a broadcast program or a commercial message or being played on the street is recorded. In step S74, the cellular phone terminal 61 uses the music data processing unit 7 to perform compression encoding or feature extraction on the music fragment data as described above.

  Next, in step S75, the mobile phone terminal 61 extracts the feature extraction request together with the music identification request generated in response to the key input from the key input unit 2 to the mobile phone operating system 62 on the mobile phone operator side. Further, in step S76, the mobile phone operation system 62 transmits the music identification request and the feature-extracted data to the music identification system 63 on the music identifier side as it is.

  The music identification system 63 that has received the music identification request and the compression-encoded data or feature-extracted data, as step S77, performs music identification processing using feature vectors as described in the second invention ( Search for candidate music). When the music data processing unit 7 on the mobile phone terminal 61 side performs processing from recording of music pieces to feature extraction, the music identification system 63 uses the feature vector sent from the mobile phone terminal 61. Music identification is performed by the used matching. When the music data processing unit 7 on the mobile phone terminal 61 side only performs recording from recording of music pieces to compression encoding processing of music data, the music identification system 63 sends it from the mobile phone terminal 61. After the compression-coded music data is decoded, the above-described feature extraction is performed, and music identification is further performed by matching. When the music identification process ends, the music identification system 63 carries data (music name, performer name, music ID number, etc.) relating to the candidate music obtained by the music identification process as step S78. The mobile phone operation system 62 transmits the data about the candidate music to the mobile phone terminal 61 as it is, in step S79.

  In the mobile phone terminal 61 that has received the data about the candidate music, in step S80, the operator (music purchaser) of the terminal instructs the key input unit 2 as to whether or not to listen to the candidate music. It is determined whether or not a predetermined input operation has been performed. In this step S80, when the operator inputs an instruction to audition the candidate music (when auditioning (Y)), the process proceeds to the music audition stage shown in FIG. When the instruction input not to audition is made (when not auditioning (N)), the process ends.

  When the process proceeds to the music trial listening stage shown in FIG. 10, first, in step S81, in the mobile phone terminal 61, a predetermined operator for selecting a music for trial listening to the key input unit 2 by the terminal operator (music purchaser). When the input operation is performed, the music for trial listening according to the input operation is selected.

  Next, in step S82, the mobile phone terminal 61 transmits a music preview request generated in response to the key input from the key input unit 2 to the mobile phone operation system 62, and further the mobile phone operation system. 62 transmits the music preview request as it is to the music identification system 63 as step S83.

  The music identification system 63 that has received the music trial listening request transmits the data for trial listening corresponding to the music trial listening request to the mobile phone operation system 62 in step S84, and the mobile phone operation system 62 further executes step S85. The trial listening data is transmitted to the mobile phone terminal 61 as it is. At this time, the cellular phone terminal 61 stores the trial listening data in the memory 18.

  Next, in step S86, in the mobile phone terminal 61 that has received the provision of the data for trial listening, the operator (music purchaser) of the terminal trials the music for trial listening to the key input unit 2. Is sent to the audio decoder 19 in response to the input operation, and the sample music data is decoded, and further, D The signal is sent to the speaker 21 via the / A converter 20. Thereby, the operator (music purchaser) of the mobile phone terminal 61 can listen to the trial music.

  Thereafter, in step S87, the mobile phone terminal 61 performs a predetermined input operation for instructing the key input unit 2 whether to listen to another piece of music by the operator (music purchaser) of the terminal. It is determined whether or not In this step S87, when the operator inputs an instruction to audition another song (in the case of audition (Y)), the process returns to the process of selecting the audition song in step S81, while the operator makes another If the instruction input not to listen to the song is performed (when not listening (N)), the process proceeds to step S88.

  When the processing proceeds to step S88, the mobile phone terminal 61 performs a predetermined input operation for instructing the key input unit 2 whether or not it is necessary to purchase the sampled music by the operator (music purchaser). A determination is made as to whether it has been made. In this step S88, when the operator inputs an instruction to purchase the sample music (Y), the process proceeds to the music purchase stage shown in FIG. 11, while the operator does not purchase the sample music. If the instruction input is made (N), the process is terminated.

  When the process proceeds to the music purchase stage shown in FIG. 11, first, in step S89, in the mobile phone terminal 61, the terminal operator (music purchaser) performs a predetermined input operation for selecting the purchased music on the key input unit 2. If it is made, music selection is performed according to the input operation.

  Next, in step S90, the mobile phone terminal 61 transmits a music purchase request indicating the purchase intention of the selected music to the mobile phone operation system 62. Further, in step S91, the mobile phone operation system 62 The music purchase request is transmitted to the music distribution system 64 on the music seller side as it is.

  The music sales system 64 that has received the music purchase request transmits the music data corresponding to the music purchase request to the mobile phone operation system 62 as step S92. Further, the mobile phone operation system 62 receives the music purchase request as step S93. The music data is transmitted to the mobile phone terminal 61 as it is. Note that the music data is stored in the memory 18 in the mobile phone terminal 61 at this time. Thereafter, when the terminal operator (music purchaser) performs a predetermined input operation for reproducing the music on the key input unit 2, the mobile phone terminal 61 responds to the input operation with the memory 18. Is sent to the audio decoder 19, the music data is decoded, and further sent to the speaker 21 via the D / A converter 20. Thereby, the operator (music purchaser) of the mobile phone terminal 61 can listen to the purchased music.

  Here, when music identification and distribution are performed as described above, the music purchaser pays the music identification person or music distributor directly or by bank transfer or credit card via the Internet. Although it is possible to pay by settlement, etc., in the electronic music identification distribution system according to the embodiment of the present invention, the payment is made through the mobile phone operator in order to collect the payment.

  First, as step S94, the music identification system 63 on the music identifier side sends the above-described charge request information for identifying the music to the mobile phone operation system 62 on the telephone operator side. The telephone operator of the mobile phone operation system 62 at this time pays the music identifier to the music identifier on behalf of the music purchaser in step S95. In addition, as a payment method in this case, for example, electronic payment processing is performed such that the price is transferred from the bank account of the telephone operator to the bank account of the music identifier, and information indicating that the payment has been performed, You may make it send to the music identification system 63 by the music identification person side from the mobile telephone operation system 62 or a bank.

  Similarly, in step S96, the music distribution system 64 on the music seller side sends the billing information for the music distributed as described above to the mobile phone operation system 62 on the telephone operator side. At this time, the telephone operator of the mobile phone operation system 62 pays the music distributor to the music distributor on behalf of the music purchaser in step S97. In addition, as a payment method in this case, for example, an electronic payment process is performed in which the price is transferred from the bank account of the telephone operator to the bank account of the music distributor, and information indicating that the payment has been performed, You may make it send to the music distribution system 64 by the music distributor side from the mobile telephone operation system 62 or a bank.

  Next, as step S98, the mobile phone operation system 62 adds the amount paid on behalf of the music identifier and music distributor to the call charge and the like, and the user of the mobile phone terminal 61 ( To the music purchaser). Thereafter, as step S99, the user (music purchaser) of the mobile phone terminal 61 gives the telephone operator the telephone charge to which the music identification fee and the music distribution fee have been added, directly or by bank transfer, Pay by lump sum or by credit card payment via the Internet. In addition, as for the payment of the call charge between the user (music purchaser) of the mobile phone terminal 61 and the telephone operator, for example, from the bank account of the user (music purchaser) of the mobile phone terminal 61 as described above. Electronic payment processing is performed in which the money is transferred to the bank account of the telephone operator, and information indicating that the payment has been made is sent from the mobile phone terminal 61 or the bank to the mobile phone operating system 62 on the telephone operator side. It is also possible to send it.

  This completes the payment settlement process.

  In the present embodiment, for the sake of convenience of explanation, the music identifier and the music seller have been described as separate items. Of course, the same thing can be realized even if the music seller himself performs the identification service. In addition, for convenience of explanation, it has been described that the music data for trial listening is provided by the identifier, but it is clear that there is no substantial difference in this part even if performed by the seller.

  Furthermore, in this embodiment, after receiving the distribution of the music data, the identification price and the music purchase price are paid, but only the music identification is performed and the music purchase is not performed. It is possible to charge only the price for music identification, and it is also possible to provide a free service for music identification.

  In the embodiment of the present invention, a mobile phone terminal is taken as an example. However, the present invention can be applied to a portable information processing terminal such as a so-called palmtop computer. However, in the present invention, as described above, the recording of music, the identification of music, the distribution of music, the purchase, the reproduction, etc. are all realized in one mobile phone terminal, thereby eliminating the need for a portable information processing terminal, A so-called portable audio recording / reproducing apparatus is also unnecessary.

  As described above, according to the embodiment of the present invention, it is possible to search for music candidates from a part of music that is played by street broadcasting or the like such as television broadcasting or radio broadcasting. , Listening to music names, purchasing music, and playing music can be performed with a single mobile phone terminal. That is, according to the embodiment of the present invention, it is possible to easily investigate music flowing through street broadcasting or the like such as television broadcasting or radio broadcasting, and the music can be easily obtained. Giving up purchases without being identified can be reduced, and the time and labor required for purchases can be significantly reduced.

  As is clear from the above description, according to the present invention, in the mobile phone terminal, the music data taken in via the acoustoelectric conversion means is recorded, and the recorded music data is compressed and encoded, or The feature quantity for identifying the music title of the music is extracted from the recorded music data and sent to the music identification / distribution side. The music identification / distribution side extracts the feature quantity from the compression-coded or extracted feature quantity. Search for candidates, send trial listening data of the candidate songs to the mobile phone terminal, send the songs that have been purchased as a result of the trial listening on the mobile phone terminal from the song identification / distribution side to the mobile phone terminal, For example, even if it is a partial piece of music that is being played, played, or broadcast, for example, the music can be easily identified and identified by charging on the telephone operation side according to the music that was auditioned and purchased. Available (for example purchase) is possible, also, payment of the price of the music, it is also possible at the same time playback, and the like of the music.

  DESCRIPTION OF SYMBOLS 1 Microphone, 2 Key input part, 3 Recording button, 4 A / D converter, 5 Switch, 6 Voice encoder, 7 Music data processing part, 8 Multiplexer, 9 Modulator, 10 Temporary storage part, 11 Demodulator , 12 Demultiplexer, 13 Display processing unit, 14 Display, 15 Audio decoder, 16, 20 D / A converter, 17, 21 Speaker, 18 Memory, 19 Audio decoder, 33, 43 Power spectrum analysis Unit, 34, 44 feature matrix generation unit, 35 database, 45 matching unit, 61 mobile phone terminal, 62 mobile phone operation system, 63 music identification system, 64 music distribution system

Claims (10)

Music fragment feature quantity extracting means for extracting music piece feature quantities for identifying candidate song names from music fragment data of music data;
A music identification apparatus comprising: identification means for identifying a candidate music name from among already registered music based on the music fragment feature value extracted by the music fragment feature value extraction means.   2. The music identification apparatus according to claim 1, wherein the music fragment feature amount extraction unit converts the music fragment data into a time-frequency distribution and extracts a music fragment feature amount. It has a music database that stores and manages the already registered music,
The music identification device according to claim 2, wherein the music database stores music feature quantities extracted from a time frequency distribution of a plurality of music. The music fragment feature amount extraction means extracts the music fragment feature matrix corresponding to the music fragment data as the music fragment feature amount by resampling the data of the time frequency distribution at a predetermined frequency and time interval,
4. The music identification apparatus according to claim 3, wherein the music database stores a music feature matrix obtained by resampling data of time frequency distribution of a plurality of music at a predetermined frequency and time interval as the music feature amount. . The identification means includes feature matrix conversion means for converting the music fragment feature matrix into an identification feature matrix,
The feature matrix conversion means performs at least one of a process of subtracting a constant matrix for reducing a stationary noise component and a process of masking the music fragment feature matrix with a mask matrix. 5. The music identification apparatus according to claim 4, wherein a plurality of masks including at least one of a matrix for masking low frequency components, a matrix for performing time masking, and a matrix for masking audio components are used by switching.   The identification means calculates the similarity with the feature matrix for identification obtained from the feature matrix conversion means while shifting the time with respect to the music feature matrix of each music registered in the music database, The music identification device according to claim 5, wherein the similarity between the music and the music fragment and the maximum time of similarity are obtained from the maximum similarity at each time and the time.   5. The music identification apparatus according to claim 4, wherein the music feature matrix data is registered in the music database in association with the music name, performer name, and music ID number of the music from which the music feature matrix is extracted. . A music fragment feature extraction step for extracting a music fragment feature for identifying a candidate song name from the music fragment data of the music data;
A music identification method comprising: an identification step of identifying a candidate song name from already registered songs based on the music fragment feature value extracted by the music fragment feature value extraction step. Recording means for recording music data;
Playback means for playing music from the music data;
Music fragment feature amount extracting means for extracting music fragment feature amounts for identifying candidate song names from the music fragment data of the music data via the mobile phone operation system;
Based on the music fragment feature quantity extracted by the music fragment feature quantity extraction means, an identification means for identifying a candidate song name from among already registered songs;
A music identification distribution device comprising: transmission means for transmitting the identified music based on the identification means. A recording process for recording music data;
A playback process for playing music from the music data;
A music fragment feature extraction step for extracting music fragment feature values for identifying candidate song names from the music fragment data of the music data via the mobile phone operation system;
Based on the music fragment feature value extracted by the music fragment feature value extraction step, an identification step for identifying a candidate song name from among already registered songs;
A music identification distribution method comprising: a transmission step of transmitting the music identified by the identification step.

Images