JP2006318384A - Musical piece retrieval system and musical piece retrieval method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a musical piece retrieval system and a musical piece retrieval method which allow musical piece data to be retrieved with a high precision by a mobile terminal device on the basis of impression degree data determined by human senses. <P>SOLUTION: In a musical piece registration device 10, a feature data extraction part 13 extracts physical feature data from inputted musical piece data, and an impression degree data conversion part 14 converts feature data to impression degree data determined by human senses, and a transmission and reception part 21 transfers musical piece data stored in a musical piece database 15 and impression degree data to a mobile terminal device 30. In the mobile terminal device 30, a transmission and reception part 31 stores the transferred musical piece data and impression degree data in a terminal musical piece database 32, and a terminal operation part 34 accepts an input as a retrieval condition of impression degree data. and a musical piece retrieval part 33 retrieves the terminal musical piece database 32 on the basis of inputted impression degree data to specify a prescribed number of musical piece data. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a music registration apparatus that stores input music data in a music database, and a portable terminal that is configured to be connectable to the music registration apparatus and that has a terminal music database to which music data stored in the music database is transferred. More particularly, the present invention relates to a music search system and a music search method for searching music data stored in a terminal music database in a mobile terminal device.

  In recent years, a large-capacity storage means such as an HDD has been developed, and a large amount of music data can be stored in the large-capacity storage means. Searching for a large amount of music data stored in a large-capacity storage means is generally performed using bibliographic data such as artist names, music titles, and other keywords. This is not suitable when it is not possible to take into account the emotions of a music piece and there is a possibility that a music piece with a different impression will be searched, and it is desired to search for a music piece with a similar impression when listening.

  Therefore, in order to make it possible to search for the music desired by the user based on the subjective impression of the music, the user's subjective requirements for the music desired to be searched are input, quantified and output, and the output , A predicted impression value obtained by quantifying the impression of the music to be searched is calculated, and a music database storing an acoustic value of a plurality of music and an impression value obtained by quantifying the impression of the music is calculated using the calculated predicted impression value as a key. There has been proposed an apparatus for searching for desired music based on a subjective image of a user's music by searching (for example, see Patent Document 1).

However, in the prior art, since the impression value obtained by converting the physical characteristics of the music is searched based on the predicted impression value obtained by quantifying the input of the user's subjective requirements, the search condition is set by the user as a search condition. The input items of the subjective requirements that are input are aggregated, and it is not possible to realize a search for music data with high accuracy based on the subjective requirements, and the possibility of actually listening to the music on the terminal device, There was a problem that only search by bibliographic data could be performed.
JP 2002-278547 A

  The present invention has been made in view of such problems, and an object of the present invention is to perform a search for music data with high accuracy based on impression degree data determined by human sensitivity with a mobile terminal device. A music search system and a music search method are provided.

In order to solve the above problems, the present invention has the following configurations.
The music search system of the present invention includes a music registration device that stores input music data in a music database, and a terminal that is connectable to the music registration device and to which the music data stored in the music database is transferred. A music search system comprising a portable terminal device having a music database, wherein the music registration device uses a feature data extraction means for extracting physical feature data from the inputted music data, and the feature data extraction means. Impression degree data converting means for converting the extracted feature data into impression degree data determined by human sensitivity, and storing the converted impression degree data in the music database together with the music data; and the music database The stored music data and impression data are stored in the mobile terminal device. Music data transmission means for sending, and the portable terminal device receives the music data for storing the music data and the impression degree data transferred from the music data transmission means of the music registration device in the terminal music database. Means, terminal operation means for receiving input as a search condition for the impression degree data, and searching the terminal music database based on the impression degree data input as a search condition from the terminal operation means to obtain a predetermined number of the It is characterized by comprising music search means for specifying music data and audio output means for outputting the music data searched by the music search means.

  Furthermore, in the music search system of the present invention, the music search means uses the impression degree data input as a search condition from the terminal operation means as an input vector and the impression degree stored in the terminal music database. Data is used as a search target vector, and a predetermined number of searches are performed in ascending order of Euclidean distance between the two.

  The music search system of the present invention is configured to be connected to a music registration device that stores input music data in a music database, and to the music registration device, and the music data stored in the music database is transferred to the music search system. A music search system comprising a mobile terminal device having a terminal music database, wherein the music registration device extracts physical feature data from the input music data, and the feature data extraction Impression degree data conversion means for converting the feature data extracted by the means into impression degree data determined by human sensibility, and input based on the impression degree data converted to the impression degree data conversion means The music data is mapped to a music map divided into a plurality of partial spaces with coordinates, and The music mapping means for creating a search table comprising the coordinates of the space and the music data arranged in the partial space, and the search created by the music mapping means together with the music data stored in the music database Music data transmission means for transferring a table to the portable terminal device, and the portable terminal device stores the music data and the search table transferred from the music data transmission means of the music registration device in the terminal music database. Music data receiving means to be stored in the terminal, terminal operation means for receiving an instruction input for representative song search specifying a representative song, and searching the search table based on the representative song specified by the terminal operation means to obtain a predetermined number Music search means for specifying the music data, and the music search means The music data, characterized by comprising an audio output means for audio output.

  Furthermore, in the music search system of the present invention, the music map is a self-organizing map that has been learned, and the plurality of partial spaces are a plurality of neurons each having a different feature vector, The mapping means uses the impression degree data converted by the impression degree data conversion means as an input vector, and is arranged in the neuron to which the feature vector having the shortest distance from the input vector is assigned. To do.

  Furthermore, in the music search system of the present invention, the impression degree data conversion means is determined by human sensitivity using a hierarchical neural network in which the feature data extracted by the feature data extraction means has been previously learned. It is characterized by converting to impression degree data.

  The music search method of the present invention is configured to be connected to a music registration device that stores input music data in a music database, and to the music registration device, and the music data stored in the music database is transferred. A music search method in a music search system comprising a mobile terminal device having a terminal music database, wherein the music registration device extracts physical feature data from the input music data, and the extracted features The data is converted into impression degree data determined by human sensitivity, the converted impression degree data is stored in the music database together with the music data, and the music data and the impression data stored in the music database are stored. Transferred to the portable terminal device, and the portable terminal device is transferred from the music registration device. The music data and the impression data are stored in the terminal music database, an input as a search condition for the impression data is accepted, and the terminal music database is searched based on the impression data received as a search condition. A predetermined number of the music data is specified, and the searched music data is output as a voice.

  Furthermore, the music search method of the present invention is configured to be connected to a music registration device that stores input music data in a music database, and to the music registration device, and the music data stored in the music database is transferred. A music search method in a music search system comprising a mobile terminal device having a terminal music database, wherein the music registration device extracts physical feature data from the input music data, and the extracted features Data is converted into impression degree data determined by human sensitivity, and the partial space is mapped to a music map divided into a plurality of partial spaces to which coordinates are assigned based on the converted impression degree data. A search table including the coordinates of the music data and the music data arranged in the partial space is created. The created search table is transferred to the portable terminal device together with the song data stored in the song, and the portable terminal device stores the song data and the search table transferred from the song registration device in the terminal song database. And receiving an instruction input for representative song search specifying the representative song, searching the search table based on the received representative song, specifying a predetermined number of the song data, and searching the song data Is output by voice.

  According to the music search system and the music search method of the present invention, in the music registration device, physical feature data is extracted from the song data input by the feature data extraction means, and the feature data is converted into human sensitivity by the impression degree data conversion means. Is converted into impression degree data determined by the data, and the converted impression degree data is stored in the music database together with the music data, and the music data and the impression degree data stored in the music database are transferred to the portable terminal device by the music data transmitting means. In the portable terminal device, the song data and the impression degree data transferred from the song data transmission means of the song registration apparatus by the song data receiving means are stored in the terminal song database, and the impression data is stored by the terminal operation means. Accepts input as a search condition and serves as a music search means The terminal music database is searched based on the impression degree data input as the search condition, the predetermined number of music data is specified, and the music data searched by the audio output means is configured to be output by voice. Music data can be searched using impression degree data converted from characteristic data of music data by a music registration device in a portable terminal device, so music data with high accuracy based on impression degree data determined by human sensitivity There is an effect that the search can be performed by the mobile terminal device.

  Furthermore, in the music search system and the music search method of the present invention, in the music registration device, physical feature data is extracted from the music data input by the feature data extraction means, and the feature data is converted into human data by the impression degree data conversion means. Is converted into impression degree data determined by the sensitivity of the music, and the music data is mapped to the music map divided into a plurality of partial spaces to which the coordinates are given by the music mapping means based on the impression degree data, and the coordinates of the partial space And a song table arranged in the partial space is created, and the song table is transferred to the portable terminal device together with the song data stored in the song database by the song data transmitting means. In the music data receiving means, the music data transmitting means of the music registration device The music data and the search table transferred from the terminal are stored in the terminal music database, the instruction input of the representative music search specifying the representative music is received by the terminal operation means, and the search table is searched based on the representative music by the music search means. By specifying a predetermined number of music data and outputting the music data retrieved by the audio output means as audio, the impression data converted from the characteristic data of the music data by the music registration device in the portable terminal device is used. Therefore, it is possible to perform a search based on the music map mapped in the above manner, so that it is possible to perform a search for music data with high accuracy based on impression degree data determined based on human sensitivity.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a block diagram showing a configuration of an embodiment of a music search system according to the present invention, FIG. 2 is a front view showing a configuration of the mobile terminal device shown in FIG. 1, and FIG. It is a block diagram which shows the structure of the neural network learning apparatus which learns in advance the neural network used for the music registration apparatus shown in FIG.

  In the present embodiment, referring to FIG. 1, the music registration device 10 and the mobile terminal device 30 are connected by a data transmission path 23 such as a USB, and the mobile terminal device 30 is separated from the music registration device 10. It is configured to be portable.

  The music registration apparatus 10 is an information processing apparatus that operates under program control such as a personal computer. Referring to FIG. 1, a music data input unit 11, a compression processing unit 12, a feature data extraction unit 13, and impression degree data. Conversion unit 14, music database 15, music mapping unit 16, music map storage unit 17, music search unit 18, PC operation unit 19, PC display unit 20, transmission / reception unit 21, and audio output unit 22 It consists of.

  The music data input unit 11 has a function of reading a storage medium in which music data such as a CD and a DVD is stored. The music data input unit 11 inputs music data from a storage medium such as a CD and a DVD, and extracts a compression processing unit 12 and feature data. To the unit 13. You may comprise so that the music data (delivery data) via networks, such as the internet, other than storage media, such as CD and DVD, may be input. When compressed music data is input, the compressed music data is decompressed and output to the feature data extraction unit 13.

  The compression processing unit 12 compresses the music data input from the music data input unit 11 in a compression format such as MP3 or ATRAC (Adaptive Transform Acoustic Coding) at the time of music registration, and the compressed music data includes an artist name, a music title, and the like. Are stored in the music database 15 together with the bibliographic data.

  The feature data extraction unit 13 extracts feature data composed of fluctuation information from the music data input from the music data input unit 11 and outputs the extracted feature data to the impression degree data conversion unit 14.

  The impression degree data conversion unit 14 converts the feature data input from the feature data extraction unit 13 into impression degree data determined by human sensitivity using a hierarchical neural network that has been learned in advance. The impression level data is output to the music mapping unit 16, and the feature data input from the feature data extraction unit 13 and the converted impression level data are associated with the music data and registered in the music database 15.

  The music database 15 is a large-capacity storage unit such as an HDD, and the music data and bibliographic data compressed by the compression processing unit 12, the feature data extracted by the feature data extraction unit 13, and the impression degree data conversion unit 14. Is stored in association with the impression degree data converted by.

  The music mapping unit 16 maps the music data to a music map that is an arbitrary space based on the impression data input from the impression data conversion unit 14, and the music map in which the music data is mapped is stored in the music map storage unit 17. A search table for searching for music data mapped to the music map is created and stored, and the created search table is stored in the music database 15.

  The music map storage unit 17 is a large-capacity storage unit such as an HDD, and stores a music map to which music data is mapped by the music mapping unit 16.

  The music search unit 18 searches the music database 15 based on the bibliographic data from the PC operation unit 19, displays the search result on the PC display unit 20, and plays music based on the representative music selected by the PC operation unit 19. The map storage unit 17 is searched, and the representative song search result is displayed on the PC display unit 20 and stored in the song database 15 as a search list.

  The PC operation unit 19 is input means such as a keyboard and a mouse, and inputs search conditions for searching for music data stored in the music database 15 and the music map storage unit 17 and music data to be output to the mobile terminal device 30. An input to select is made.

  The PC display unit 20 is a display unit such as a liquid crystal display, for example, displays the mapping status of the song data stored in the song map storage unit 17, and the song data stored in the song database 15 and the song map storage unit 17. The search conditions for searching for music, the searched music data (search results), etc. are displayed.

  The transmission / reception unit 21 is configured to be connectable to the transmission / reception unit 31 of the portable terminal device 30 by a data transmission path 23 such as a USB, and corresponds to the music data stored in the music database 15 and the music data. The neuron coordinates in the musical score map, the search table, and the search list are output to the transmission / reception unit 31 of the mobile terminal device 30.

  The audio output unit 22 is an audio player that decompresses and reproduces music data stored in the music database 15.

  The portable terminal device 30 is an audio playback device such as a portable audio device having a large capacity storage means such as an HDD. Referring to FIG. 1, a transmission / reception unit 31, a terminal music database 32, a music search unit 33, and a terminal An operation unit 34, a terminal display unit 35, and an audio output unit 36 are included.

  The transmission / reception unit 31 is configured to be connectable to the transmission / reception unit 21 of the music registration device 10 via a data transmission path 23 such as a USB, and the music data input from the transmission / reception unit 21 of the music registration device 10 is used as the terminal music. Store in the database 32.

  The terminal song database 32 is a large-capacity storage unit such as an HDD, and the song data stored in the song database 15, impression degree data corresponding to the song data, the coordinates of neurons in the song map, a search table, The search list is stored.

  The music search unit 33 searches the terminal music database 32 based on the bibliographic data from the terminal operation unit 34, displays the search result on the terminal display unit 35, and based on the representative music specified by the terminal operation unit 34. The search table is searched, and the search result is displayed on the terminal display unit 35.

  The terminal operation unit 34 is an input unit for performing input related to the reproduction of music data. Referring to FIG. 2, input for selecting music data to be reproduced, input for selecting a search method, input for volume control, and the like are performed. It consists of a cross key 341 and an enter key 342 for performing an input for instructing output of the selected music data, an input for instructing execution of the selected search method, and the like.

  The terminal display unit 35 is a display unit such as a liquid crystal display, for example, and displays search conditions for searching for music data stored in the terminal music database 32 and displays searched music data (search results). .

  The audio output unit 36 is an audio player that decompresses and reproduces music data that is compressed and stored in the terminal music database 32.

  The neural network learning device 40 is a device that learns the hierarchical neural network used in the impression degree data conversion unit 14 and the music map used in the music mapping unit 16. Referring to FIG. 3, the music data input unit 41, voice output unit 42, feature data extraction unit 43, impression degree data input unit 44, combination weight value learning unit 45, music map learning unit 46, combination weight value output unit 47, and feature vector output Part 48.

  The music data input unit 41 has a function of reading a storage medium in which music data such as a CD and a DVD is stored. The music data input unit 41 inputs music data from a storage medium such as a CD and a DVD, and extracts an audio output unit 42 and feature data. To the unit 43. You may comprise so that the music data (delivery data) via networks, such as the internet, other than storage media, such as CD and DVD, may be input. When compressed music data is input, the compressed music data is decompressed and output to the audio output unit 42 and the feature data extraction unit 43.

  The audio output unit 42 is an audio player that decompresses and reproduces music data input from the music data input unit 41.

  The feature data extraction unit 43 extracts feature data composed of fluctuation information from the song data input from the song data input unit 41 and outputs the extracted feature data to the connection weight value learning unit 45.

  The impression degree data input unit 44 accepts input of impression degree data by the evaluator based on the audio output from the audio output unit 42, and uses the received impression degree data as a joint signal as a teacher signal used for learning of the hierarchical neural network. The value is output to the value learning unit 45 and is also output to the music map learning unit 46 as an input vector to the self-organizing map (SOM).

  The connection weight value learning unit 45 performs learning on the hierarchical neural network based on the feature data input from the feature data extraction unit 43 and the impression degree data input from the impression degree data input unit 44, and The connection weight value is updated, and the updated connection weight value is output via the connection weight value output unit 47. The learned hierarchical neural network (updated connection weight value) is transplanted to the impression degree data conversion unit 14 of the music registration device 10.

  The music map learning unit 46 learns the self-organizing map using the impression degree data input from the impression degree data input unit 44 as an input vector to the self-organizing map, updates the feature vector of each neuron, and The updated feature vector is output via the output unit 48. The learned self-organizing map (updated feature vector) is stored in the music map storage unit 17 of the music registration device 10 as a music map.

First, the music registration operation in the music registration apparatus 10 will be described in detail with reference to FIGS.
4 is a flowchart for explaining the music registration operation in the music registration apparatus shown in FIG. 1, and FIG. 5 is a flowchart for explaining the feature data extraction operation in the feature data extraction unit shown in FIG. FIG. 6 is a flowchart for explaining the tempo determination operation in the feature data extraction unit shown in FIG. 1, and FIG. 7 is an explanatory diagram showing an example of a hierarchical neural network used in the impression degree data conversion unit shown in FIG. 8 is an explanatory diagram illustrating an example of a music map in which the music mapping unit illustrated in FIG. 1 maps music data, and FIG. 9 illustrates an example of a search table stored in the music database illustrated in FIG. It is.

  A storage medium storing music data such as CD and DVD is set in the music data input section 11 and music data is input from the music data input section 11 (step A1).

  The compression processing unit 12 compresses the music data input from the music data input unit 11 (step A2), and stores the compressed music data in the music database 15 together with the bibliographic data such as artist name and music name (step A3). .

  The feature data extraction unit 13 extracts feature data including fluctuation information from the music data input from the music data input unit 11 (step A4).

  With reference to FIGS. 4 to 6, the feature data extraction operation in the feature data extraction unit 13 is performed when music data is input to the music data input unit 11 from a music playback device such as a CD player or a network such as the Internet (step). B1) The music data input unit 11 downsamples the music data input to the music data input unit 11 from 44.1 kHz to 22.05 kHz for the purpose of speeding up, and the downsampled music data is a feature data extraction unit. 13 is output.

  Next, the feature data extraction unit 13 performs FFT processing for a certain frame length from a predetermined data analysis start point (30 s from the beginning of the song) of the song data, and calculates a power spectrum (step B2). ). In the present embodiment, the tempo period is extracted as the tempo of the music, which is one of the feature data, and the sampling period is set to 22 .2 assuming that the tempo period is in the range of 0.3 to 1 s. It was configured to perform 1024-point FFT processing on music data at 05 kHz. That is, the frame length for performing the FFT processing is set to 1024 / 22.05 kHz≈46 ms, which is shorter than the minimum value of the tempo cycle in the assumed music.

  Next, the feature data extraction unit 13 sets frequency bands of Low (0 to 200 Hz), Middle (200 to 600 Hz), and High (600 to 11050 Hz) in advance, and the power of the three bands of Low, Middle, and High. The spectrum is integrated, the average power is calculated (step B3), and it is determined whether or not the number of frames for which the processing operations of steps B2 to B3 have been performed has reached a predetermined set value (2048) (step B4). ) If the number of frames that have undergone the processing operations in steps B2 to B3 has not reached a predetermined set value, the data analysis start point is shifted (step B5) while the processing in steps B2 to B3 is performed. Repeat the operation. As a result, the processing operations of step B2 to step B3 are performed for a set value of a predetermined number of frames, and time series data of average power in the Low, Middle, and High3 bands can be respectively obtained. In this embodiment, the analysis time length is 60 s, the data analysis start point is shifted by 60 s * 22.05 kHz / 2048≈646 points, and FFT processing is performed to obtain time series data of 2048 points and 60 s average power. Configured to create.

  Next, the feature data extraction unit 13 performs FFT on the time series data of the average power of Low, Middle, and High calculated by the processing operations of Step B2 to Step B5, and calculates fluctuation information (Step B6). In this embodiment, 2048-point FFT processing is performed on time series data of average power.

  Next, the feature data extraction unit 13 calculates an approximate straight line in a graph with the logarithmic frequency on the horizontal axis and the logarithmic power spectrum on the vertical axis from the FFT analysis results in Low, Middle, and High by the least square method or the like (step B7 ), The inclination of the approximate straight line and the Y intercept of the approximate straight line are obtained (step B8), and the slope and Y intercept of the approximate straight line in each of Low, Middle, and High are extracted as feature data.

  Further, the feature data extraction unit 13 calculates, as difference information, a difference between the fluctuation information calculated in Step B6 and the approximate straight line calculated in Step B7 in the FFT analysis result in Low (Step B9). By analyzing the difference information calculated based on the obtained logic, the tempo cycle is determined (step B10), the determined tempo is extracted as feature data, and the slope of the approximate straight line in each of Low, Middle, and High and The result is output to the impression degree data converter 14 together with the Y intercept.

  Note that the tempo determination operation in the feature data extraction unit 13 is, as shown in FIG. 6, first separated from the approximate straight line by 1.25 dB or more in the period 0.3 to 2 s by analyzing the difference information calculated in step B9. It is determined whether there is a frequency component that is more than 1.25 dB away from the approximate line within the frequency range of 1 / 0.3 to 1/2 Hz (step C1). If there is a frequency component separated from the straight line by 1.25 dB or more, the frequency component having the longest period among the frequency components separated from the approximate line by 1.25 dB or more is set as a candidate A (step C2).

  Next, the feature data extraction unit 13 determines whether or not the candidate A is 1 s or more (step C3). If the candidate A is 1 s or more, the candidate A / 2 is determined as the tempo period ( Step C4) If the candidate A is not 1 s or longer, the candidate A is determined as a tempo period (step C5).

  If there is no frequency component separated from the approximate line by 1.25 dB or more in step C1, the feature data extraction unit 13 determines whether there is a component separated from the approximate line by 0.7 dB or more in the period of 0.3 to 2s. That is, it is determined whether or not there is a frequency component that is separated from the approximate line by 0.7 dB or more within the frequency range of 1 / 0.3 to 1/2 Hz (step C6). If there is a frequency component that is present, among the frequency components that are separated from the approximate line by 0.7 dB or more, the cycle of the frequency component with the longest cycle is set as a candidate A (step C7).

  Next, the feature data extraction unit 13 determines whether or not the candidate A is 1 s or longer (step C8), and when the candidate A is 1 s or longer, determines the candidate A / 2 as a tempo period ( Step C9) If the candidate A is not 1 s or longer, it is determined whether the candidate A is 0.6 s or longer (Step C10). When the candidate A is 0.6 s or more, it is determined whether there are two or more frequency components apart from the approximate line by 0.7 dB or more other than the candidate A (step C11). If there are two or more frequency components separated from the straight line by 0.7 dB or more, the candidate A / 2 is determined as the tempo period (step C9). If candidate A is not 0.6 s or more in step C10, and if there are no more than two frequency components apart from the approximate line by 0.7 dB or more other than candidate A in step C11, candidate A is determined as the tempo period. (Step C12).

  If there is no frequency component separated by 0.7 dB or more from the approximate line in step C11, the feature data extraction unit 13 determines whether there is a component separated from the approximate line by 0.6 dB or more in the period of 0.3 to 2 s. That is, it is determined whether or not there is a frequency component that is separated from the approximate line by 0.6 dB or more within the frequency range of 1 / 0.3 to 1/2 Hz (step C13), and the approximate line is separated by 0.6 dB or more. If there is a frequency component that is present, among the frequency components that are separated from the approximate line by 0.6 dB or more, the cycle of the frequency component with the longest cycle is set as a candidate A (step C7).

  Next, the feature data extraction unit 13 determines whether or not the candidate A is 1 s or longer (step C8), and when the candidate A is 1 s or longer, determines the candidate A / 2 as a tempo period ( Step C9) If the candidate A is not 1 s or longer, it is determined whether the candidate A is 0.6 s or longer (Step C10). If the candidate A is 0.6 s or more, it is determined whether there are two or more frequency components apart from the approximate line by 0.6 dB or more other than the candidate A (step C11). When there are two or more frequency components separated from the straight line by 0.6 dB or more, the candidate A / 2 is determined as the tempo period (step C9), and when the candidate A is not 0.6s or more in step C10, If there are no more than two frequency components apart from the approximate straight line by 0.6B or more other than candidate A in step C11, candidate A is determined as the tempo period (step C12).

  If there is no frequency component separated by 0.6 dB or more from the approximate line in step C13, the feature data extraction unit 13 determines whether there is a component separated from the approximate line by 0.6 dB or more in the period 3 to 4s. That is, it is determined whether or not there is a frequency component that is separated from the approximate line by 0.6 dB or more within the frequency range of 1/3 to 1/4 Hz (step C14), and the frequency component that is separated from the approximate line by 0.6 dB or more. If there is a frequency component that is 0.6 dB or more away from the approximate line, the cycle of the frequency component with the longest cycle is set as the candidate A (step C15), and the candidate A / 4 is determined as the cycle of the tempo (step S15). Step C16).

  If there is no frequency component separated by 0.6 dB or more from the approximate line in step C14, the feature data extraction unit 13 has a component separated from the approximate line by 0.7 dB or more in the period of 0.1 to 0.3 s. Whether there is a frequency component that is more than 0.7 dB away from the approximate line within the frequency range of 1 / 0.1 to 1 / 0.3 Hz (step C17). When there is no frequency component separated by .7 dB or more, 1s is determined as the tempo period (step C18), and when there is a frequency component separated by 0.7 dB or more from the approximate line, 0.3 s is set. The tempo period is determined (step C19).

  In the present embodiment, the tempo period is extracted as the tempo of the music, but it may be converted into BPM (beat per minutes) generally used as a unit representing the tempo. You may make it process a tempo period with a fixed processing method.

Next, referring to FIG. 4, the impression degree data conversion unit 14 is a hierarchical type composed of an input layer (first layer), an intermediate layer (n-th layer), and an output layer (N-th layer) as shown in FIG. By inputting the feature data extracted by the feature data extraction unit 13 into the input layer (first layer) using a neural network, impression level data is output from the output layer (Nth layer), that is, the feature data is converted into the impression level. The data is converted into data (step A5), the impression degree data output from the output layer (Nth layer) is output to the music mapping unit 16, and the feature data input from the feature data extraction unit 13 and the output layer ( The impression degree data output from the (Nth layer) is stored in the music database 15 together with the music data. Note that the connection weight value w of each neuron in the intermediate layer (nth layer) is learned in advance by the evaluator. In the case of this embodiment, the feature data input to the input layer (first layer), that is, the feature data items extracted by the feature data extraction unit 13 are Low, Middle, High as described above. There are seven items, the slope of the approximate straight line and the Y-intercept and the tempo, and the evaluation items of the impression degree data are “bright and dark”, “clear and dark” judged by human sensitivity, Three items of “Intense and Calm” were set, and each evaluation item was set so as to be expressed by a seven-step evaluation. Accordingly, the number of neurons L ₁ in the input layer (first layer) is 7, and the number of neurons L _{N in} the output layer (Nth layer) is 3, and the intermediate layer (nth layer: n = 2). ,..., N-1) is set appropriately.

  The music mapping unit 16 maps the music data input from the music data input unit 11 to a corresponding portion of the music map stored in the music map storage unit 17 (step A6). The music map which is an arbitrary space used for the mapping operation in the music mapping unit 16 is divided into a plurality of partial spaces, and the music data is arranged in any of the plurality of partial spaces. As the music map, for example, a self-organizing map in which neurons are regularly arranged in two dimensions (9 * 9 square in the example shown in FIG. 8) can be used. In this case, a plurality of neurons are used. Corresponds to the subspace.

  In this embodiment, a two-dimensional SOM in which neurons are arranged in a square of 100 * 100 is used as a music map, and coordinates (1, 1) to (100, 100) are assigned to each neuron of the music map. The music mapping unit 16 creates a search table for searching for music data mapped to the music map, and uses the created search table as a music database. 15 and the coordinates of the neuron in which the music data is arranged are stored in the music database 15 together with the music data. The search table is for searching for the corresponding neuron and specifying the music data arranged in the searched neuron. As shown in FIG. 9, the coordinates of each neuron in the music map and each neuron It is composed of arranged music data (numbers specifying music data, etc.). In this embodiment, a two-dimensional SOM in which neurons are arranged in a square is used as the music map. However, the arrangement of neurons may be a rectangle or a honeycomb, and a three-dimensional SOM is used. You may do it.

In addition, the music map used for the mapping operation in the music mapping unit 16 is learned in advance, and each neuron is given a pre-learned n-dimensional feature vector m _i (t) εR ^n. cage, music mapping unit 16, and impression data that has been converted by the impression-data-conversion unit 14, and a tempo extracted by the feature data extraction unit 13 and the input vector x _j, nearest neuron to the input vector x _j, That is, the input music data is arranged in a neuron that minimizes the Euclidean distance ‖x _j −m _i 、, and the music map to which the music data is mapped is stored in the music map storage unit 17. Note that R indicates the number of evaluation stages of each evaluation item of impression degree data, and n indicates the number of items of impression degree data.

Next, the learning operation of the hierarchical neural network used for the conversion operation (step A5) in the impression degree data conversion unit 14 will be described in detail with reference to FIG.
FIG. 10 is a flowchart for explaining the learning operation of the hierarchical neural network in the neural network learning apparatus shown in FIG.

  Learning of the hierarchical neural network (connection weight value w) by the evaluator is performed using, for example, the neural network learning device 40 shown in FIG. 2. First, the hierarchical neural network (connection weight value w) is pre-learned. The pre-learning data (feature data feature data + impression degree data) is input.

  A music medium such as a CD or DVD is set in the music data input unit 41, music data is input from the music data input unit 41 (step D1), and the feature data extraction unit 43 inputs the music data. Feature data is extracted from the music data input from the unit 41 (step D2). The feature data extracted by the feature data extraction unit 43 is the same as the feature data extracted by the feature data extraction unit 13 of the music registration device 10.

  The audio output unit 42 outputs the music data input from the music data input unit 41 as audio (step D3), and the evaluator listens to the audio output from the audio output unit 42 to determine the impression level of the music. Evaluation is performed based on sensitivity, and the evaluation result is input as impression degree data from the impression degree data input unit 44 (step D4). The combined weight value learning unit 45 uses the impression degree data input from the impression degree data input unit 44 as a teacher signal. Accept as. In the present embodiment, as the evaluation items for impression degree, three items of “bright, dark”, “clear, fuzzy”, and “violent, gentle” determined by human sensitivity are set. A seven-stage evaluation for the evaluation item is configured to be received by the impression degree data input unit 44 as impression degree data.

Next, it is determined whether a learning data consisting of characteristic data and the inputted impression data reaches the number of samples T ₁ for a predetermined (step D5), until the learning data reaches the number of samples T ₁ Steps D1 to D4 are repeated.

Learning of the hierarchical neural network in the connection weight value learning unit 45, that is, the update of the connection weight value w of each neuron is performed using an error back propagation learning method.
First, as an initial value, the connection weight value w of all the neurons of the intermediate layer (nth layer) is set to a small value in the range of about −0.1 to 0.1 by a random number, and the connection weight value learning unit 45 inputs the feature data extracted by the feature data extraction unit 43 to the input layer (first layer) as an input signal x _j (j = 1, 2,..., 8), and from the input layer (first layer). The output of each neuron is calculated toward the output layer (Nth layer).

Next, the combined weight value learning unit 45 uses the impression degree data input from the impression degree data input unit 44 as a teacher signal y _j (j = 1, 2,..., 8), and outputs the output layer (Nth layer). The learning rule δ _j ^N is calculated from the error between the output out _j ^N and the teacher signal y _j by the following equation.

Next, the joint weight value learning unit 45 calculates the error signal δ _j ⁿ of the intermediate layer (nth layer) using the learning rule δ _j ^N by the following equation.

  In Equation 2, w represents a connection weight value between the n-th layer j-th neuron and the (n −1) -th layer k-th neuron.

Next, the connection weight value learning unit 45 calculates the amount of change Δw of the connection weight value w of each neuron using the following equation using the error signal δ _j ⁿ of the intermediate layer (nth layer), and the connection weight of each neuron. The value w is updated (step D6). In the following equation, η represents a learning rate, and is set to η ₁ (0 <η ₁ ≦ 1) in learning by the evaluator.

At step D6, each of the pre-training data sample number T ₁ learning is performed, then, or smaller or not than the reference value E ₁ for pre-learning is square error E shown in the following equation predetermined There is judged (step D7), the operation of step D6 to the square error E is smaller than the reference value E ₁ is repeated. Incidentally, the learning iterations S squared error E is assumed to be smaller than the reference value E ₁ is set in advance, may be the operation of step D6 to repeat the learning iterations S times.

In the case of the square error E is determined to be smaller than the reference value E ₁ Step D7, connection weights learning unit 45, the connection weights output unit 47 the coupling weight value w for each neuron is prior learning The connection weight value w of each neuron outputted and output from the connection weight value output unit 47 is stored in the impression degree data conversion unit 14.

Next, the music map learning operation used in the mapping operation (step A6) in the music mapping unit 16 will be described in detail with reference to FIG.
FIG. 11 is a flowchart for explaining the music map learning operation in the neural network learning apparatus shown in FIG.

  A music medium such as a CD or DVD is set in the music data input unit 41, music data is input from the music data input unit 41 (step E1), and the feature data extraction unit 43 inputs the music data. The tempo is extracted as feature data from the music data input from the unit 41 (step E2), and the extracted tempo is output to the music map learning unit 46. Note that the tempo extracted by the feature data extraction unit 43 is the same as the tempo extracted by the feature data extraction unit 13 of the music registration device 10.

  Next, the audio output unit 42 outputs the music data input from the music data input unit 41 as audio (step E3), and the evaluator listens to the audio output from the audio output unit 42 to thereby determine the degree of impression of the music. Is evaluated by sensitivity, and the evaluation result is input as impression degree data from the impression degree data input unit 44 (step E4).

  The music map learning unit 46 receives the tempo input from the feature data extraction unit 43 and the impression level data input from the impression level data input unit 44 as input vectors to the self-organizing map. In the present embodiment, as the evaluation items for impression degree, three items of “bright, dark”, “clear, fuzzy”, and “violent, gentle” determined by human sensitivity are set. A seven-stage evaluation for the evaluation item is configured to be received by the impression degree data input unit 44 as impression degree data.

The music map learning unit 46 uses the tempo input from the feature data extraction unit 43 and the impression level data input from the impression level data input unit 44 as an input vector x _j (t) εR ^n, and features of each neuron. Train the vector m _i (t) εR ⁿ . Note that t represents the number of learning times, a preset value T that determines the number of learning times is set in advance, and learning is performed for the learning number t = 0, 1,. Note that R represents the evaluation stage of each evaluation item, and n represents the number of items of impression degree data.

First, as an initial value, feature vectors m _c (0) of all neurons are set at random in the range of 0 to 1, and the music map learning unit 46 determines that the neuron c, which is closest to x _j (t), That is, a winner neuron c that minimizes ‖x _j (t) −m _c (t) ‖ is obtained, and a feature vector m _c (t) of the winner neuron c and a set Nc of neighboring neurons i in the vicinity of the winner neuron c Each feature vector m _i (t) (iεNc) is updated according to the following equation (step E5). It is assumed that the neighborhood radius for determining the neighborhood neuron i is set in advance.

In Equation 5, h _ci (t) represents a learning rate and is obtained by the following equation.

Α _init is an initial value of the learning rate, and R ² (t) is a monotonically decreasing linear function or exponential function.

Next, the music map learning unit 46 determines whether or not the learning count t has reached the set value T (step E6), and performs the processing operations of steps E1 to E5 until the learning count t reaches the set value T. Repeatingly, when the learning count t reaches the set value T, the same processing operation is performed again from the first sample. When the number of iterations reaches a predetermined number of times S, the feature vector m _i (T) εR ⁿ learned through the feature vector output unit 48 is output (step E7). The output feature vector m _i (T) of each neuron i is stored as a music map in the music map storage unit 17 of the music registration device 10.

Next, a music search operation in the music registration apparatus 10 will be described in detail with reference to FIGS.
12 is a flowchart for explaining a music search operation in the music registration device shown in FIG. 1, and FIG. 13 is a diagram showing an example of a search condition input screen displayed on the PC display unit shown in FIG. 14 is a diagram showing an example of a condition search result display screen displayed on the PC display unit shown in FIG. 1, and FIG. 15 is an example of a representative song search result display screen displayed on the PC display unit shown in FIG. 16 is a diagram showing an example of a music list display screen displayed on the PC display unit shown in FIG. 1, and FIG. 17 is a music map display screen displayed on the PC display unit shown in FIG. It is a figure which shows an example.

  The music search unit 18 displays a search condition input screen 51 for inputting search conditions on the PC display unit 20, and accepts user input from the PC operation unit 19. As shown in FIG. 13, the search condition input screen 51 includes a bibliographic data input area 511 for inputting bibliographic data as a search condition, a condition search execution button 512 for instructing execution of a search, an all song list display button 513, A map display button 514 is provided, and the user inputs bibliographic data as a search condition from the PC operation unit 19 (step F1), and clicks the condition search execution button 512 to perform a search based on the bibliographic data. To instruct.

  The music search unit 18 searches the music database 15 based on the bibliographic data input from the PC operation unit 19 (step F2), and causes the PC display unit 20 to display a condition search result display screen 52 that displays the condition search results. (Step F3). Referring to FIG. 14, the conditional search result display screen 52 has a representative song selection area 521 for selecting a representative song from the conditional search results, and a representative song search execution button 522 for instructing a search based on the selected representative song. And a registration button 523 for registering a condition search result as a search list.

  Next, the user selects a representative song from the condition search results displayed in the representative song selection area 521 (step F4), and clicks the representative song search execution button 522 to perform a search based on the representative song. The music search unit 18 is instructed. When the registration button 523 is clicked at this time, the music search unit 18 stores the condition search results displayed in the representative song selection area 521 in the music database 15 as a search list.

  The music search unit 18 searches the music database 15 based on the selected representative music, thereby specifying the coordinates of a neuron (hereinafter referred to as a representative neuron) in which the representative music is arranged in the music map (step F5). ).

  Next, the music search unit 18 sets 0 to the variable n (step F6), and refers to the search table shown in FIG. 9 stored in the music map storage unit 17, thereby identifying the representative neuron specified in step F5. Is searched for music data arranged in a neuron at a distance n (step F7), and it is determined whether or not the number of searched music is greater than or equal to a predetermined number of searches (step F8).

  If the number of songs searched in step F8 is less than the predetermined search number, the song search unit 18 increments the variable n (step F9), returns to step F7, and the number of searched songs is preset. Steps F7 and F8 are repeated until the number of searches exceeds a predetermined number.

  If the number of songs searched in step F8 is equal to or greater than the predetermined number of searches, the music search unit 18 narrows down the searched music data to the predetermined number of searches (step F10) and then narrows down. The representative song search result display screen 53 is displayed on the PC display unit 20 with the song data as the representative song search result (step F11). As shown in FIG. 15, the representative song search result display screen 53 includes an output song selection area 531 for selecting song data to be output, and an output button 532 for instructing output of the selected song data.

In step F10, the searched music data is narrowed down by using the representative neuron feature vector or the impression data converted by the representative music impression data converter 14 as the input vector _xj, and the impression data of the searched music data. When the search target vector X _j, the input vectors x _j closer search target vector X _j, i.e. is performed by searching to the Euclidean distance ‖X _j -x _j searches that ‖ is sequentially predetermined ones less. It should be noted that the music data searched in step F10 is limited only to music data arranged in a neuron farthest from the representative neuron, that is, a neuron separated from the representative neuron by the maximum value of the variable n used for the search. By doing so, the number of objects for which distance calculation is performed can be reduced, so that the search time (calculation time) can be shortened. Further, all of the searched music data may be displayed as the representative music search result on the representative music search result display screen 53 without narrowing down the searched music data in step F10. In this case, Since the search can be performed without calculating the distance, the search time (calculation time) can be further shortened.

  Next, when the output button 532 is clicked, the music search unit 18 stores the representative music search result displayed in the output music selection area 531 as a search list in the music database 15 (step F12).

  When the all song list display button 513 is clicked on the search condition input screen 51, the song search unit 18 displays a song list display screen 54 that displays a song list of all stored song data on the PC display unit 20. Display. Referring to FIG. 16, the song list display screen 54 is a representative song selection area 541 for selecting a representative song from the song list of all song data, and a representative song search execution button for instructing a search based on the selected representative song. 542.

  When the user selects a representative song from the song list displayed in the representative song selection area 541 (step F4) and clicks the representative song search execution button 542, a search based on the selected representative song is performed. Instructed by the unit 18, the music search operation in steps F5 to F12 is performed.

  Further, when the music map display button 514 is clicked on the search condition input screen 51, the music search unit 18 causes the PC display unit 20 to display a music map display screen 55 that displays neurons of the music map as points. Referring to FIG. 17, the music map display screen 55 includes a neuron selection area 551 for selecting a neuron as a search center and a map search execution button 552 for instructing a search based on the selected neuron.

  When the user selects a neuron as a search center from the neurons displayed in the neuron selection area 551 and clicks the map search execution button 552, the neuron is selected instead of the representative neuron specified in step F5. A search based on the neuron selected in the region 551 is instructed to the music search unit 18, and the music search operation in steps F 6 to F 12 is performed thereafter.

  In the neuron selection area 551, only the neuron where the music data is arranged is displayed, and when the cursor is moved to the neuron to which the music is mapped, the music data of the music data arranged in the neuron where the cursor is located A name pop-up display 553 is performed.

Next, the music data transfer operation from the music registration device 10 to the mobile terminal device 30 will be described in detail.
FIG. 18 is an explanatory diagram for explaining an operation of transferring music data from the music registration device shown in FIG. 1 to the mobile terminal device.

  In the music database 15 of the music registration device 10, as shown in FIG. 18, music data, feature data corresponding to the music data (extracted by the feature data extraction unit 13), bibliographic data, impression data (impression data) Conversion unit 14), neuron coordinates in the music map (mapped by music mapping unit 16), search table (created by music mapping unit 16), and search list (searched by music search unit 18) are stored. The music data, the bibliographic data corresponding to the music data, the impression data, and the neuron coordinates in the music map, the search table, and the search list are transferred to the terminal music database 32 of the mobile terminal device 30. .

  When the portable terminal device 30 is connected to the transmission / reception unit 21 of the music registration device 10 and the transmission / reception unit 31 of the portable terminal device 30 via the data transmission path 23, the stored contents of the music database 15 of the music registration device 10 and the portable The content stored in the terminal music database 32 of the terminal device 30 is compared, and the content stored in the music database 15 of the music registration device 10 is reflected in the terminal music database 32 of the mobile terminal device 30.

Next, the music search operation in the mobile terminal device 30 will be described in detail with reference to FIG.
FIG. 19 is a diagram illustrating a display example in the terminal display unit of the mobile terminal device illustrated in FIG. 2.

  When the mobile terminal device 30 is turned on by a power key (not shown), the music search unit 33 searches for music data to be reproduced as “search by bibliographic data”, “search by impression data”, “search list”. A search method selection screen 351 for selecting any one of “search by” is displayed on the terminal display unit 35, and the user selects the search method displayed on the search method selection screen 351 with the up and down of the cross key 341, Pressing the enter key 342 instructs the music search unit 33 to search for music data based on the selected search method.

  When “search by bibliographic data” is selected on the search method selection screen 351, the music search unit 33 selects a list display for selecting one of “artist”, “album”, and “title”. The screen 352 is displayed on the terminal display unit 35, and the user selects one of “artist”, “album”, and “title” displayed on the list display selection screen 352 with the up and down keys of the cross key 341, and presses the enter key. Pressing 342 instructs the music search unit 33 to display the selected list.

  When “artist” or “album” is selected on the list display selection screen 352, the music search unit 33 searches for the artist name or album name from the bibliographic data corresponding to all music data stored in the terminal music database 32. And an artist / album list screen 353 for displaying a list of artist names or album names is displayed on the terminal display unit 35. The user selects either the artist name or the album name displayed on the artist / album list screen 353 with the up and down keys of the cross key 341 and presses the enter key 342 to search for the artist name or album name. The search unit 33 is instructed, and the music search unit 33 searches the terminal music database 32 based on the selected artist name or album name, and displays a title list of the corresponding music data on the terminal display unit 35 as a search result screen 354. indicate. When “title” is selected on the list display selection screen 352, the music search unit 33 reads the titles corresponding to all the music data stored in the terminal music database 32 and searches the read title list. The result screen 354 is displayed on the terminal display unit 35.

When “Search by impression degree data” is selected on the search method selection screen 351, the music search unit 33 displays an impression degree data input screen 355 for inputting impression degree data as a search condition on the terminal display unit 35. The user selects the value of each item of the impression degree data with the up / down / left / right directions of the cross key 341 and presses the enter key 342 to instruct the music search unit 33 to perform a search based on the impression degree data. When the search based on the impression degree data is instructed, the music search unit 33 searches the music data stored in the terminal music database 32 based on the impression degree data input as the search condition, and the searched music data The title list is displayed on the terminal display unit 35 as a search result screen 354. Music retrieval data based on the input impression data as a search condition, with the input vector x _j the input impression data as a search condition, corresponding to all music data stored in the terminal song database 32 Each impression degree data is set as a search target vector m _i (t) εR ^n, and a predetermined number is searched from the one having a smaller Euclidean distance ‖x _j −m _i ‖. Note that R indicates the number of evaluation stages of each evaluation item of impression degree data, and n indicates the number of items of impression degree data.

  If “search by search list” is selected on the search method selection screen 351, the music search unit 33 uses the search list list 356 as a search list list stored in the terminal music database 32, and the terminal display unit 35. The user selects one of the search lists displayed on the search list list screen 356 with the up and down keys of the cross key 341 and presses the enter key 342 to display the display of the selected search list. The music search unit 33 displays the title list of the selected search list on the terminal display unit 35 as a search result screen 354.

  When the right side of the cross key 341 is pressed while the search result screen 354 is displayed, the music search unit 33 sends music data corresponding to the title list displayed on the search result screen 354 from the terminal music database 32. Sequentially read out, and the read music data is output to the audio output unit 22 for audio output. When the search result screen 354 is displayed and the left of the cross key 341 is pressed after selecting one of the titles with the up and down keys of the cross key 341, the music search unit 33 selects the selected title. Only the music data corresponding to is read from the terminal music database 32, and the read music data is output to the audio output unit 22 for audio output.

  Further, when the search result screen 354 is displayed and one of the titles is selected with the up and down keys of the cross key 341 and then the enter key 342 is pressed, the music search unit 33 selects the selected title. A representative song search as a representative song is performed, and the representative song search result is displayed on the terminal display unit 35 as a search result screen 354. The representative music search performed by the music search unit 33 is performed based on the neuron coordinates and the search table stored in the terminal music database 32, and the representative music search performed by the music search unit 18 of the music registration apparatus 10 described above. The procedure is the same as the song search.

  As described above, according to the present embodiment, in the music registration device 10, physical feature data is extracted from the music data input by the feature data extraction unit 13, and the feature data is converted by the impression degree data conversion unit 14. The data is converted into impression degree data determined by human sensitivity, the converted impression degree data is stored in the music database 15 together with the music data, and the music data and the impression data stored in the music database 15 by the transmission / reception unit 21. Is transferred to the mobile terminal device 30. In the mobile terminal device 30, the music data and impression degree data transferred from the transmission / reception unit 21 of the music registration device 10 by the transmission / reception unit 31 are stored in the terminal music database 32, The terminal operation unit 34 accepts input as a search condition for impression degree data, The terminal music database 32 is searched based on the impression degree data input as a search condition by the search unit 33 to specify a predetermined number of music data, and the music data searched by the audio output unit 36 is output as a voice. By doing so, the music data can be searched using the impression data converted from the feature data of the music data in the music registration device 10 in the mobile terminal device 30, so that the impression data determined by human sensitivity can be obtained. The mobile terminal device 30 can search for music data with high accuracy based on the above.

  Further, according to the present embodiment, in the music registration device 10, physical feature data is extracted from the music data input by the feature data extraction unit 13, and the feature data is converted into human data by the impression degree data conversion unit 14. It is converted into impression degree data determined by sensitivity, and the music mapping unit 16 maps the music data to a music map divided into a plurality of partial spaces to which coordinates are given based on the impression degree data, and also coordinates in the partial space And a song table arranged in the partial space are created, and the search table is transferred together with the song data stored in the song database 15 by the transmission / reception unit 21 to the portable terminal device 30. In the device 30, the music transferred from the transmission / reception unit 21 of the music registration device 10 by the transmission / reception unit 31. The data and the search table are stored in the terminal music database 32, the instruction input of the representative music search specifying the representative music is received by the terminal operation unit 34, the search table is searched based on the representative music by the music search unit 33, and a predetermined number The music degree data converted by the music registration device 10 from the feature data of the music data in the portable terminal device 30 is configured to specify the music data of the music data and output the music data searched by the voice output unit 36 as voice. Since the search based on the music map mapped by using can be performed, the mobile terminal device 30 can search for the music data with high accuracy based on the impression degree data determined by the human sensitivity.

  Note that the present invention is not limited to the above-described embodiments, and it is obvious that the embodiments can be appropriately changed within the scope of the technical idea of the present invention. In addition, the number, position, shape, and the like of the constituent members are not limited to the above-described embodiment, and can be set to a suitable number, position, shape, and the like in practicing the present invention. In each figure, the same numerals are given to the same component.

It is a block diagram which shows the structure of embodiment of the music search system which concerns on this invention. It is a front view which shows the structure of the portable terminal device shown in FIG. It is a block diagram which shows the structure of the neural network learning apparatus which learns in advance the neural network used for the music registration apparatus shown in FIG. It is a flowchart for demonstrating the music registration operation | movement in the music registration apparatus shown in FIG. It is a flowchart for demonstrating the feature data extraction operation | movement in the feature data extraction part shown in FIG. It is a flowchart for demonstrating the tempo determination operation | movement in the feature data extraction part shown in FIG. It is explanatory drawing which shows the hierarchical neural network example used by the impression degree data conversion part shown in FIG. It is explanatory drawing which shows the example of a music map which a music mapping part shown in FIG. 1 maps music data. It is a figure which shows the example of a search table memorize | stored in the music database shown in FIG. 3 is a flowchart for explaining a learning operation of a hierarchical neural network in the neural network learning apparatus shown in FIG. 3 is a flowchart for explaining a music map learning operation in the neural network learning apparatus shown in FIG. 2. It is a flowchart for demonstrating the music search operation | movement in the music registration apparatus shown in FIG. It is a figure which shows the example of a search condition input screen displayed on the PC display part shown in FIG. It is a figure which shows the example of a condition search result display screen displayed on the PC display part shown in FIG. It is a figure which shows the example of a representative music search result display screen displayed on the PC display part shown in FIG. It is a figure which shows the example of a music list display screen displayed on the PC display part shown in FIG. It is a figure which shows the example of a music map display screen displayed on the PC display part shown in FIG. It is explanatory drawing for demonstrating the transfer operation | movement of the music data from the music registration apparatus shown in FIG. 1 to a portable terminal device. It is a figure which shows the example of a display in the terminal display part of the portable terminal device shown in FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Music registration apparatus 11 Music data input part 12 Compression processing part 13 Feature data extraction part 14 Impression degree data conversion part 15 Music database 16 Music mapping part 17 Music map memory | storage part 18 Music search part 19 PC operation part 20 PC display part 21 Transmission / reception Unit 22 voice output unit 23 data transmission path 30 portable terminal device 31 transmission / reception unit 32 terminal music database 33 music search unit 34 terminal operation unit 35 terminal display unit 36 voice output unit 40 neural network learning device 41 music data input unit 42 voice output unit Reference Signs List 43 Feature data extraction unit 44 Impression degree data input unit 45 Join weight value learning unit 46 Music map learning unit 47 Join weight value output unit 48 Feature vector output unit 51 Search condition input screen 52 Condition search result display screen 53 Representative song search result display Screen 54 Song list Display screen 55 Music map display screen 341 Cross key 342 Enter key 351 Search method selection screen 352 List display selection screen 353 Artist / album list screen 354 Search result screen 355 Impression degree data input screen 356 Search list list screen 511 Bibliographic data input area 512 Condition search execution button 513 All song list display button 514 Music map display button 521 Representative song selection area 522 Representative song search execution button 523 Registration button 531 Output song selection area 532 Output button 541 Representative song selection area 542 Representative song search execution button 551 Neuron selection Area 552 Map search execution button 553 Pop-up display

Claims (7)

A music registration device that stores input music data in a music database, and a portable terminal device that is configured to be connectable to the music registration device and has a terminal music database to which the music data stored in the music database is transferred A music search system comprising:
The music registration device includes feature data extraction means for extracting physical feature data from the input music data;
Impression degree data conversion means for converting the feature data extracted by the feature data extraction means into impression degree data determined by human sensitivity and storing the converted impression degree data in the music database together with the music data When,
Music data transmission means for transferring the music data and impression data stored in the music database to the mobile terminal device;
The portable terminal device includes music data receiving means for storing the music data and the impression degree data transferred from the music data transmitting means of the music registration device in the terminal music database;
Terminal operating means for receiving input as a search condition for the impression degree data;
Music search means for searching the terminal music database based on the impression data input as search conditions from the terminal operation means to identify a predetermined number of the music data;
A music search system comprising voice output means for outputting the music data searched by the music search means.   The music search means uses the impression degree data input as a search condition from the terminal operation means as an input vector, and uses the impression degree data stored in the terminal music database as a search target vector. The music search system according to claim 1, wherein a predetermined number is searched in ascending order of distance. A music registration device that stores input music data in a music database, and a portable terminal device that is configured to be connectable to the music registration device and has a terminal music database to which the music data stored in the music database is transferred A music search system comprising:
The music registration device includes feature data extraction means for extracting physical feature data from the input music data;
Impression degree data converting means for converting the feature data extracted by the feature data extracting means into impression degree data determined by human sensitivity;
Based on the impression degree data converted by the impression degree data converting means, the input music data is mapped to a music map divided into a plurality of partial spaces to which coordinates are given, and coordinates of the partial space And music mapping means for creating a search table comprising the music data arranged in the partial space,
Music data transmission means for transferring the search table created by the music mapping means together with the music data stored in the music database to the mobile terminal device;
The portable terminal device includes music data receiving means for storing the music data transferred from the music data transmitting means of the music registration device and the search table in the terminal music database;
A terminal operation means for receiving an instruction input for representative song search specifying a representative song;
Music search means for searching the search table based on the representative music specified by the terminal operation means and specifying a predetermined number of the music data;
A music search system comprising voice output means for outputting the music data searched by the music search means. The music map is a self-organizing map that has been learned,
The plurality of subspaces are a plurality of neurons each having a different feature vector,
The music mapping means has the impression degree data converted by the impression degree data conversion means as an input vector, and is arranged in the neuron to which the feature vector having the shortest distance from the input vector is assigned. The music search system according to claim 3, wherein:   The impression degree data converting means converts the feature data extracted by the feature data extracting means into impression degree data determined by human sensitivity using a previously learned hierarchical neural network. The music search system according to claim 1 to 4. A music registration device that stores input music data in a music database, and a portable terminal device that is configured to be connectable to the music registration device and has a terminal music database to which the music data stored in the music database is transferred A music search method in a music search system comprising:
The music registration device extracts physical feature data from the input music data,
The extracted feature data is converted into impression degree data determined by human sensitivity, and the converted impression degree data is stored in the music database together with the music data,
Transferring the music data and the impression data stored in the music database to the mobile terminal device;
The portable terminal device stores the music data and the impression degree data transferred from the music registration device in the terminal music database,
Accepting input as a search condition for the impression degree data,
The terminal music database is searched based on the impression degree data received as a search condition to identify a predetermined number of the music data,
A music search method comprising: outputting the searched music data by voice. A music registration device that stores input music data in a music database, and a portable terminal device that is configured to be connectable to the music registration device and has a terminal music database to which the music data stored in the music database is transferred A music search method in a music search system comprising:
The music registration device extracts physical feature data from the input music data,
Converting the extracted feature data into impression degree data determined by human sensitivity,
Based on the converted impression degree data, mapping to a music map divided into a plurality of partial spaces to which coordinates are assigned, and from the coordinates of the partial space and the music data arranged in the partial space. Create a search table
Transfer the created search table together with the music data stored in the music database to the mobile terminal device,
The portable terminal device stores the music data and the search table transferred from the music registration device in the terminal music database,
Accepting the instruction input of the representative song search specifying the representative song,
Search the search table based on the accepted representative song to identify a predetermined number of the song data,
A music search method comprising: outputting the searched music data by voice.

Images