JP2005309712A - Musical piece retrieval system and musical piece retrieval method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a musical piece retrieval system and a musical piece retrieval method capable of performing correction of impression data corresponding to musical piece data under reproduction in a terminal device. <P>SOLUTION: A correction instruction inputted by a terminal operation part 33 is stored in a retrieval result storage part 32. When the terminal device 30 is connected to a musical piece retrieval device 10, the instruction is transmitted to the device 10 by a transceiving part 31. A musical piece retrieval part 18 to which the correction instruction is inputted automatically corrects impression data of a musical piece the correction of which is instructed in a direction leaving a retrieval condition to update the impression data stored in a musical piece database 15 to automatically corrected correction data, and reads characteristic data from the musical piece database 15 to store the correction data and the characteristic data as relearnt data in a correction data storage part 23. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a music search system and a music search method for searching for desired music from music data stored in large quantities in a large-capacity storage means such as an HDD, and in particular, impression degree data determined by human sensitivity. The present invention relates to a music search system and a music search method capable of searching music based on the above.

  In recent years, a large-capacity storage means such as an HDD has been developed, and a large amount of music data can be stored in the large-capacity storage means. Searching for a large amount of music data stored in a large-capacity storage means is generally performed using bibliographic data such as artist names, music titles, and other keywords. This is not suitable when it is not possible to take into account the emotions of a music piece and there is a possibility that a music piece with a different impression will be searched, and it is desired to search for a music piece with a similar impression when listening.

  Therefore, in order to make it possible to search for the music desired by the user based on the subjective impression of the music, the user's subjective requirements for the music desired to be searched are input, quantified and output, and the output The music database that stores the estimated impression value obtained by quantifying the impression of the music to be searched and stores the acoustic signal of a plurality of music and the impression value obtained by quantifying the impression of the music using the calculated predicted impression value as a key. There has been proposed an apparatus for searching for desired music based on a subjective image of a user's music by searching (for example, see Patent Document 1).

However, in the prior art, when searching for an impression value obtained by converting the physical characteristics of a music piece, the user's subjective requirements are input using a numerical predicted impression value. As a result, the input items of the subjective requirements that are input as are aggregated, and it is not possible to search for music data with high accuracy based on the subjective requirements, and the impression value that quantifies the impression of the music is fixed. Therefore, there is a problem that the impression value after conversion may not always match the preference of individual users.
JP 2002-278547 A

  The present invention has been made in view of such problems, and the object of the present invention is determined by a hierarchical neural network based on feature data consisting of a plurality of physical items of music and human sensitivity. Impression degree judged by human sensibility without aggregating items of impression degree data judged by human sensibility input as search conditions by the user by directly associating with impression degree data consisting of items Provided is a music search system and a music search method capable of searching music data with high accuracy based on data and further correcting impression data corresponding to music data being played back in a terminal device. In the point.

In order to solve the above problems, the present invention has the following configuration.
The music search system of the present invention is a music search system that searches for desired music data from a plurality of music data stored in a music database, and outputs the searched music data to a terminal device. Music data input means, feature data extraction means for extracting physical feature data from the music data input by the music data input means, and the feature data extracted by the feature data extraction means according to human sensitivity Impression degree data conversion means for converting the impression degree data to be determined, and storage for storing the impression degree data converted by the impression degree data conversion means in the music database together with the music data input by the music data input means Control means; impression degree data input means for inputting the impression degree data as a search condition; Music search means for searching the music database based on the impression degree data input from the impression degree data input means, and music data output means for outputting the music data searched by the music search means to the terminal device A correction instruction receiving means for receiving a correction instruction for the impression degree data from the terminal device, and automatically moving the impression degree data away from the search condition based on the correction instruction received by the correction instruction receiving means. Impression degree data updating means for creating modified correction data and updating the impression degree data stored in the music database by the correction data.

  Further, in the music search system according to the present invention, the impression degree data conversion means determines the feature data extracted by the feature data extraction means using a hierarchical neural network that has been learned in advance based on human sensitivity. The impression level data is converted to the impression level data.

  Furthermore, the music search system of the present invention is characterized in that the feature data extraction means extracts a plurality of items including fluctuation information as the feature data.

  Furthermore, in the music search system of the present invention, the music search means uses the impression degree data input from the impression degree data input means as an input vector, and uses the impression degree data stored in the music database. The search target vector is used, and the search is performed in ascending order of the Euclidean distance between the two.

  The music search system of the present invention is a music search system that searches for desired music data from a plurality of music data stored in a music database and outputs the searched music data to a terminal device. A music data input means for inputting music data, a feature data extraction means for extracting physical feature data from the music data input by the music data input means, and a feature data extraction means Further, the impression data conversion means for converting the feature data into impression data determined by human sensitivity, and the music data input means based on the impression data converted by the impression data conversion means A music map that maps the music data to a music map that is a self-organized map that has been learned in advance. Ping means, music map storage means for storing music data mapped by the music mapping means, representative music selection means for selecting representative music from music data mapped to the music map, and the representative Music search means for searching a music map based on the representative music selected by the music selection means, music data output means for outputting music data searched by the music search means to the terminal device, and from the terminal device A correction instruction receiving unit that receives a correction instruction for the impression degree data, and correction data that automatically corrects the impression degree data in a direction away from the representative song based on the correction instruction received by the correction instruction receiving unit Impression degree data correction means, and the music mapping means is provided by the impression degree data correction means. And wherein the remapping music data to the music map based on the correction data created.

  Further, in the music search system according to the present invention, the impression degree data conversion means determines the feature data extracted by the feature data extraction means using a hierarchical neural network that has been learned in advance based on human sensitivity. The impression level data is converted to the impression level data.

  Furthermore, the music search system of the present invention is characterized in that the feature data extraction means extracts a plurality of items including fluctuation information as the feature data.

  Further, in the music search system according to the present invention, the music mapping means inputs the impression degree data converted by the impression degree data conversion means as an input vector, and inputs the neuron closest to the input vector by the music data input means. The recorded music data is mapped.

  Furthermore, the music search system of the present invention is characterized in that the music search means searches for music data contained in a neuron to which a representative music is mapped.

  Furthermore, the music search system of the present invention is characterized in that the music search means searches for music data contained in a neuron to which a representative music is mapped and its neighboring neurons.

  Furthermore, the music search system of the present invention is characterized in that a neighborhood radius for determining a neighborhood neuron in the music search means can be arbitrarily set.

  Furthermore, the terminal device of the present invention is a terminal device configured to be connectable to a music search device that searches for desired music data from a plurality of music data stored in a music database, the music search device Search result input means for inputting the music data from, search result storage means for storing the music data input by the search result input means, and audio output means for reproducing the music data stored in the search result storage means And a correction instruction input means for inputting an impression degree data correction instruction corresponding to the music data being reproduced at the time of reproduction of the music data by the audio output means, and the correction instruction input when connected to the music search device Correction instruction transmitting means for transmitting the correction instruction input by the means to the music search device.

  The music search method of the present invention is a music search method for searching for desired music data from a plurality of music data stored in a music database and outputting the searched music data to a terminal device. Is extracted, physical feature data is extracted from the input music data, the extracted feature data is converted into impression degree data determined by human sensitivity, and the converted impression degree data is Stored in the music database together with the input music data, accepts input of the impression degree data as a search condition, searches the music database based on the input impression degree data, the searched music data Output to the terminal device, receive a correction instruction of the impression degree data from the terminal apparatus, and based on the received correction instruction, the impression degree Create a modified data automatically correct over data in a direction away from said search condition, and updates the impression data stored in the music database by said modified data said created.

  Furthermore, the music search method of the present invention is characterized in that the extracted feature data is converted into the impression degree data determined by human sensibility using a hierarchical neural network that has been learned in advance.

  Furthermore, the music search method of the present invention is characterized in that a plurality of items consisting of fluctuation information are extracted as the feature data.

  Furthermore, in the music search method of the present invention, the received impression degree data is used as an input vector, and the impression degree data stored in the music database is used as a search target vector, and search is performed in ascending order of the Euclidean distance between the two. It is characterized by doing.

  The music search method of the present invention is a music search method for searching for desired music data from a plurality of music data stored in a music database and outputting the searched music data to a terminal device. Is extracted, physical feature data is extracted from the input music data, the extracted feature data is converted into impression degree data determined by human sensitivity, and the converted impression degree data is converted into the converted impression degree data. Based on the above, the input music data is mapped to a music map that is a self-organized map that has been learned in advance, a representative music is selected from the music data mapped to the music map, and the selected Search music data mapped to the music map based on the representative music, and output the searched music data to the terminal device. The impression degree data correction instruction is received, and based on the received correction instruction, the impression degree data is automatically corrected in a direction away from the representative song, and the correction data is created. The music data is remapped to the music map.

  Furthermore, the music search method of the present invention is characterized in that the extracted feature data is converted into the impression degree data determined by human sensibility using a hierarchical neural network that has been learned in advance.

  Furthermore, the music search method of the present invention is characterized in that a plurality of items consisting of fluctuation information are extracted as the feature data.

  Furthermore, the music search method of the present invention is characterized in that the input music data is mapped to a neuron closest to the input vector using the converted impression data as an input vector.

  Furthermore, the music search method of the present invention is characterized in that music data included in a neuron to which the representative music is mapped is searched.

  Furthermore, the music search method of the present invention is characterized by searching music data contained in a neuron to which the representative music is mapped and its neighboring neurons.

  Furthermore, the music search method of the present invention is characterized in that the neighborhood radius for determining the neighborhood neuron can be arbitrarily set.

  The music search system and the music search method of the present invention directly associate feature data consisting of a plurality of physical items of music with impression degree data consisting of items determined by human sensitivity, using a hierarchical neural network. Thus, the feature data extracted from the music data can be converted into impression data and stored, and the stored impression data can be searched based on the impression data input by the user. It is possible to search music data with high accuracy based on impression degree data determined by human sensibility without aggregating items determined by human sensibility input as search conditions by the user. A song with the same impression when listening from a large amount of song data stored in the storage means In addition, in the terminal device with a limited number of operation buttons, the correction instruction can be issued by automatically correcting the impression degree data of the music data based on the correction instruction received from the terminal device. Impression data corresponding to the music data being played can be corrected just by inputting, and the number of music data searched against the user's intention can be reduced as the usage period becomes longer. The effect that can be improved.

  Furthermore, the music search system and the music search method of the present invention are configured to extract a plurality of items including fluctuation information as feature data, thereby accurately extracting the physical features of the music data, There is an effect that the accuracy of the impression degree data converted from the feature data can be improved.

  Furthermore, the music search system and the music search method of the present invention set various items by setting impression degree data converted from feature data and impression degree data input by a user to the same plurality of items. This is advantageous in that the user can easily perform a search based on the impression degree data.

  Furthermore, the music search system and the music search method of the present invention use the impression degree data input by the user as an input vector, the impression degree data stored in the music database as a search target vector, and both Euclidean distances. By making the search in ascending order, the search can be performed accurately even if the impression degree data items are increased, and the search accuracy can be improved.

  Furthermore, the music search system and the music search method of the present invention are self-organized maps that have been previously learned, and use a music map in which music data is mapped based on impression degree data of the music data. By simply selecting a representative song, it is possible to quickly search for a song having the same impression as the representative song from a large amount of song data stored in a large-capacity storage means. In addition, by configuring the impression degree data of the music data to be automatically corrected based on the correction instruction received from the terminal device, playback can be performed by simply inputting the correction instruction on the terminal device with limited operation buttons. The impression degree data corresponding to the music data in the music can be corrected, and the longer the usage period, the more the music data that is searched against the user's intention. It can be reduced, an effect that it is possible to improve search accuracy.

  Furthermore, the music search system and the music search method of the present invention use a self-organized map that has been learned in advance as a music map, so that music having similar impressions is arranged in the vicinity, thereby improving search efficiency. The effect of doing.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a block diagram showing a configuration of an embodiment of a music search system according to the present invention, and FIG. 2 is a diagram of a neural network learning device that learns in advance a neural network used in the music search device shown in FIG. It is a block diagram which shows a structure.

  In this embodiment, referring to FIG. 1, the music search device 10 and the terminal device 30 are connected by a data transmission path such as a USB, and the terminal device 30 is separated from the music search device 10 and is carried. It has a configuration that can.

  Referring to FIG. 1, the music search device 10 includes a music data input unit 11, a compression processing unit 12, a feature data extraction unit 13, an impression degree data conversion unit 14, a music database 15, and a music mapping unit 16. The music map storage unit 17, the music search unit 18, the PC operation unit 19, the PC display unit 20, the transmission / reception unit 21, the voice output unit 22, the correction data storage unit 23, and the neural network learning unit 24 Consists of.

  The music data input unit 11 has a function of reading a storage medium in which music data such as a CD and a DVD is stored. The music data input unit 11 inputs music data from a storage medium such as a CD and a DVD, and extracts a compression processing unit 12 and feature data. To the unit 13. You may comprise so that the music data (delivery data) via networks, such as the internet, other than storage media, such as CD and DVD, may be input. When compressed music data is input, the compressed music data is decompressed and output to the feature data extraction unit 13.

  The compression processing unit 12 compresses the music data input from the music data input unit 11 in a compression format such as MP3 or ATRAC (Adaptive Transform Acoustic Coding) at the time of registration of the music, and the compressed music data includes the artist name and the music title. Are stored in the music database 15 together with bibliographic data such as.

  The feature data extraction unit 13 extracts feature data composed of fluctuation information from the music data input from the music data input unit 11 and outputs the extracted feature data to the impression degree data conversion unit 14.

  The impression degree data conversion unit 14 converts the feature data input from the feature data extraction unit 13 into impression degree data determined by human sensitivity using a hierarchical neural network that has been learned in advance. The impression level data is output to the music mapping unit 16, and the feature data input from the feature data extraction unit 13 and the converted impression level data are associated with the music data and registered in the music database 15.

  The music database 15 is a large-capacity storage unit such as an HDD, and the music data and bibliographic data compressed by the compression processing unit 12, the feature data extracted by the feature data extraction unit 13, and the impression degree data conversion unit 14. Is stored in association with the impression degree data converted by.

  The music mapping unit 16 maps music data to a music map, which is a self-organized map that has been learned in advance, based on the impression degree data input from the impression degree data conversion unit 14, and the music data is mapped to the music data The map is stored in the music map storage unit 17.

  The music map storage unit 17 is a large-capacity storage unit such as an HDD, and stores a music map to which music data is mapped by the music mapping unit 16.

  The music search unit 18 searches the music database 15 based on the impression data and bibliographic data input from the PC operation unit 19, displays the search result on the PC display unit 20, and is selected by the PC operation unit 19. The music map storage unit 17 is searched based on the representative music, and the representative music search result is displayed on the PC display unit 20. In addition, the music search unit 18 outputs the music data selected by the PC operation unit 19 via the transmission / reception unit 21 to the terminal device 30.

  In addition, the music search unit 18 reads the music data and the impression degree data from the music database 15, outputs the read music data to the audio output unit 22 and outputs the audio, and based on the instruction of the user who auditioned the audio output. The impression degree data is corrected, the impression degree data stored in the music database 15 is updated, the feature data is read from the music database 15, and the correction data and the feature data are used as relearning data in the correction data storage unit 23. Remember.

  The PC operation unit 19 is input means such as a keyboard and a mouse, inputs search conditions for searching for music data stored in the music database 15 and the music map storage unit 17, and selects music data to be output to the terminal device 30. Input, correction input of impression degree data, input for instructing automatic correction of impression degree data, input for instructing re-learning of the hierarchical neural network, and input for instructing re-registration of music data.

  The PC display unit 20 is a display unit such as a liquid crystal display, for example, displays the mapping status of the song data stored in the song map storage unit 17, and the song data stored in the song database 15 and the song map storage unit 17. The search conditions for searching for music, the searched music data (search results), etc. are displayed.

  The transmission / reception unit 21 is configured to be connectable to the transmission / reception unit 31 of the terminal device 30 by a data transmission path such as USB, and the music data searched by the music search unit 18 and selected by the PC operation unit 19 is received. While outputting to the transmission / reception part 31 of the terminal device 30, the correction instruction | indication from the terminal device 30 is received.

  The audio output unit 22 is an audio player that decompresses and reproduces music data stored in the music database 15.

  The correction data storage unit 23 is a storage unit such as an HDD that stores the corrected impression degree data and feature data as relearning data.

  The neural network learning unit 24 is a means for re-learning the hierarchical neural network used in the impression degree data conversion unit 14, reads out the connection weight value of each neuron from the impression degree data conversion unit 14, and The connection weight value is set as an initial value, and the relearning data stored in the correction data storage unit 23 is used to re-learn the hierarchical neural network, that is, the connection weight value of each neuron is re-learned. The neuron's connection weight value w is updated to the re-learned connection weight value of each neuron.

  The terminal device 30 is an audio playback device such as a portable audio or MD player having a large capacity storage means such as an HDD. Referring to FIG. 1, a transmission / reception unit 31, a search result storage unit 32, and a terminal operation unit 33 are provided. The terminal display unit 34 and the audio output unit 35 are included.

  The transmission / reception unit 31 is configured to be connectable to the transmission / reception unit 21 of the music search device 10 via a data transmission path such as a USB, and stores the music data input from the transmission / reception unit 21 of the music search device 10 as a search result. When the terminal device 30 is connected to the music search device 10, the correction instruction stored in the search result storage unit 32 is transmitted to the music search device 10.

  The terminal operation unit 33 is an input for instructing selection / reproduction of music data stored in the search result storage unit 32, an input related to reproduction of music data such as an input of volume control, and an impression degree corresponding to a music being reproduced. A correction instruction input for instructing correction of data is performed.

  The terminal display unit 34 is a display means such as a liquid crystal display, for example, and displays the name of a song being played and various operation guidance.

  The audio output unit 35 is an audio player that decompresses and reproduces music data that is compressed and stored in the search result storage unit 32.

  The neural network learning device 40 is a device that learns the hierarchical neural network used in the impression degree data conversion unit 14 and the music map used in the music mapping unit 16, and referring to FIG. 41, voice output unit 42, feature data extraction unit 43, impression degree data input unit 44, combination weight value learning unit 45, music map learning unit 46, combination weight value output unit 47, and feature vector output Part 48.

  The music data input unit 41 has a function of reading a storage medium in which music data such as a CD and a DVD is stored. The music data input unit 41 inputs music data from a storage medium such as a CD and a DVD, and extracts an audio output unit 42 and feature data. To the unit 43. You may comprise so that the music data (delivery data) via networks, such as the internet, other than storage media, such as CD and DVD, may be input. When compressed music data is input, the compressed music data is decompressed and output to the audio output unit 42 and the feature data extraction unit 43.

  The audio output unit 42 is an audio player that decompresses and reproduces music data input from the music data input unit 41.

  The feature data extraction unit 43 extracts feature data composed of fluctuation information from the song data input from the song data input unit 41 and outputs the extracted feature data to the connection weight value learning unit 45.

  The impression degree data input unit 44 accepts input of impression degree data by the evaluator based on the audio output from the audio output unit 42, and uses the received impression degree data as a joint signal as a teacher signal used for learning of the hierarchical neural network. The value is output to the value learning unit 45 and is also output to the music map learning unit 46 as an input vector to the self-organizing map.

  The connection weight value learning unit 45 performs learning on the hierarchical neural network based on the feature data input from the feature data extraction unit 43 and the impression degree data input from the impression degree data input unit 44, and The connection weight value is updated, and the updated connection weight value is output via the connection weight value output unit 47. The learned hierarchical neural network (updated connection weight value) is transplanted to the impression degree data conversion unit 14 of the music search apparatus 10.

  The music map learning unit 46 learns the self-organizing map using the impression degree data input from the impression degree data input unit 44 as an input vector to the self-organizing map, updates the feature vector of each neuron, and The updated feature vector is output via the output unit 48. The learned self-organizing map (updated feature vector) is stored in the music map storage unit 17 of the music search device 10 as a music map.

Next, the operation of the present embodiment will be described in detail with reference to FIGS.
FIG. 3 is a flowchart for explaining the music registration operation in the music search apparatus shown in FIG. 1, and FIG. 4 is a flowchart for explaining the feature data extraction operation in the feature data extraction unit shown in FIG. FIG. 5 is a flowchart for explaining the learning operation of the hierarchical neural network in the neural network learning apparatus shown in FIG. 2, and FIG. 6 explains the music map learning operation in the neural network learning apparatus shown in FIG. FIG. 7 is a flowchart for explaining the music search operation in the music search apparatus shown in FIG. 1, and FIG. 8 shows the learning algorithm of the hierarchical neural network in the neural network learning apparatus shown in FIG. FIG. 9 is an explanatory diagram for explanation, and FIG. FIG. 10 is an explanatory diagram for explaining a music map learning algorithm in the neural network learning apparatus, FIG. 10 is a diagram showing an example of a display screen of the PC display unit shown in FIG. 1, and FIG. 11 is a search shown in FIG. FIG. 12 and FIG. 13 are diagrams showing a display example of the search result display region shown in FIG. 10, and FIG. 14 is displayed on the display screen example shown in FIG. FIG. 15 is a diagram showing an example of a keyword search area displayed in the display screen example shown in FIG. 10, and FIG. 16 is a diagram of the music search system according to the present invention. FIG. 17 is a flowchart for explaining a re-learning operation of the hierarchical neural network according to the embodiment, FIG. 17 is a diagram illustrating an example of a correction screen displayed on the display screen example illustrated in FIG. 10, and FIG. FIG. 19 is a flowchart for explaining the relearning operation of the hierarchical neural network used in the impression degree data conversion unit shown in FIG. 19, and FIG. 19 shows a display example of the relearning instruction area displayed in the display screen example shown in FIG. FIG. 20 is a flowchart for explaining the re-registration operation of music data in the music search device shown in FIG.

First, the music registration operation in the music search apparatus 10 will be described in detail with reference to FIG.
A storage medium storing music data such as CD and DVD is set in the music data input section 11 and music data is input from the music data input section 11 (step A1).

  The compression processing unit 12 compresses the music data input from the music data input unit 11 (step A2), and stores the compressed music data in the music database 15 together with the bibliographic data such as artist name and music name (step A3). .

  The feature data extraction unit 13 extracts feature data including fluctuation information from the music data input from the music data input unit 11 (step A4).

  Referring to FIG. 4, the feature data extraction operation in the feature data extraction unit 13 accepts input of music data (step B1), and performs FFT (FFT) for a certain frame length from a predetermined data analysis start point of music data. (Fast Fourier transform) is performed (step B2), and a power spectrum is calculated. Note that downsampling may be performed for the purpose of speeding up before step B2.

  Next, the feature data extraction unit 13 sets the frequency bands of Low, Middle, and High in advance, integrates the power spectrum of the three bands of Low, Middle, and High to calculate the average power (step B3). Among the frequency bands of Low, Middle, and High, the band having the maximum power is set as the data analysis start point value of the pitch, and the pitch is measured (step B4).

  The processing operations in step B2 to step B4 are performed for a predetermined number of frames, and the feature data extraction unit 13 determines whether the number of frames for which the processing operations in steps B2 to B4 have been performed has reached a predetermined setting value. (Step B5), and if the number of frames for which the processing operations in steps B2 to B4 have been performed does not reach a predetermined set value, the data analysis start point is shifted (step B6). , The processing operations of Step B2 to Step B4 are repeated.

  When the number of frames subjected to the processing operations of Step B2 to Step B4 reaches a predetermined set value, the feature data extraction unit 13 calculates Low, Middle, High calculated by the processing operations of Step B2 to Step B4. The FFT is performed on the time series data of the average power and the time series data of the Pitch measured by the processing operations of Step B2 to Step B4 (Step B7).

  Next, the feature data extraction unit 13 calculates the slope of the regression line in the graph with the horizontal axis representing the logarithmic frequency and the vertical axis representing the logarithmic power spectrum from the FFT analysis results in Low, Middle, High, and Pitch, and the Y intercept of the regression line. Are calculated as fluctuation information (step B8), and the slope of the regression line and the Y-intercept in each of Low, Middle, High, and Pitch are output to the impression degree data conversion unit 14 as feature data of eight items.

The impression data conversion unit 14 uses a hierarchical neural network including an input layer (first layer), an intermediate layer (n-th layer), and an output layer (N-th layer) as shown in FIG. By inputting the feature data extracted by the feature data extraction unit 13 into the first layer), the impression data is output from the output layer (Nth layer), that is, the feature data is converted into impression data (step A5). Impression degree data output from the output layer (Nth layer) is output to the music mapping unit 16, and feature data input from the feature data extraction unit 13 and impression level output from the output layer (Nth layer) The data is stored in the music database 15 together with the music data. Note that the connection weight value w of each neuron in the intermediate layer (nth layer) is learned in advance by the evaluator. In the case of the present embodiment, the feature data input to the input layer (first layer), that is, the feature data extracted by the feature data extraction unit 13 is eight items as described above, Impression data items are determined by human sensitivity (bright, dark), (heavy, light), (hard, soft), (stable, unstable), (clear, fuzzy), ( Eight items of (smooth, crisp), (violent, gentle), (thick, thin) were set, and each item was set to be expressed by a seven-level evaluation. Therefore, the number of neurons L _N number of neurons L ₁ and the output layer of the input layer (first layer) (the N th layer) is adapted to 8 respectively, the intermediate layer (the n-th layer: n = 2, ... , N−1) is set as appropriate.

  The music mapping unit 16 maps the music input from the music data input unit 11 to a corresponding portion of the music map stored in the music map storage unit 17. The music map used for the mapping operation in the music mapping unit 16 is, for example, a self-organizing map (SOM) in which neurons are regularly arranged in two dimensions (9 * 9 square in the example shown in FIG. 9). A learning neural network that does not require a teacher signal, and is a neural network that autonomously acquires the ability to classify input pattern groups according to their similarity. In the present embodiment, a two-dimensional SOM in which neurons are arranged in a 100 * 100 square is used. However, the arrangement of neurons may be a square or a honeycomb.

The music map used for the mapping operation in the music mapping unit 16 has been learned in advance, and each neuron contains an n-dimensional feature vector m _i (t) ∈R ^{n that} has been learned in advance. cage, music mapping unit 16, the impression data converted by the impression-data-conversion unit 14 to the input vector x _j, nearest neuron to the input vector x _j, i.e., the minimum Euclidean distance ‖x _j -m _i ‖ The input music piece is mapped to the neuron to perform (step A6), and the music piece map storage unit 17 stores the mapped music map. R represents the number of evaluation stages for each item of impression degree data, and n represents the number of items of impression degree data.

Next, the learning operation of the hierarchical neural network used for the conversion operation (step A5) in the impression degree data conversion unit 14 will be described in detail with reference to FIG. 5 and FIG.
Learning of the hierarchical neural network (connection weight value w) by the evaluator is performed using, for example, the neural network learning device 40 shown in FIG. 2. First, the hierarchical neural network (connection weight value w) is pre-learned. The pre-learning data (feature data feature data + impression degree data) is input.

  A music medium such as a CD or DVD is set in the music data input unit 41, music data is input from the music data input unit 41 (step C1), and the feature data extraction unit 43 inputs the music data. Feature data composed of fluctuation information is extracted from the music data input from the unit 41 (step C2).

  The audio output unit 42 outputs the music data input from the music data input unit 41 as audio (step C3), and the evaluator listens to the audio output from the audio output unit 42 to determine the impression level of the music. Evaluation is performed based on sensitivity, and the evaluation result is input as impression degree data from the impression degree data input unit 44 (step C4). The combined weight value learning unit 45 uses the impression degree data input from the impression degree data input unit 44 as a teacher signal. Accept as. In this embodiment, the evaluation items for impression degree are determined by human sensitivity (bright, dark), (heavy, light), (hard, soft), (stable, unstable), ( 8 items of clear, sloppy), (smooth, crisp), (violent, gentle), (thick, thin) are set, and impression level data input section with 7-level evaluation for each item as impression level data 44 to accept.

Next, it is determined whether a learning data consisting of characteristic data and the inputted impression data reaches the number of samples T ₁ for a predetermined (step C5), until the learning data reaches the number of samples T ₁ Steps C1 to C4 are repeated.

The learning of the hierarchical neural network in the connection weight value learning unit 45, that is, the update of the connection weight value w of each neuron is performed using an error back propagation learning method.
First, as an initial value, the connection weight value w of all the neurons of the intermediate layer (nth layer) is set to a small value in the range of about −0.1 to 0.1 by a random number, and the connection weight value learning unit 45 inputs the feature data extracted by the feature data extraction unit 43 to the input layer (first layer) as an input signal x _j (j = 1, 2,..., 8), and from the input layer (first layer). The output of each neuron is calculated toward the output layer (Nth layer).

Next, the combined weight value learning unit 45 uses the impression degree data input from the impression degree data input unit 44 as a teacher signal y _j (j = 1, 2,..., 8), and outputs the output layer (Nth layer). The learning rule δ _j ^N is calculated from the error between the output out _j ^N and the teacher signal y _j by the following equation.

Next, the joint weight value learning unit 45 calculates the error signal δ _j ⁿ of the intermediate layer (nth layer) using the learning rule δ _j ^N by the following equation.

  In Equation 2, w represents a connection weight value between the n-th layer j-th neuron and the (n −1) -th layer k-th neuron.

Next, the connection weight value learning unit 45 calculates the change amount Δw of the connection weight value w of each neuron by the following equation using the error signal δ _j ⁿ of the intermediate layer (nth layer), and the connection weight of each neuron. The value w is updated (step C6). In the following equation, η represents a learning rate, and is set to η ₁ (0 <η ₁ ≦ 1) in learning by the evaluator.

In step C6, for each of the pre-training data sample number T ₁ learning is performed, then, or smaller or not than the reference value E ₁ for pre-learning is square error E shown in the following equation predetermined There is judged (step C7), the operation of step C6 to square error E is smaller than the reference value E ₁ is repeated. Incidentally, the learning iterations S squared error E is assumed to be smaller than the reference value E ₁ is set in advance, may be the operation of the step C6 to repeat the learning iterations S times.

In the case of the square error E is determined to be smaller than the reference value E ₁ Step C7, connection weights learning unit 45, the connection weights output unit 47 the coupling weight value w for each neuron is prior learning The connection weight value w of each neuron outputted and output from the connection weight value output unit 47 is stored in the impression degree data conversion unit 14.

  Next, the music map learning operation used for the mapping operation (step A6) in the music mapping unit 16 will be described in detail with reference to FIGS.

  The music data input unit 41 is set with a storage medium storing music data such as CD and DVD, the music data is input from the music data input unit 41 (step D1), and the audio output unit 42 is a music data input unit. The music data input from 41 is output as audio (step D2), and the evaluator listens to the audio output from the audio output unit 42 to evaluate the impression level of the music based on the sensitivity, and the evaluation result is used as impression data. Input from the impression level data input unit 44 (step D3), the music map learning unit 46 accepts the impression level data input from the impression level data input unit 44 as an input vector to the self-organizing map. In this embodiment, the evaluation items for impression degree are determined by human sensitivity (bright, dark), (heavy, light), (hard, soft), (stable, unstable), ( 8 items of clear, sloppy), (smooth, crisp), (violent, gentle), (thick, thin) are set, and impression level data input section with 7-level evaluation for each item as impression level data 44 to accept.

The music map learning unit 46 uses the impression degree data input from the impression degree data input unit 44 as an input vector x _j (t) ∈R ⁿ and learns a feature vector m _i (t) ∈R ⁿ of each neuron. Let Note that t represents the number of learning times, a preset value T that determines the number of learning times is set in advance, and learning is performed for the learning number t = 0, 1,. Note that R indicates the evaluation stage of each impression degree item, and n indicates the number of items of impression degree data.

First, as an initial value, feature vectors m _c (0) of all neurons are set at random in the range of 0 to 1, and the music map learning unit 46 determines that the neuron c, which is closest to x _j (t), That is, a winner neuron c that minimizes ‖x _j (t) −m _c (t) ‖ is obtained, and a feature vector m _c (t) of the winner neuron c and a set Nc of neighboring neurons i in the vicinity of the winner neuron c Each feature vector m _i (t) (iεNc) is updated according to the following equation (step D4). It is assumed that the neighborhood radius for determining the neighborhood neuron i is set in advance.

In Equation 5, h _ci (t) represents a learning rate and is obtained by the following equation.

Α _init is an initial value of the learning rate, and R ² (t) is a monotonically decreasing linear function or exponential function.

Next, the music map learning unit 46 determines whether or not the learning count t has reached the set value T (step D5), and performs the processing operations of steps D1 to D4 until the learning count t reaches the set value T. Repeatingly, when the learning count t reaches the set value T, the same processing operation is performed again from the first sample. When the number of iterations reaches a predetermined number of times S, the feature vector m _i (T) εR ⁿ learned through the feature vector output unit 48 is output (step D6). The output feature vector m _i (T) of each neuron i is stored as a music map in the music map storage unit 17 of the music search device 10.

Next, the music search operation in the music search apparatus 10 will be described in detail with reference to FIG.
The music search unit 18 displays a search screen 50 as shown in FIG. 10 on the PC display unit 20 and accepts user input from the PC operation unit 19. The search screen 50 includes a music map display area 51 in which the mapping status of music data stored in the music map storage unit 17 is displayed, a search condition input area 52 for inputting search conditions, and a search in which search results are displayed. The result display area 53 and a relearning instruction area 70 for instructing relearning of the hierarchical neural network. The points shown in the music map display area 51 in FIG. 10 indicate the neurons of the music map to which the music data is mapped.

  As shown in FIG. 11, the search condition input area 52 has an impression degree data input area 521 for inputting impression degree data as a search condition, a bibliographic data input area 522 for input of bibliographic data as a search condition, and executes the search. The search execution button 523 for instructing the user, the user inputs impression degree data and bibliographic data as search conditions from the PC operation unit 19 (step E1), and clicks the search execution button 523, whereby the impression degree data and bibliography are clicked. A search based on data is instructed to the music search unit 18. Note that the impression data is input from the PC operation unit 19 by inputting each item of the impression data in a seven-step evaluation as shown in FIG.

  The music search unit 18 searches the music database 15 based on the impression data and bibliographic data input from the PC operation unit 19 (step E2), and displays the search results as shown in FIG. To do.

The search based on the impression degree data input from the PC operation unit 19 uses the impression degree data input from the PC operation unit 19 as the input vector _xj, and searches the impression degree data stored together with the song data in the song database 15. If the target vector _{X j,} the input vectors _{x j} closer search target vector _{X j,} i.e. go search the Euclidean distance ‖X _j -x _j || is ascending order. The number of searches may be determined in advance or arbitrarily set by the user. If both impression level data and bibliographic data are set as search conditions, after searching based on bibliographic data, searching based on impression level data is performed.

  In addition to the search using the search condition input area 52, a search using the music map display area 51 may be performed. In this case, by designating the search target area in the music map display area 51, the music data mapped in the search target area is displayed in the search result display area 53 as a search result.

  Next, the user selects a representative song from the search results displayed in the search result display area 53 (step E3), and clicks the representative song search execution button 531 to perform a search based on the representative song. The search unit 18 is instructed. When the output button 532 is clicked at this time, the music search unit 18 outputs the music data of the search result displayed in the search result display area 53 to the terminal device 30 via the transmission / reception unit 21. .

  The music search unit 18 searches the music map stored in the music map storage unit 17 based on the selected representative music (step E4), and maps it to the neuron to which the representative music is mapped and its neighboring neurons. The stored music data is displayed in the search result display area 53 as a representative music search result. The neighborhood radius for determining the neighborhood neuron may be set in advance or arbitrarily set by the user.

  Next, the user selects music data to be output to the terminal device 30 from the representative music search results displayed in the search result display area 53 as shown in FIG. 13 (step E5), and the output button 532 is selected. By clicking, the music search unit 18 is instructed to output the selected music data, and the music search unit 18 outputs the music data selected by the user via the transmission / reception unit 21 to the terminal device 30 (step E6). .

  In addition to the search for representative songs using the search condition input area 52 and the music map display area 51, a search is made for an all music list display area 54 in which a list of all stored music is displayed as shown in FIG. It is configured to display on the screen 50 and directly select a representative song from the entire song list and click the representative song selection execution button 541 to instruct the music search unit 18 to search based on the selected representative song. May be.

In addition to the search described above, neurons (or songs) corresponding to keywords expressed in words such as “bright songs”, “fun songs”, and “healed songs” are set and keywords are selected. It may be configured so that music can be searched. That is, as shown in FIG. 15A, a keyword search area 55 is displayed on the search screen 50, one is selected from the keyword list displayed in the keyword selection area 551, and the automatic search button 553 is clicked. Thus, the music search unit 18 is instructed to search based on the neuron corresponding to the selected keyword. In the set music display area 552 shown in FIG. 15A, when a music corresponding to the selected keyword is set, the music is displayed as the set music. In this case, an automatic search button 553 is displayed. By clicking, the music search unit 18 is instructed to search for the set music corresponding to the selected keyword as a representative music. The set music change button 554 shown in FIG. 15A is used when changing the music corresponding to the keyword. When the set music change button 554 is clicked, the entire music list is displayed. The music corresponding to the keyword can be changed by selecting the music from the entire music list. The setting of neurons (or songs) that correspond to the keywords in advance by assigning impression data to the keyword, the impression data as input vectors x _j, the nearest neuron to the input vector x _j (or music) You may make it match | combine and you may comprise so that it can set arbitrarily by a user.

  Thus, when the neuron corresponding to the keyword is set, as shown in FIG. 15B, when the neuron to which the music is mapped is clicked in the music map display area 51, the clicked neuron is displayed. If the corresponding keyword is configured to be pop-up displayed as the keyword display 511, it is possible to easily search for a song using the song map display area 51.

  Next, the impression data correction operation in the music search device will be described in detail with reference to FIGS. 16 and 17.

  As shown in FIGS. 12 and 13, the search result display area 53 is provided with a test listening button 533 for each searched music. When the test listening button 533 is clicked by the PC operation unit 19 (step F <b> 1), the music is displayed. The search unit 18 reads the music data and impression degree data of the corresponding music from the music database 15, outputs the read music data to the audio output unit 22, and displays the correction instruction area 60 shown in FIG. 17 on the PC display unit 20. The impression degree data read and displayed in the correction data input area 61 is displayed (step F2). In the impression degree data displayed in the correction data input area 61, as shown in FIG. 17, the stage for each item is expressed by the position of the point.

  The audio output unit 22 outputs the music data input from the music search unit 18 by voice (step F3), and the user auditiones the music searched based on the impression degree data and bibliographic data and selects a representative music. In addition, the music searched based on the representative music can be auditioned and the music output to the terminal device 30 can be confirmed. Further, when the audio output stop button 64 in the correction instruction area 60 is clicked by the PC operation unit 19 during the trial listening, the music search unit 18 stops outputting the music data to the audio output unit 22 and stops the audio output. The display of the correction instruction area 60 is erased.

  When the user feels uncomfortable with the sound output from the sound output unit 22, that is, when the impression degree data displayed in the correction data input area 61 is different, or when the user feels that the search result is inappropriate The impression degree data can be corrected by one of the following two methods.

  The first method of correcting the impression degree data is that after the correction input of the impression degree data displayed in the correction data input area 61 by the PC operation unit 19, that is, the position of the point for each item is moved (step F4). ) The correction execution button 62 is clicked (step F5).

  When the correction execution button 62 is clicked, the corrected impression degree data (hereinafter, the corrected impression degree data is referred to as correction data) is input to the music search unit 18, and the music search unit 18 is stored in the music database 15. The impression data is updated to the input correction data, the feature data is read from the music database 15, and the correction data and the feature data are stored in the correction data storage unit 23 as relearning data (step F6).

  In the second method of correcting the impression degree data, the automatic correction button 63 is clicked by the PC operation unit 19. When the automatic correction button 63 is clicked (step F7), a correction instruction is input to the music search unit 18, and the music search unit 18 automatically corrects the impression degree data of the music instructed to be corrected away from the search condition. (Step F8), the impression degree data stored in the music database 15 is updated to the corrected data automatically corrected, the feature data is read from the music database 15, and the corrected data and the feature data are corrected as the relearning data. The data is stored in the data storage unit 23 (step F6).

  The automatic correction by the music search unit 18 is executed when the search is performed based on the impression degree data input to the impression degree data input area 521 or the impression degree data of the representative song, and the impression degree data as a search condition is changed. The most characteristic item is identified, and the stage of the item is moved in a direction away from the predetermined number. For example, if the impression degree data as the search condition is the one shown in the impression degree data input area 521 in FIG. 11, the item indicating the stage of (bright, dark) is the brightest evaluation, and therefore (bright The item indicating the stage of (dark) is identified as the most characteristic item, and the stage of (bright, dark) is moved in the dark direction. If the search is based on bibliographic data, the second method cannot be selected. In this case, control is performed so that the automatic correction button 63 cannot be clicked, or from the correction instruction area 60. The automatic correction button 63 may be deleted.

  When new correction data is stored in the music map storage unit 17, the music mapping unit 16 remaps the music with the correction data (step F9), and the music map that has been remapped based on the correction data is stored in the music map storage unit. 17 to memorize.

Next, the impression data correction operation in the terminal device 30 will be described in detail.
When the user feels uncomfortable while listening to the music on the terminal device 30, that is, when the user feels that the search result is inappropriate, a correction instruction input for instructing correction of the impression degree data corresponding to the music being played back is input. This is performed by the terminal operation unit 33. The correction instruction input is performed, for example, by providing a dedicated button for inputting the correction instruction to the terminal operation unit 33 and pressing the button during the reproduction of the music. Instead of the dedicated button, a correction instruction input function may be assigned to any button during the reproduction of the music.

  The correction instruction input by the terminal operation unit 33 is stored in the search result storage unit 32, and is transmitted to the music search device 10 by the transmission / reception unit 31 when the terminal device 30 is connected to the music search device 10. The correction instruction from the terminal operation unit 33 is received by the transmission / reception unit 21 of the music search device 10, and the received correction instruction is output to the music search unit 18.

  The music search unit 18 to which the correction instruction is input automatically corrects the impression degree data of the music for which the correction is instructed in a direction away from the search condition, and automatically corrects the impression degree data stored in the music database 15. And the feature data is read from the music database 15, and the correction data and the feature data are stored in the correction data storage unit 23 as relearning data.

Next, the relearning operation of the hierarchical neural network used in the impression degree data conversion unit 14 will be described in detail with reference to FIG.
The neural network learning unit 24 counts the number of re-learning data newly stored in the correction data storage unit 23 (step G1), and the number of re-learning data newly stored in the correction data storage unit 23 is It is determined whether or not the predetermined number has been reached (step G2), and when the number of re-learning data newly stored in the correction data storage unit 23 reaches the predetermined number, the impression data conversion unit 14 connects each neuron. The weight value w is read (step G3), the connection weight value w of each read neuron is set as an initial value, and the relearning data stored in the correction data storage unit 23 is used to re-learn the hierarchical neural network, that is, each neuron. The connection weight value w is relearned (step G4). Note that the number of relearning data for starting relearning of the hierarchical neural network, that is, a predetermined number, may be determined in advance or may be set by the user. In addition, the elapsed time from the last time the relearning of the hierarchical neural network is measured, and when the elapsed time reaches a predetermined time, the relearning of the hierarchical neural network may be started. The elapsed time for starting relearning of the neural network, that is, the predetermined time may be determined in advance or may be set by the user.

  Re-learning of the hierarchical neural network in the neural network learning unit 24 is performed by a method similar to the learning method performed by the neural network learning device 40, and the neural network learning unit 24 performs each neuron of the impression degree data conversion unit 14. The connection weight value w is updated to the connection weight value w of each neuron that has been relearned (step G5). Note that the relearning data used for relearning may be deleted from the correction data storage unit 23. However, if the relearning data is stored in the correction data storage unit 23 and used for the next relearning, Since the number of re-learning data used at the time of re-learning of the neural network increases, the accuracy of re-learning is improved. However, when re-learning data used for re-learning is stored in the correction data storage unit 23, when new correction data for the same music is stored, the previous correction data is deleted and the same It is necessary to prevent the two correction data from existing for the music.

  The re-learning operation of the hierarchical neural network by the neural network learning unit 24 can be performed in a time-sharing manner so as not to disturb other processing such as music registration operation and music search operation. In other words, when another process is started during the relearning of the hierarchical neural network by the neural network learning unit 24, the relearning operation is interrupted and the relearning operation is resumed after the other process is completed. Yes. In addition, the re-learning operation of the hierarchical neural network by the neural network learning unit 24 may be performed during idling when starting up the music search apparatus 10 or during termination processing when disconnecting.

  Note that the relearning of the hierarchical neural network can be performed according to a user instruction. As shown in FIG. 19, the re-learning instruction area 70 of the search screen 50 includes a correction information display area 71, a re-learning execution button 72, and a re-registration execution button 73. The number of correction data stored in the data storage unit 23, the elapsed time since the last re-learning of the hierarchical neural network, and the elapsed time since the last re-registration of the music data are displayed. Is done. The number of correction data displayed in the correction information display area 71 includes the number of correction data newly stored (correction data not used for relearning) and corrections that have been used for relearning. The number of data is displayed, and the number of corrected data that has been used for relearning is displayed in parentheses.

  When the user confirms the information displayed in the correction information display area 71 and determines that it is necessary to re-learn the hierarchical neural network, the user clicks the re-learn execution button 72 by the PC operation unit 19. . When the re-learning execution button 72 is clicked, a re-learning instruction is output to the neural network learning unit 24, and the neural network learning unit 24 reads the connection weight value w of each neuron from the impression degree data conversion unit 14 and reads it out. Using the connection weight value w of each neuron as an initial value, the relearning data stored in the correction data storage unit 23 is used for relearning of the hierarchical neural network, that is, the connection weight value w of each neuron is relearned, and impression degree data The connection weight value w of each neuron in the conversion unit 14 is updated to the connection weight value w of each neuron that has been relearned.

Next, the music data re-registration operation in the music search device 10 will be described in detail with reference to FIG.
When the re-registration execution button 73 is clicked by the PC operation unit 19 in the correction information display area 71, the re-registration operation of all the music data stored in the music database 15 is started.

  When the re-registration execution button 73 is clicked by the PC operation unit 19 (step H1), a re-registration instruction is output to the neural network learning unit 24, and the neural network learning unit 24 outputs each neuron from the impression degree data conversion unit 14. The connection weight value w is read (step H2), the read connection weight value w of each neuron is set as an initial value, and the relearning data stored in the correction data storage unit 23 is used to relearn the hierarchical neural network, that is, each neuron. Are re-learned (step H3), and the connection weight value w of each neuron of the impression degree data converter 14 is updated to the re-learned connection weight value w of each neuron (step H4).

  Next, the neural network learning unit 24 instructs the music mapping unit 16 to delete all the music mapped in the music map, and the music mapping unit 16 stores the music stored in the music map storage unit 17. All the music pieces mapped on the map are deleted (step H5).

  Next, the neural network learning unit 24 instructs the impression degree data conversion unit 14 to update the impression degree data stored in the music database 15, and the impression degree data conversion unit 14 is stored in the music database 15. The feature data of the music data being read is read (step H6), and the read feature data is converted into impression degree data using the relearned hierarchical neural network (step H7), and the music stored in the music database 15 is stored. The impression level data of the data is updated (step H8), and the converted impression level data is output to the music mapping unit 16. The music mapping unit 16 re-maps the music based on the updated impression degree data input from the impression degree data conversion unit 14 (step H9).

  The neural network learning unit 24 determines whether or not there is music data whose impression degree data is not updated (step H10), and when there is music data whose impression degree data is not updated, step H6 is performed. When the process of Step H9 is repeated and there is no music data whose impression degree data is not updated, that is, when the impression degree data of all the music data stored in the music database 15 is updated, End the re-registration operation.

  As described above, according to the present embodiment, the impression degree data conversion unit 14 performs the characteristic data including a plurality of physical items included in the music and the impression degree data including items determined by human sensitivity. The feature data extracted from the music data is converted into impression data using a hierarchical neural network that directly associates the music data with each other, and the converted impression data is stored in the music database 15. Impression degree data determined by human sensitivity inputted as a search condition by the user by configuring the impression degree data stored in the music database 15 to be searched based on the impression degree data inputted by the user. Based on impression degree data that is judged by human sensitivity without being aggregated The music data can be searched with high accuracy, and only the music with the same impression when listening from a large amount of music data stored in the large-capacity storage means can be searched. Since the impression degree data of the music data is automatically corrected based on the correction instruction received from the terminal device 30, the terminal device 30 with limited operation buttons is being reproduced simply by inputting the correction instruction. Impression degree data corresponding to the music data of the user can be corrected, and the number of music data searched against the user's intention can be reduced and the search accuracy can be improved as the usage period becomes longer. There is an effect.

  Furthermore, according to the present embodiment, the feature data extraction unit 13 is configured to extract a plurality of items of fluctuation information as feature data, thereby accurately extracting the physical features of the music data. It is possible to improve the accuracy of the impression degree data converted from the feature data.

  Furthermore, according to the present embodiment, the impression degree data converted from the feature data by the impression degree data conversion unit 14 and the impression degree data input from the PC operation unit 19 are made into the same plurality of items, Various items can be set, and the user can easily perform a search based on the impression degree data.

  Furthermore, according to the present embodiment, the music search unit 18 uses the impression degree data input from the PC operation unit 19 as an input vector and the impression degree data stored in the music database 15 as a search target vector. By configuring to search in order of increasing Euclidean distance between the two, it is possible to perform an accurate search even if the number of impression degree data items is increased, and it is possible to improve the search accuracy.

  Further, according to the present embodiment, the music map storage unit 17 is a music map that is a self-organizing map that has been learned in advance, and in which the music data is mapped based on the impression degree data of the music data. The music search unit 18 is configured to search using the music map stored in the music map storage unit 17, so that the representative song is selected and stored in a large-capacity storage unit. It is possible to quickly search for a song having the same impression as the representative song from a large amount of song data that has been recorded, and to automatically correct the impression data of the song data based on the correction instruction received from the terminal device 30. With such a configuration, in the terminal device 30 with limited operation buttons, the impression degree data corresponding to the music data being played can be simply input by inputting a correction instruction. Modifications can be performed, it is possible to reduce the number of music data use period is searched against a user's intention as longer, an effect that it is possible to improve search accuracy.

  Furthermore, according to the present embodiment, by using a self-organizing map that has been learned in advance as a music map, music having a similar impression is arranged in the vicinity, so that the search efficiency is improved. Play.

  Note that the present invention is not limited to the above-described embodiments, and it is obvious that the embodiments can be appropriately changed within the scope of the technical idea of the present invention. In addition, the number, position, shape, and the like of the constituent members are not limited to the above-described embodiment, and can be set to a suitable number, position, shape, and the like in practicing the present invention. In each figure, the same numerals are given to the same component.

It is a block diagram which shows the structure of embodiment of the music search system which concerns on this invention. It is a block diagram which shows the structure of the neural network learning apparatus which learns in advance the neural network used for the music search apparatus shown in FIG. It is a flowchart for demonstrating the music registration operation | movement in the music search apparatus shown in FIG. It is a flowchart for demonstrating the feature data extraction operation | movement in the feature data extraction part shown in FIG. 3 is a flowchart for explaining a learning operation of a hierarchical neural network in the neural network learning apparatus shown in FIG. 3 is a flowchart for explaining a music map learning operation in the neural network learning apparatus shown in FIG. 2. It is a flowchart for demonstrating the music search operation | movement in the music search apparatus shown in FIG. It is explanatory drawing for demonstrating the learning algorithm of the hierarchical neural network in the neural network learning apparatus shown in FIG. It is explanatory drawing for demonstrating the learning algorithm of the music map in the neural network learning apparatus shown in FIG. It is a figure which shows the example of a display screen of the PC display part shown in FIG. It is a figure which shows the example of a display of the search condition input area shown in FIG. It is a figure which shows the example of a display of the search result display area shown in FIG. It is a figure which shows the example of a display of the search result display area shown in FIG. It is a figure which shows the example of all the music list display areas displayed on the example of a display screen shown in FIG. It is a figure which shows the example of a keyword search area | region displayed on the example of a display screen shown in FIG. It is a flowchart for demonstrating the relearning operation | movement of the hierarchical neural network of embodiment of the music search system which concerns on this invention. It is a figure which shows the example of a correction screen displayed on the example of a display screen shown in FIG. It is a flowchart for demonstrating the relearning operation | movement of the hierarchical neural network used by the impression degree data conversion part shown in FIG. It is a figure which shows the example of a display of the relearning instruction | indication area | region displayed on the example of a display screen shown in FIG. It is a flowchart for demonstrating the re-registration operation | movement of the music data in the music search apparatus shown in FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Music search apparatus 11 Music data input part 12 Compression processing part 13 Feature data extraction part 14 Impression degree data conversion part 15 Music database 16 Music mapping part 17 Music map memory | storage part 18 Music search part 19 PC operation part 20 PC display part 21 Transmission / reception Unit 22 voice output unit 23 correction data storage unit 24 neural network learning unit 30 terminal device 31 transmission / reception unit 32 search result storage unit 33 terminal operation unit 34 terminal display unit 35 voice output unit 40 neural network learning device 41 music data input unit 42 voice Output unit 43 Feature data extraction unit 44 Impression degree data input unit 45 Joint weight value learning unit 46 Music map learning unit 47 Joint weight value output unit 48 Feature vector output unit 50 Search screen 51 Music map display region 52 Search condition input region 53 Search Result display area 54 All music list display area 55 Keyword search area 60 Correction instruction area 61 Correction data input area 62 Correction execution button 63 Automatic correction button 64 Audio output stop button 70 Relearning instruction area 71 Correction information display area 72 Relearning execution button 73 Re-registration Execute button 511 Keyword display 521 Impression degree data input area 522 Bibliographic data input area 523 Search execution button 531 Representative song search execution button 532 Output button 533 Audition button 541 Representative song selection execution button 551 Keyword selection area 552 Setting music display area 553 Automatic search Button 554 Setting music change button

Claims (24)

A music search system that searches for desired music data from a plurality of music data stored in a music database and outputs the searched music data to a terminal device,
Music data input means for inputting music data;
Feature data extraction means for extracting physical feature data from the music data input by the music data input means;
Impression degree data converting means for converting the feature data extracted by the feature data extracting means into impression degree data determined by human sensitivity;
Storage control means for storing the impression degree data converted by the impression degree data conversion means in the music database together with the music data input by the music data input means;
Impression degree data input means for inputting the impression degree data as a search condition;
Music search means for searching the music database based on the impression data input from the impression data input means;
Music data output means for outputting the music data searched by the music search means to the terminal device;
Correction instruction receiving means for receiving a correction instruction for the impression degree data from the terminal device;
Based on the correction instruction received by the correction instruction receiving means, correction data is created by automatically correcting the impression degree data in a direction away from the search condition, and the impression stored in the music database by the correction data. And an impression degree data updating means for updating degree data.   The impression degree data converting means converts the feature data extracted by the feature data extracting means into the impression degree data determined by human sensitivity using a hierarchical neural network that has been learned in advance. The music search system according to claim 1, wherein:   The music search system according to claim 1, wherein the feature data extraction unit extracts a plurality of items including fluctuation information as the feature data.   The music search means uses the impression degree data input from the impression degree data input means as an input vector, uses the impression degree data stored in the music database as a search target vector, and the Euclidean distance between them is The music search system according to any one of claims 1 to 3, wherein search is performed in ascending order. A music search system that searches for desired music data from a plurality of music data stored in a music database and outputs the searched music data to a terminal device,
Music data input means for inputting music data;
Feature data extraction means for extracting physical feature data from the music data input by the music data input means;
Impression degree data converting means for converting the feature data extracted by the feature data extracting means into impression degree data determined by human sensitivity;
A music mapping means for mapping music data input by the music data input means to a music map which is a self-organized map that has been previously learned based on the impression data converted by the impression data conversion means. When,
Music map storage means for storing music data mapped by the music mapping means;
Representative song selection means for selecting a representative song from the song data mapped to the song map;
Music search means for searching a music map based on the representative music selected by the representative music selection means;
Music data output means for outputting the music data searched by the music search means to the terminal device;
Correction instruction receiving means for receiving a correction instruction for the impression degree data from the terminal device;
Impression degree data correction means for creating correction data obtained by automatically correcting the impression degree data in a direction away from the representative song based on the correction instruction received by the correction instruction reception means,
The music search system, wherein the music mapping means re-maps music data to the music map based on the correction data created by the impression degree data correction means.   The impression degree data converting means converts the feature data extracted by the feature data extracting means into the impression degree data determined by human sensitivity using a hierarchical neural network that has been learned in advance. The music search system according to claim 5, wherein:   7. The music search system according to claim 5 or 6, wherein the feature data extraction means extracts a plurality of items including fluctuation information as the feature data.   The music mapping means uses the impression degree data converted by the impression degree data conversion means as an input vector, and maps the music data input by the music data input means to a neuron closest to the input vector. The music search system according to any one of claims 5 to 7.   9. The music search system according to claim 5, wherein the music search means searches for music data included in a neuron to which a representative music is mapped.   The music search system according to any one of claims 5 to 9, wherein the music search means searches for music data contained in a neuron to which a representative music is mapped and its neighboring neurons.   The music search system according to claim 10, wherein a neighborhood radius for determining a nearby neuron in the music search means can be arbitrarily set. A terminal device configured to be connectable to a music search device that searches for desired music data from a plurality of music data stored in a music database,
Search result input means for inputting music data from the music search device;
Search result storage means for storing music data input by the search result input means;
Audio output means for reproducing the music data stored in the search result storage means;
A correction instruction input means for inputting a correction instruction for impression degree data corresponding to the music data being reproduced at the time of reproduction of the music data by the audio output means;
A terminal device, comprising: a correction instruction transmission unit that transmits the correction instruction input by the correction instruction input unit to the music search device when connected to the music search device. A music search method for searching for desired music data from a plurality of music data stored in a music database and outputting the searched music data to a terminal device,
Accept music data input,
Extract physical feature data from the input music data,
Converting the extracted feature data into impression degree data determined by human sensitivity,
Storing the converted impression data together with the input music data in the music database;
Accept the input of impression degree data as a search condition,
Search the music database based on the input impression data,
Outputting the searched music data to the terminal device;
Receiving a correction instruction of the impression degree data from the terminal device;
Based on the received correction instruction, create correction data that automatically corrects the impression degree data in a direction away from the search condition,
A music search method, wherein the impression degree data stored in the music database is updated by the created correction data.   14. The music search method according to claim 13, wherein the extracted feature data is converted into the impression degree data determined by human sensitivity using a hierarchical neural network that has been learned in advance.   The music search method according to claim 13 or 14, wherein a plurality of items including fluctuation information are extracted as the feature data.   14. The received impression degree data is used as an input vector, and the impression degree data stored in the music database is used as a search target vector, and a search is performed in ascending order of both Euclidean distances. The music search method according to any one of 15. A music search method for searching for desired music data from a plurality of music data stored in a music database and outputting the searched music data to a terminal device,
Accept music data input,
Extract physical feature data from the input music data,
Converting the extracted feature data into impression degree data determined by human sensitivity,
Based on the converted impression degree data, the input music data is mapped to a music map that is a self-organizing map that has been previously learned,
Select a representative song from the song data mapped to the song map,
Search music data mapped to the music map based on the selected representative music,
Outputting the searched music data to the terminal device;
Receiving a correction instruction of the impression degree data from the terminal device;
Based on the received correction instruction, create correction data that automatically corrects the impression degree data in a direction away from the representative song,
A music search method comprising re-mapping music data on the music map based on the created correction data.   18. The music search method according to claim 17, wherein the extracted feature data is converted into the impression degree data determined by human sensitivity using a hierarchical neural network that has been learned in advance.   The music search method according to claim 17 or 18, wherein a plurality of items including fluctuation information are extracted as the feature data.   20. The music search method according to claim 17, wherein the converted music impression data is used as an input vector, and the input music data is mapped to a neuron closest to the input vector.   21. The music search method according to claim 17, wherein music data included in a neuron to which the representative music is mapped is searched.   The music search method according to any one of claims 17 to 21, wherein music data included in a neuron to which the representative music is mapped and its neighboring neurons are searched.   23. The music search method according to claim 22, wherein a neighborhood radius for determining a neighborhood neuron can be arbitrarily set. A music search program for causing a computer to execute the music search method according to any one of claims 13 to 23.

Images