JP2006031637A - Musical piece retrieval system and musical piece retrieval method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a musical piece retrieval system and a musical piece retrieval method, allowing a strong reflection of taste of a user even if using a hierarchical neural network previously applied with advance learning by an evaluator so as to reduce a work amount of the user for applying learning to the hierarchical neural network. <P>SOLUTION: A plurality of advance learning completion hierarchical neural networks are stored in a learning data storage part 23, one of the plurality of advance learning completion hierarchical neural networks is made to be selected by an initial setting part 22, the advance learning completion hierarchical neural network selected by a hierarchical neural network learning part 25 is made to reflect the taste of the user and is made to be learned, and physical characteristic data of a musical piece and impression degree data decided by human sensitivity are directly associated by the hierarchical neural network made to perform the learning by the hierarchical neural network learning part 25. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a music search system and a music search method for searching for desired music from music data stored in large quantities in a large-capacity storage means such as an HDD, and in particular, impression degree data determined by human sensitivity. The present invention relates to a music search system and a music search method capable of searching music based on the above.

  In recent years, a large-capacity storage means such as an HDD has been developed, and a large amount of music data can be stored in the large-capacity storage means. Searching for a large amount of music data stored in a large-capacity storage means is generally performed using bibliographic data such as artist names, music titles, and other keywords. This is not suitable when it is not possible to take into account the emotions of a music piece and there is a possibility that a music piece with a different impression will be searched, and it is desired to search for a music piece with a similar impression when listening.

  Therefore, in order to make it possible to search for the music desired by the user based on the subjective impression of the music, the user's subjective requirements for the music desired to be searched are input, quantified and output, and the output , A predicted impression value obtained by quantifying the impression of the music to be searched is calculated, and a music database storing an acoustic value of a plurality of music and an impression value obtained by quantifying the impression of the music is calculated using the calculated predicted impression value as a key. There has been proposed an apparatus for searching for desired music based on a subjective image of a user's music by searching (for example, see Patent Document 1).

However, in the prior art, since the impression value obtained by converting the physical characteristics of the music is searched based on the predicted impression value obtained by quantifying the input of the user's subjective requirements, the search condition is set by the user as a search condition. Input items of subjective requirements that are input are aggregated, and it is not possible to achieve high-accuracy music data search based on subjective requirements, and rules that convert physical characteristics of music into impression values Is fixed, there is a problem that the impression value after conversion does not necessarily match the preference of each user.
JP 2002-278547 A

  The present invention has been made in view of such problems, and an object of the present invention is to use a hierarchical neural network that reflects user's preference to learn from a plurality of physical items that a song has. High-precision music based on impression data determined by human sensitivity input as a search condition by a user by directly associating feature data and impression data consisting of items determined by human sensitivity Users can also search for data and use the hierarchical neural network that has been pre-learned by the evaluator in order to reduce the user's workload for learning the hierarchical neural network. Is to provide a music search system and a music search method capable of strongly reflecting the music.

In order to solve the above problems, the present invention has the following configurations.
The music search system of the present invention is a music search system for storing a plurality of music data in a music database and searching for the desired music data from the plurality of music data stored in the music database. Learning data storage means in which a plurality of hierarchical neural networks for converting physical feature data of the music data, which has been pre-learned by a person, into impression degree data determined by human sensitivity, are stored; Initial setting means for selecting one of a plurality of hierarchical neural networks stored in the learning data storage means, audio output means for outputting the music data as audio, and the music data output as audio from the audio output means The impression degree data input means for inputting the impression degree data corresponding to the The hierarchical neural network learning means for learning the hierarchical neural network using the feature data included in the music data output from the voice output means and the impression degree data input from the impression degree data input means Learning is performed by music data input means for inputting the music data, feature data extraction means for extracting the feature data from the music data input by the music data input means, and the hierarchical neural network learning means. Impression degree data conversion means for converting the feature data extracted by the feature data extraction means into the impression degree data using the hierarchical neural network thus formed, and the impression degree converted by the impression degree data conversion means Data by the music data input means Storage control means for storing in the music database together with the received music data, music search means for searching the music database based on the impression degree data input as a search condition, and search performed by the music search means And music data output means for outputting the music data.

  Furthermore, the music search system of the present invention includes personal information input means for inputting personal information of a user, and includes a plurality of hierarchical neural networks that are pre-learned by different evaluators in the learning data storage means. The initial setting means stores a plurality of hierarchies stored in the learning data storage means by comparing the personal information of the user with the personal information of the evaluator. One type of neural network is selected.

  Furthermore, the music search system of the present invention comprises a genre input means for inputting the genre of the music data, and a plurality of hierarchical neural networks each pre-trained by the music data of different genres in the learning data storage means And the initial setting means selects any one of a plurality of hierarchical neural networks stored in the learning data storage means based on the genre input by the genre input means. To do.

The music search method of the present invention is a music search method for storing a plurality of music data in a music database, and searching for the desired music data from the plurality of music data stored in the music database. A plurality of hierarchical neural networks that have been pre-learned by a person and convert physical feature data of the music data into impression degree data determined by human sensitivity, Select one of a plurality of hierarchical neural networks, output the music data as audio, accept input of the impression data corresponding to the audio data output, and select the selected hierarchical neural network as the audio Learning using the feature data of the outputted music data and the received impression data And receiving the input of the music data, extracting the feature data from the received music data, and converting the extracted feature data into the impression degree data using the learned hierarchical neural network. , Storing the converted impression degree data in the music database together with the received music data,
The input of the impression level data is received as a search condition, the music database is searched based on the received impression level data, and the searched music data is output.

  Furthermore, the music search method of the present invention receives a user's personal information input, stores a plurality of hierarchical neural networks that have been pre-learned by the evaluator, together with the evaluator's personal information, One of the plurality of stored hierarchical neural networks is selected by comparing the received personal information of the user and the personal information of the evaluator.

  Further, the music search method of the present invention receives an input of a genre of the music data, stores a plurality of hierarchical neural networks that are pre-learned by music data of different genres, and stores the received genre One of the plurality of stored hierarchical neural networks is selected based on the above.

  The music search system and the music search method according to the present invention include a plurality of hierarchical neural networks that convert physical feature data of music data that has been pre-learned by an evaluator into impression degree data that is determined by human sensitivity. A network is stored, one of a plurality of stored hierarchical neural networks is selected based on a user input, and the selected hierarchical neural network is configured to learn by reflecting user preferences. By means of a hierarchical neural network that has learned by reflecting the user's preference, feature data consisting of a plurality of physical items of music and impression degree data consisting of items judged by human sensitivity Can be directly related, and depending on the human sensitivity input as a search condition by the user. It is possible to search music data with high accuracy based on the judged impression degree data, and in advance, evaluator performs pre-learning in order to reduce the user's workload for learning the hierarchical neural network. Even if the hierarchical neural network is used, the user's preference can be strongly reflected.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

(This embodiment)
FIG. 1 is a block diagram showing the configuration of the present embodiment of the music search system according to the present invention, FIG. 2 is a block diagram showing the configuration of the terminal device shown in FIG. 1, and FIG. It is a block diagram which shows the structure of the neural network learning apparatus which learns in advance the hierarchical neural network and music map which are used for the music search apparatus shown in FIG.

  In this embodiment, referring to FIG. 1, the music search device 10 and the terminal device 30 are connected by a data transmission path such as a USB, and the terminal device 30 is separated from the music search device 10 and is carried. It has a configuration that can.

  Referring to FIG. 1, the music search device 10 includes a music data input unit 11, a compression processing unit 12, a feature data extraction unit 13, an impression degree data conversion unit 14, a music database 15, and a music mapping unit 16. The music map storage unit 17, the music search unit 18, the PC operation unit 19, the PC display unit 20, the search result output unit 21, the initial setting unit 22, the learning data storage unit 23, and the voice output unit 24. And a hierarchical neural network learning unit 25.

  The music data input unit 11 has a function of reading a storage medium in which music data such as a CD and a DVD is stored. The music data input unit 11 inputs music data from a storage medium such as a CD and a DVD, and extracts a compression processing unit 12 and feature data. To the unit 13. You may comprise so that the music data (delivery data) via networks, such as the internet, other than storage media, such as CD and DVD, may be input. When compressed music data is input, the compressed music data is decompressed and output to the feature data extraction unit 13.

  The compression processing unit 12 compresses the music data input from the music data input unit 11 in a compression format such as MP3 or ATRAC (Adaptive Transform Acoustic Coding), and the compressed music data together with bibliographic data such as artist names and music titles. It is stored in the music database 15.

  The feature data extraction unit 13 extracts feature data composed of fluctuation information from the music data input from the music data input unit 11 and outputs the extracted feature data to the impression degree data conversion unit 14.

  The impression degree data conversion unit 14 converts the feature data input from the feature data extraction unit 13 into impression degree data determined by human sensitivity using a hierarchical neural network that has been learned in advance. The received impression degree data is output to the music mapping unit 16.

  The music database 15 is a large-capacity storage unit such as an HDD, and stores music data and bibliographic data compressed by the compression processing unit 12 in association with feature data extracted by the feature data extraction unit 13. .

  The music mapping unit 16 maps music data to a music map, which is a self-organized map that has been learned in advance, based on the impression degree data input from the impression degree data conversion unit 14, and the music data is mapped to the music data The map is stored in the music map storage unit 17.

  The music map storage unit 17 is a large-capacity storage unit such as an HDD, and stores a music map to which music data is mapped by the music mapping unit 16.

  The music search unit 18 searches the music database 15 based on the impression data and bibliographic data input from the PC operation unit 19, displays the search result on the PC display unit 20, and is selected by the PC operation unit 19. The music map storage unit 17 is searched based on the representative music, and the representative music search result is displayed on the PC display unit 20. Further, the music search unit 18 outputs the music data selected by the PC operation unit 19 to the terminal device 30 via the search result output unit 21.

  The PC operation unit 19 is input means such as a keyboard and a mouse, inputs search conditions for searching for music data stored in the music database 15 and the music map storage unit 17, and selects music data to be output to the terminal device 30. Input is made. Further, the PC operation unit 19 accepts input of impression degree data by the user based on the audio output from the audio output unit 24, and outputs the received impression degree data to the initial setting unit 22.

  The PC display unit 20 is a display unit such as a liquid crystal display, for example, displays an initial setting selection screen and an initial setting input screen, displays a mapping status of music data stored in the music map storage unit 17, the music database 15, and Display of search conditions for searching for music data stored in the music map storage unit 17 and display of searched music data (search results) are performed.

  The search result output unit 21 is configured to be connectable to the search result input unit 31 of the terminal device 30 through a data transmission path such as USB, and is searched by the music search unit 18 and selected by the PC operation unit 19. The received music data is output to the search result input unit 31 of the terminal device 30.

  The learning data storage unit 23 is a storage unit such as a memory, and is used for a hierarchical neural network that has been pre-learned by a third party evaluator (hereinafter referred to as a pre-learned hierarchical neural network) and for pre-learning. Used learning data (hereinafter referred to as pre-learning data) is stored, and a plurality of pre-learned hierarchical neural networks are stored using different evaluators and different pre-learning data. The learning data storage unit 23 stores a plurality of learning maps that have been pre-learned using different evaluators and different pre-learning data. Further, the learning data storage unit 23 includes initial setting music data randomly arranged regardless of the genre, and a plurality of initial setting music data that are arranged in a uniform genre. Are stored as initial setting samples. Note that the initial setting music data of the initial setting sample does not require all of the music, but only the portion used for extracting the feature data.

  When the power is first turned on, the initial setting unit 22 displays an initial setting selection screen 49 on the PC display unit 20, and accepts personal information, a genre of music, or an evaluator's input, thereby learning a hierarchical neural network. In addition to specifying a pre-learned hierarchical neural network to be used for the hierarchical neural network in the unit 25, a learning map to be stored in the music map storage unit 17 is specified, and the specified pre-learned hierarchical neural network and learning map are hierarchized. The data is output to the neural network learning unit 25. In addition, the initial setting unit 22 displays the initial setting input screen 50 on the PC display unit 20 and, based on an instruction from the PC operation unit 19, of the initial setting samples stored in the learning data storage unit 23. The initial setting music data is output to the audio output unit 24, and the impression degree data input from the PC operation unit 19 by the user corresponding to the music data is received and used for learning the pre-learned hierarchical neural network. The prior learning data, the feature data in the initial setting sample, and the impression degree data input by the user are output to the hierarchical neural network learning unit 25 as initial setting data.

  The audio output unit 24 is an audio player that decompresses and reproduces the initial setting music data input from the initial setting unit 22.

  The hierarchical neural network learning unit 25 uses the error back propagation learning method, and inputs the connection weight value of each neuron of the neural network to the pre-learned hierarchical type by the pre-learning data, the feature data of the initial setting sample, and the user. Learning based on the initial setting data composed of the impression degree data, that is, updating the connection weight value of each neuron, and updating the connection weight value of each neuron, that is, letting it learn the hierarchical neural network In addition to outputting to the conversion unit 14, the learning map specified by the initial setting unit 22 is output to the music map storage unit 17.

  The terminal device 30 is an audio reproduction device such as a portable audio having a large capacity storage means such as an HDD. Referring to FIG. 2, a search result input unit 31, a search result storage unit 32, a terminal operation unit 33, It consists of a terminal display unit 34 and an audio output unit 35.

  The search result input unit 31 is configured to be connectable to the search result output unit 21 of the music search device 10 via a data transmission path such as a USB, and the music data input from the search result output unit 21 of the music search device 10 is received. The search result storage unit 32 stores the result.

  The terminal operation unit 33 performs input related to reproduction of music data such as input for instructing selection / reproduction of music data stored in the search result storage unit 32 and input of volume control.

  The terminal display unit 34 is a display means such as a liquid crystal display, for example, and displays the name of a song being played and various operation guidance.

  The audio output unit 35 is an audio player that decompresses and reproduces music data that is compressed and stored in the search result storage unit 32.

  A neural network learning device 40 shown in FIG. 3 is a device that learns a hierarchical neural network used in the impression degree data conversion unit 14 and a music map used in the music mapping unit 16, and referring to FIG. Music data input unit 41, audio output unit 42, feature data extraction unit 43, impression degree data input unit 44, combination weight value learning unit 45, music map learning unit 46, and combination weight value output unit 47 And a feature vector output unit 48.

  The music data input unit 41 has a function of reading a storage medium in which music data such as a CD and a DVD is stored. The music data input unit 41 inputs music data from a storage medium such as a CD and a DVD, and extracts an audio output unit 42 and feature data. To the unit 43. You may comprise so that the music data (delivery data) via networks, such as the internet, other than storage media, such as CD and DVD, may be input. When compressed music data is input, the compressed music data is decompressed and output to the audio output unit 42 and the feature data extraction unit 43.

  The audio output unit 42 is an audio player that decompresses and reproduces music data input from the music data input unit 41.

  The feature data extraction unit 43 extracts feature data composed of fluctuation information from the song data input from the song data input unit 41 and outputs the extracted feature data to the connection weight value learning unit 45.

  The impression degree data input unit 44 accepts input of impression degree data by the evaluator based on the audio output from the audio output unit 42, and uses the received impression degree data as a joint signal as a teacher signal used for learning of the hierarchical neural network. The value is output to the value learning unit 45 and is also output to the music map learning unit 46 as an input vector to the self-organizing map.

  The connection weight value learning unit 45 performs learning on the hierarchical neural network based on the feature data input from the feature data extraction unit 43 and the impression degree data input from the impression degree data input unit 44, and The connection weight value is updated, and the updated connection weight value w of each neuron and the pre-learning data (music data feature data + impression degree data) used for updating are updated by the connection weight value output unit 47 by the learning data storage unit 23. Output to.

  The music map learning unit 46 learns the self-organizing map using the impression degree data input from the impression degree data input unit 44 as an input vector to the self-organizing map, updates the feature vector of each neuron, and The updated feature vector is output via the output unit 48. The learned self-organizing map (updated feature vector) is stored in the music map storage unit 17 of the music search device 10 as a music map.

Next, the operation of the present embodiment will be described in detail with reference to FIGS.
4 is an explanatory diagram for explaining a learning algorithm of the hierarchical neural network in the hierarchical neural network learning unit shown in FIG. 1, and FIG. 5 is a music map learning algorithm in the neural network learning apparatus shown in FIG. It is explanatory drawing for demonstrating,
FIG. 6 is a flowchart for explaining the pre-learning operation of the hierarchical neural network and the music map learning operation in the neural network learning apparatus shown in FIG. 3, and FIG. 7 is the storage of the learning data storage unit shown in FIG. FIG. 8 is a flowchart for explaining the learning operation of the hierarchical neural network and the selection operation of the learning map performed at the initial setting in the hierarchical neural network learning unit shown in FIG. 9 is a diagram showing an example of an initial setting selection screen displayed on the PC display unit shown in FIG. 1, and FIG. 10 is a diagram showing an example of an initial setting input screen displayed on the PC display unit shown in FIG. 11 is a flowchart for explaining a music registration operation in the music search apparatus shown in FIG. 1, and FIG. FIG. 13 is a flowchart for explaining the feature data extraction operation in the data extraction unit, FIG. 13 is a flowchart for explaining the music search operation in the music search device shown in FIG. 1, and FIG. 14 is the PC shown in FIG. 15 is a diagram showing an example of a search screen displayed on the display unit, FIG. 15 is a diagram showing a display example of a search condition input area shown in FIG. 14, and FIGS. 16 and 17 are search result displays shown in FIG. FIG. 18 is a diagram showing an example of the entire music list display area displayed on the search screen example shown in FIG. 14, and FIG. 19 is a diagram showing the search screen example shown in FIG. It is a figure which shows the example of a keyword search area | region performed.

  In the music search device 10 of the present embodiment, as an initial setting prior to use, learning of the hierarchical neural network used by the impression degree data conversion unit 14 and selection of a learning map stored in the music map storage unit 17 are performed. Is done.

  The learning of the hierarchical neural network performed in the initial setting is performed based on the pre-learned hierarchical neural network and the pre-learning data stored in the learning data storage unit 23. A plurality of pre-learned hierarchical neural networks and pre-learning data that have been pre-learned using different evaluators and different pre-learning data are stored, and are stored in the learning data storage unit 23 by default. Select one of the pre-trained hierarchical neural networks and pre-learning data from the pre-trained hierarchical neural network and pre-learning data, and based on the selected pre-trained hierarchical neural network and pre-learning data Hierarchical neural network Learning is performed.

  In addition, the selection of the learning map performed in the initial setting is to select one of the plurality of learning maps stored in the learning data storage unit 23, and the plurality of learning maps stored in the learning data storage unit 23 are selected. These learning maps are pre-learned using different evaluators and different pre-learning data.

  Hereinafter, prior learning of a hierarchical neural network and a music map by a third party evaluator will be described in detail with reference to FIGS.

As shown in FIG. 4, the hierarchical neural network used in the impression degree data conversion unit 14 includes an input layer (first layer), an intermediate layer (nth layer), and an output layer (Nth layer). By inputting feature data into the (first layer), impression level data is output from the output layer (Nth layer), that is, feature data is converted into impression level data and output from the output layer (Nth layer) The connection weight value w of each neuron in the intermediate layer (nth layer) is pre-learned. The feature data is data of 8 items to be extracted from the music data by the characteristic data extraction unit 13, the neuron number L ₁ of the input layer (first layer) has a eight (characteristic data extraction The method of extracting feature data by the unit 13 will be described later). In addition, impression data is judged by human sensitivity (bright, dark), (heavy, light), (hard, soft), (stable, unstable), (clear, dirty), (smooth 8 items of (crisp, crisp), (violent, gentle), (thick, thin) are data expressed by 7-level evaluation, and the number of neurons L _N in the output layer (Nth layer) is 8 It has become. The number of neurons L _n in the intermediate layer (nth layer: n = 2,..., N−1) is set as appropriate.

Further, as shown in FIG. 5, the music map stored in the music map storage unit 17 is a self-organization in which neurons are regularly arranged two-dimensionally (9.9 squares in the example shown in FIG. 5). This is a learning network that does not require a teacher signal and is a neural network that autonomously acquires the ability to classify input pattern groups according to their similarity. In the present embodiment, a two-dimensional SOM in which neurons are arranged in 100/100 squares is used. However, the neurons may be square or beehives. Each neuron of the music map, the n-dimensional feature vectors m _{i (t)} ∈R ⁿ are included, the feature vectors are contained in each neuron by prior learning by pre-learning data m _{i (t)} ∈R ⁿ Is learned.

Prior learning of the hierarchical neural network (connection weight value w) and the music map (feature vector m _i (t) εR ⁿ ) by the evaluator is performed using the neural network learning device 40 shown in FIG. Prior learning data (feature data feature data + impression degree data) for pre-learning a hierarchical neural network (connection weight value w) and a music map (feature vector m _i (t) εR ⁿ ) is input. .

  A music medium such as a CD or a DVD is stored in the music data input unit 41, the music data is input from the music data input unit 41 (step A1), and the feature data extraction unit 43 inputs the music data. Feature data consisting of fluctuation information is extracted from the music data input from the unit 41 (step A2).

  The voice output unit 42 outputs the music data input from the music data input unit 41 as a voice (step A3), and the evaluator listens to the voice output from the voice output unit 42, thereby increasing the impression level of the music. Evaluation is performed based on sensitivity, and the evaluation result is input as impression degree data from the impression degree data input unit 44 (step A4). The combined weight value learning unit 45 uses the impression degree data input from the impression degree data input unit 44 as a teacher signal. Accept as. In this embodiment, the evaluation items for impression degree are determined by human sensitivity (bright, dark), (heavy, light), (hard, soft), (stable, unstable), ( 8 items of clear, sloppy), (smooth, crisp), (violent, gentle), (thick, thin) are set, and impression level data input section with 7-level evaluation for each item as impression level data 44 to accept.

Next, steps until it is determined whether the pre-learned data that is input has reached the number of samples T ₁ for a predetermined (step A5), pre-learning data input reaches the number of samples T ₁ Al to A4 Is repeated.

If prior learning data reaches the number of samples T ₁ predetermined, on the basis of the pre-learning data, the connection weights learning unit 45 with learning of the hierarchical neural network is performed (updated connection weights values w) (step A6) Based on the impression degree data input from the impression degree data input unit 44 in the pre-learning data, the music map learning unit 46 learns the learning map (updates the feature vector m _i (t) ∈R ⁿ . ) Is performed (step A9).

The learning of the hierarchical neural network performed in step A6, that is, the update of the connection weight value w of each neuron is performed using the error back propagation learning method.
First, as an initial value, the connection weight value w of all the neurons of the intermediate layer (nth layer) is set to a small value in the range of about −0.1 to 0.1 by a random number, and the connection weight value learning unit 45 inputs the feature data extracted by the feature data extraction unit 43 into the input layer (first layer) as an input signal x _j (j = 1, 2,..., 8), and from the input layer (first layer). The output of each neuron is calculated toward the output layer (Nth layer).

Next, the combined weight value learning unit 45 uses the impression degree data input from the impression degree data input unit 44 as a teacher signal y _j (j = 1, 2,..., 8), and outputs the output layer (Nth layer). The learning rule δ _j ^N is calculated from the error between the output out _j ^N and the teacher signal y _j by the following equation.

Next, the joint weight value learning unit 45 calculates the error signal δ _j ⁿ of the intermediate layer (nth layer) using the learning rule δ _j ^N by the following equation.

  In Equation 2, w represents a connection weight value between the n-th layer j-th neuron and the (n −1) -th layer k-th neuron.

Next, the connection weight value learning unit 45 calculates the amount of change Δw of the connection weight value w of each neuron using the following equation using the error signal δ _j ⁿ of the intermediate layer (nth layer), and the connection weight of each neuron. The value w is updated (step A6). In the following equation, η represents a learning rate, and is set to η ₁ (0 <η ₁ ≦ 1) in the prior learning by the evaluator.

In step A6, for each of the pre-training data sample number T ₁ learning is performed, then, or smaller or not than the reference value E ₁ for pre-learning is square error E shown in the following equation predetermined There is judged (step A7), the operation of step A6 to the square error E is smaller than the reference value E ₁ is repeated. Incidentally, the learning iterations S squared error E is assumed to be smaller than the reference value E ₁ is set in advance, the operation of step A6 may be repeated S times.

In the case of the square error E is determined to be smaller than the reference value E ₁ Step A7, connection weights learning unit 45, the connection weight values w of each neuron is pre-learning, i.e. pre-learned hierarchical neural The combined weight value output unit 47 outputs the network and the pre-learning data (music data feature data + impression degree data) used for the pre-learning, and the pre-learned hierarchical neural network and the pre-learning data that are output, It memorize | stores in the learning data memory | storage part 23 (step A8).

In the music map learning performed in step A9, the music map learning unit 46 uses the impression degree data input from the impression degree data input unit 44 as an input vector x _j (t) εR ^n, and the feature vector m of each neuron. _i (t) ∈ R ⁿ is learned. Note that t represents the number of learning times, a preset value T that determines the number of learning times is set in advance, and learning is performed for the learning number t = 0, 1,. Note that R indicates the evaluation stage of each impression degree item, and n indicates the number of items of impression degree data.

First, as an initial value, feature vectors m _c (0) of all neurons constituting the music map are randomly set in the range of 0 to 1, and the music map learning unit 46 sets x _j (t) to x _j (t). Find the nearest neuron c, that is, the winner neuron c that minimizes ‖x _j (t) -m _c (t) ‖, the feature vector m _c (t) of the winner neuron c, and the neighborhood in the vicinity of the winner neuron c Each feature vector m _i (t) (iεN _c ) of the set N _c of neurons i is updated according to the following equation (step A9). It is assumed that the neighborhood radius for determining the neighborhood neuron i is set in advance.

In Equation 5, h _ci (t) represents a learning rate and is obtained by the following equation.

Α _init is an initial value of the learning rate, and R ² (t) is a monotonically decreasing linear function or exponential function.

Next, the music map learning unit 46 determines whether or not the learning count t has reached the set value T (step A10), and performs the processing operations of step A9 to step A10 until the learning count t reaches the set value T. Repeatedly, when the learning count t reaches the set value T, the feature vector m _i (T) ∈R ⁿ learned through the feature vector output unit 48 is outputted (step A11), and the feature of each neuron i outputted is outputted. The vector m _i (T) is stored in the learning data storage unit 23 of the music search device 10 as a music map that has been pre-learned.

The operations in steps A1 to A11 described above are performed for different evaluators and different pre-learning data, respectively, and a plurality of pre-learned hierarchical neural networks and pre-learning data are stored in the learning data storage unit 23 as shown in FIG. A plurality of music maps are stored. In the example shown in FIG. 7, # ₁ to # 5 pre-learned hierarchical type trained in the learning data storage unit 23 by the evaluators A to E based on the music data of the sample number T ₁ randomly arranged, respectively. neural network, prior learning with data and song map is stored, the evaluator F is ♯6~♯10 pre learned that was learned, respectively on the basis of the music data of the sample number T ₁ that is aligned for each genre, respectively A hierarchical neural network, pre-learning data, and a music map are stored.

Further, in the learning data storage unit 23, as shown in FIG. 7, the evaluator respective personal information A~E is evaluator A~E is based on the music data of the sample number T ₁ which is aligned at random It is stored in correspondence with the previously learned # 1 to # 5 pre-learned hierarchical neural network, pre-learning data, and music map. The personal information indicates unique information of the evaluator including items such as blood type, gender, age, constellation, personality (shortness, weakness, temperament).

Further, in the learning data storage unit 23, as shown in FIG. 7, the initial setting music data ♯1 sample number T ₂ which is aligned at random regardless of genre, the genre (pop, jazz, rock, country music , Enka) and the initial setting music data # _{2 to} # ₆ of the number of samples T2 that are aligned and stored together with the feature data of the initial setting music data are stored as initial setting samples.

  Next, the learning operation of the hierarchical neural network and the music map selection operation performed at the initial setting in the hierarchical neural network learning unit 25 will be described in detail with reference to FIGS.

  When the power is first turned on, the initial setting unit 22 displays an initial setting selection screen 49 on the PC display unit 20 (step B1). The initial setting selection screen 49 selects one of a plurality of pre-learned hierarchical neural networks and pre-learning data stored in the learning data storage unit 23 by personal information, music genre or evaluator. FIG. 9 is a screen for selecting one of a plurality of music maps stored in the learning data storage unit 23. As shown in FIG. 9, the user's personal information To select a personal information input field 491 for inputting (blood type, gender, age, constellation, personality), a genre input field 492 for inputting a genre of music, and a selection by an evaluator. An evaluator input field 493 for inputting a desired evaluator and a selection execution button 494 for instructing execution of selection Consisting of.

  When personal information, a genre of music or an evaluator is input by the user and the selection execution button 494 is clicked, the initial setting unit 22 determines whether the selection is based on personal information, that is, in the personal information input field 491, It is determined whether the selection execution button 494 has been clicked after the personal information has been input (step B2). If the selection is based on personal information, the user's personal information is compared with the evaluator's personal information (step B2). B3) A pre-learned hierarchical neural network and a learning map learned by an evaluator closest to the user's personal information are specified (step B4). That is, as shown in FIG. 7, personal information PA to PE of the evaluators A to E is stored in the learning data storage unit 23. As a result of comparing the user's personal information with the personal information PA to PE, When the personal information PA is closest to the personal information, the pre-learned hierarchical neural network # 1 and the learning map # 1 learned by the evaluator A are specified. The comparison between the user's personal information and the evaluator's personal information is performed by, for example, vectorizing the user's personal information and the evaluator's personal information so that the evaluator's personal information having the smallest Euclidean distance is the closest personal It should be data. Further, in the present embodiment, the personal information personality is directly input. However, a personality diagnosis questionnaire may be prepared, and the personality may be specified based on a user's answer to the personality diagnosis questionnaire.

  Next, the initial setting unit 22 displays the initial setting input screen 50 on the PC display unit 20 (step B5). As shown in FIG. 10, the initial setting input screen 50 has impression data, that is, (bright, dark), (heavy, light), (hard, soft), (stable, unstable), (clear, dirty) (I), (smooth, crisp), (severe, gentle), (thick, thin), the impression degree data input area 51 for inputting the seven-level evaluation for each of the eight items, and the initial instruction for starting the initial setting It includes a setting start button 52 and an input confirmation button 53 for confirming the input of impression degree data input to the impression degree data input area 51.

  When the initial setting start button 52 is clicked by the user, the initial setting unit 22 randomly aligns the plurality of initial setting samples # 1 to # 6 stored in the learning data storage unit 23 regardless of the genre. The first one of the initial setting music samples # 1 thus read is read, and the initial setting music data of the read initial setting samples is output to the audio output unit 24. The audio output unit 24 The music data for initial setting input from is output as a sound (step B6).

The user evaluates the impression level of the music based on the sensitivity by listening to the audio output from the audio output unit 24, and inputs the evaluation result, that is, the impression level data, from the PC operation unit 19 to the impression level data input area 51 (step B7), the input confirmation button 53 is clicked. When the input confirmation button 53 is clicked by the user, the initial setting unit 22 determines whether or not the evaluation result by the user, that is, impression degree data has been input for all of the initial setting samples (sample number T ₂ ). (Step B8), the operations of Steps B6 to B7 are repeated until the evaluation result by the user, that is, the impression degree data, is input to all the initial setting samples (sample number T ₂ ).

  If it is determined in step B2 that the selection is not based on personal information, the initial setting unit 22 selects whether the selection is made by the evaluator, that is, after the personal information of the user is input in the evaluator input field 493. It is determined whether or not the execution button 494 has been clicked (step B9). If the selection is made by the evaluator, the pre-learned hierarchical neural network and learning map learned by the selected evaluator are specified (step B10). ) To Step B5.

If it is determined in step B9 that the selection is not made by the evaluator, the initial setting unit 22 is selected because the selection by the genre and the selection execution button 494 is clicked after the genre of the music is input in the genre input field 493. Specifies a pre-learned hierarchical neural network and a learning map that have been learned using pre-learning data arranged in the selected genre (step B11). That is, as shown in FIG. 7, a pre-learned hierarchy that the evaluator F has learned based on the music data of the number of samples T ₁ arranged for each genre (pops, jazz, rock, country music, and enka), respectively. Type neural networks # 6 to # 10 and learning maps # 6 to # 10 are stored, and when pops is selected as a genre, learning is performed based on music data of the selected genre (pops). A pre-learned hierarchical neural network # 6 and a learning map # 6 are specified.

  Next, the initial setting unit 22 displays the initial setting input screen 50 on the PC display unit 20 (step B12), and when the user clicks the initial setting start button 52, the initial setting unit 22 displays the learning data storage unit. 23, the first one of the initial setting music samples aligned with the selected genre is read out from the plurality of initial setting samples # 1 to # 6 stored in the initial setting sample. The initial setting music data is output to the audio output unit 24, and the audio output unit 24 outputs the initial setting music data input from the initial setting unit 22 as audio (step B13).

The user evaluates the impression level of the music based on the sensitivity by listening to the audio output from the audio output unit 24, and inputs the evaluation result, that is, the impression level data, from the PC operation unit 19 to the impression level data input area 51 (step B14), the input confirmation button 53 is clicked. When the input confirmation button 53 is clicked by the user, the initial setting unit 22 determines whether or not the evaluation result by the user, that is, impression degree data has been input for all of the initial setting samples (sample number T ₂ ). (Step B15), the operations of Steps B13 to B14 are repeated until the evaluation results by the user, that is, the impression degree data, are input to all the initial setting samples (sample number T ₂ ).

If it is determined in step B8 or step B15 that the evaluation result by the user, that is, impression degree data, has been input for all of the initial setting samples (sample number T ₂ ), it is specified in step B4, step B10 or step B11. The learning weight storage unit 23 reads out the connection weight value w of each neuron that has been pre-learned from the pre-learned hierarchical neural network and the learning map specified in step B4, step B10, or step B11, to obtain the hierarchical neural network. Output to the learning unit 25, and the pre-learning data used for the pre-learning of the specified pre-learned hierarchical neural network, and the feature data in the initial setting sample used for the input of impression degree data Is read from the learning data storage unit 23, And it outputs the hierarchical neural network learning portion 25 plus the impression data input by the characteristic data and the user of the initial setting for the sample before the training data as the initial setting data. That is, there sample number T ₁ of the prior learning data 100 music pieces, sample number T ₂ of the sample for initial setting in the case of 10 songs, the data of 100 music pieces evaluated by the evaluator, the user has rated Feature data and impression degree data for 110 songs (sample number T ₁ + sample number T ₂ ) composed of data for 10 songs are output to the hierarchical neural network learning unit 25 as initial setting data.

  The hierarchical neural network learning unit 25 learns the hierarchical neural network, that is, updates the connection weight value w of each neuron using the error back propagation learning method based on the initial setting data.

First, the connection weight value w of neurons in the intermediate layer (n-th layer) is set as an initial value to the connection weight value w learned in advance, and the hierarchical neural network learning unit 25 is input from the initial setting unit 22. The inputted feature data is input to the input layer (first layer) as an input signal x _j (j = 1, 2,..., 8), and directed from the input layer (first layer) to the output layer (Nth layer). Calculate the output of each neuron.

Next, the hierarchical neural network learning unit 25 uses the impression degree data input from the initial setting unit 22 as a teacher signal y _j (j = 1, 2,..., 8), and outputs the output layer (Nth layer). From the error between out _j ^N and the teacher signal y _j , the learning rule δ _j ^N is calculated by Equation 1.

Next, the hierarchical neural network learning unit 25 uses the learning rule δ _j ^N to calculate the error signal δ _j ⁿ of the intermediate layer (the n-th layer) using Equation 2.

Next, the hierarchical neural network learning unit 25 calculates the amount of change Δw of the connection weight value w of each neuron using Equation 3 using the error signal δ _j ⁿ of the intermediate layer (nth layer), and connects each neuron. The weight value w is updated (step B16). Note that the learning rate in Equation 3 is set to η ₂ (0 <η ₂ ≦ 1), and η ₂ may be a value larger than the learning rate η ₁ used in the prior learning by the evaluator. Furthermore, when learning is performed using data of 10 songs evaluated by the user, that is, impression degree data input by the user as a teacher signal, the learning rate is set to η ₃ (0 <η ₁ , η ₂ <η ₃ ≦ 1). ) Is preferable because the user's preference is strongly reflected in the learning of the hierarchical neural network.

In step B16, learning is performed for each of the initial setting data of the number of samples T ₁ + the number of samples T ₂ , and then, the initial setting reference value E _{2 in which the} square error E shown in Expression 4 is predetermined. It is determined whether or not it is smaller than (initial reference reference value E ₂ <pre-learning reference value E ₁ ) (step B 17). Step B 16 is continued until the square error E becomes smaller than the reference value E ₂ . The operation is repeated. Incidentally, the learning iterations S squared error E is assumed to be smaller than the reference value E ₂ is set in advance, the operation of step B16 may be repeated S times. Further, if the number of learnings using the data (sample number T ₂ ) evaluated by the user in the initial setting data is made larger than the other data (sample number T ₁ ), the user's preference becomes higher than that of the hierarchical neural network. It is suitable because it is strongly reflected in learning. That is, in the learning in step B16, whereas train the initial setting data of the sample number T ₁ once, the user is assessed to learn the data for initial setting of the sample number T ₂ by a plurality of times The number of learnings using the data (sample number T ₂ ) evaluated by the user in the initial setting data is set to be larger than that of the other data (sample number T ₁ ). When the operation of step B16 is determined in advance as the number of learning iterations S, the number of iterations of learning using the data (sample number T ₂ ) evaluated by the user in the initial setting data is set to other data (samples). It is preferable to increase the number of learning iterations using the number T ₁ ).

Square when the error E is determined to be smaller than the reference value E ₂ in step B17, the hierarchical neural network learning portion 25, the coupling weight value w for each neuron that has learned in the impression-data-conversion unit 14 While outputting, the learning map specified by step B4, step B10, or step B11 is output to the music map memory | storage part 17 (step B18).

Next, the music registration operation in the music search apparatus 10 will be described in detail with reference to FIG.
A storage medium in which music data such as CD and DVD is stored is set in the music data input unit 11, and music data is input from the music data input unit 11 (step C1).

  The compression processing unit 12 compresses the music data input from the music data input unit 11 (step C2), and stores the compressed music data in the music database 15 together with the bibliographic data such as artist name and music name (step C3). .

The feature data extraction unit 13 extracts feature data composed of fluctuation information from the song data input from the song data input unit 11 (step C4).
Referring to FIG. 12, the feature data extraction operation in the feature data extraction unit 13 accepts input of music data (step D1), and performs FFT (FFT) for a certain frame length from a predetermined data analysis start point of music data. (Fast Fourier transform) is performed (step D2), and a power spectrum is calculated. Note that downsampling may be performed for the purpose of speeding up before step D2.

  Next, the feature data extraction unit 13 sets the frequency bands of Low, Middle, and High in advance, integrates the power spectrum of the three bands of Low, Middle, and High to calculate the average power (step D3). Among the frequency bands of Low, Middle, and High, the band having the maximum power is set as the Pitch data analysis start point value, and the Pitch is measured (step D4).

  The processing operations in steps D2 to D4 are performed for a predetermined number of frames, and the feature data extraction unit 13 determines whether the number of frames for which the processing operations in steps D2 to D4 have been performed has reached a predetermined set value. (Step D5), and if the number of frames for which the processing operations of steps D2 to D4 have not reached the predetermined set value, the data analysis start point is shifted (step D6). , The processing operation of Step D2 to Step D4 is repeated.

  When the number of frames for which the processing operations of Step D2 to Step D4 have been performed reaches a predetermined set value, the feature data extraction unit 13 calculates Low, Middle, High calculated by the processing operations of Step D2 to Step D4. The FFT is performed on the time series data of the average power and the pitch time series data measured by the processing operations in steps D2 to D4 (step D7).

  Next, the feature data extraction unit 13 calculates the slope of the regression line in the graph with the horizontal axis representing the logarithmic frequency and the vertical axis representing the logarithmic power spectrum from the FFT analysis results in Low, Middle, High, and Pitch, and the Y intercept of the regression line. Are calculated as fluctuation information (step D8), and the slope of the regression line and the Y-intercept in each of Low, Middle, High, and Pitch are output to the impression degree data conversion unit 14 as feature data of eight items.

The impression data conversion unit 14 uses a hierarchical neural network including an input layer (first layer), an intermediate layer (n-th layer), and an output layer (N-th layer) as shown in FIG. By inputting the feature data extracted by the feature data extraction unit 13 into the first layer), impression level data is output from the output layer (Nth layer), that is, the feature data is converted into impression level data (step C5), Impression degree data output from the output layer (Nth layer) is output to the music mapping unit 16 and stored in the music database 15 together with the music data. Note that the connection weight value w of each neuron in the intermediate layer (nth layer) is learned in advance by the neural network learning device 40. In the case of the present embodiment, the feature data input to the input layer (first layer), that is, the feature data extracted by the feature data extraction unit 13 is eight items as described above, Impression data items are determined by human sensitivity (bright, dark), (heavy, light), (hard, soft), (stable, unstable), (clear, fuzzy), ( Eight items of (smooth, crisp), (violent, gentle), (thick, thin) were set, and each item was set to be expressed by a seven-level evaluation. Therefore, the number of neurons L _N of input layer neurons number L ₁ and the output layer (first layer) (the N th layer) is a eight respectively, the intermediate layer (the n-th layer: n = 2, ... , N−1), the number of neurons L _n is set as appropriate.

The music mapping unit 16 maps the music input from the music data input unit 11 to a corresponding portion of the music map stored in the music map storage unit 17. Each neuron of the music map used for the mapping operation in the music mapping unit 16 includes an n-dimensional feature vector m _i (t) εR ⁿ learned in advance, and the music mapping unit 16 stores the impression degree data. The impression degree data converted by the conversion unit 14 is set as an input vector x _j , and the input music is mapped to a neuron closest to the input vector x _j , that is, a neuron that minimizes the Euclidean distance ‖ x _j −m _i ‖. (Step C6), the mapped music map is stored in the music map storage unit 17. R represents the number of evaluation stages for each item of impression degree data, and n represents the number of items of impression degree data.

Next, a music search operation in the music search apparatus 10 will be described in detail with reference to FIGS.
The music search unit 18 displays a search screen 60 as shown in FIG. 14 on the PC display unit 20 and accepts user input from the PC operation unit 19. The search screen 60 includes a music map display area 61 in which the mapping status of music data stored in the music map storage unit 17 is displayed, a search condition input area 62 for inputting search conditions, and a search in which search results are displayed. It consists of a result display area 63. The points shown in the music map display area 61 in FIG. 14 indicate the neurons of the music map to which the music data is mapped.

  As shown in FIG. 15, the search condition input area 62 includes an impression degree data input area 621 for inputting impression degree data as a search condition, a bibliographic data input area 622 for inputting bibliographic data as a search condition, and executes a search. The search execution button 623 for instructing the user inputs impression degree data and bibliographic data as search conditions from the PC operation unit 19 (step E1), and clicks the search execution button 623, whereby the impression degree data and bibliography are displayed. A search based on data is instructed to the music search unit 18. Note that the impression level data is input from the PC operation unit 19 by inputting each item of the impression level data in a seven-step evaluation, as shown in FIG.

  The music search unit 18 searches the music database 15 based on the impression data and bibliographic data input from the PC operation unit 19 (step E2), and displays the search results as shown in FIG. To do.

The search based on the impression degree data input from the PC operation unit 19 uses the impression degree data input from the PC operation unit 19 as the input vector _xj, and searches the impression degree data stored together with the song data in the song database 15. If the target vector _{X j,} the input vectors _{x j} closer search target vector _{X j,} i.e. go search the Euclidean distance ‖x _j -X _j ‖ is ascending order. The number of searches may be determined in advance or arbitrarily set by the user. If both impression level data and bibliographic data are set as search conditions, after searching based on bibliographic data, searching based on impression level data is performed.

  In addition to the search using the search condition input area 62, a search using the music map display area 61 may be performed. In this case, by designating a search target area in the music map display area 61, the music data mapped in the search target area is displayed in the search result display area 63 as a search result.

  Next, the user selects a representative song from the search results displayed in the search result display area 63 (step E3), and clicks the representative song search execution button 631 to perform a search based on the representative song. The search unit 18 is instructed.

  The music search unit 18 searches the music map stored in the music map storage unit 17 based on the selected representative music (step E4), and maps it to the neuron to which the representative music is mapped and its neighboring neurons. The stored music data is displayed in the search result display area 63 as a representative music search result. The neighborhood radius for determining the neighborhood neuron may be set in advance or arbitrarily set by the user.

  Next, the user selects music data to be output to the terminal device 30 from the representative music search results displayed in the search result display area 63 (step E5), and the output button 632 is selected. By clicking, the music search unit 18 is instructed to output the selected music data, and the music search unit 18 outputs the music data selected by the user via the search result output unit 21 to the terminal device 30 (step). E6).

  In addition to searching for representative songs using the search condition input area 62 and the music map display area 61, a search is made for an all music list display area 64 in which a list of all stored music is displayed as shown in FIG. The screen is displayed on the screen 60, and a representative song is directly selected from the entire song list, and a search based on the selected representative song is instructed to the song search unit 18 by clicking a representative song selection execution button 641. May be.

In addition to the search described above, neurons (or songs) corresponding to keywords expressed in words such as “bright songs”, “fun songs”, and “healed songs” are set and keywords are selected. It may be configured so that music can be searched. That is, as shown in FIG. 19A, the keyword search area 65 is displayed on the search screen 60, one is selected from the keyword list displayed in the keyword selection area 651, and the automatic search button 653 is clicked. Thus, the music search unit 18 is instructed to search based on the neuron corresponding to the selected keyword. In the set music display area 652 shown in FIG. 19A, when a music corresponding to the selected keyword is set, the music is displayed as the set music. In this case, an automatic search button 653 is displayed. By clicking, the music search unit 18 is instructed to search for the set music corresponding to the selected keyword as a representative music. The set music change button 654 shown in FIG. 19A is used when changing the music corresponding to the keyword. When the set music change button 654 is clicked, the entire music list is displayed. The music corresponding to the keyword can be changed by selecting the music from the entire music list. The setting of neurons (or songs) that correspond to the keywords in advance by assigning impression data to the keyword, the impression data as input vectors x _j, the nearest neuron to the input vector x _j (or music) You may make it match | combine and you may comprise so that it can set arbitrarily by a user.

  In this way, when the neuron corresponding to the keyword is set, as shown in FIG. 19B, when the neuron to which the music is mapped is clicked in the music map display area 61, the clicked neuron is displayed. If the corresponding keyword is configured to be displayed as a pop-up as the keyword display 611, it is possible to easily search for music using the music map display area 61.

  As described above, according to the present embodiment, the learning data storage unit 23 stores a plurality of hierarchical neural networks that have been pre-trained by the evaluator, and the initial setting unit 22 stores the learning data storage unit. 23 selects one of the plurality of hierarchical neural networks stored on the basis of a user input, and the hierarchical neural network selected by the initial setting unit 22 determines the user's preference by the hierarchical neural network learning unit 25. By configuring to reflect and learn, it is judged by feature data consisting of multiple physical items of music and human sensibilities by a hierarchical neural network that learns by reflecting user preferences Impression data consisting of Therefore, it is possible to search music data with high accuracy based on impression degree data determined by human sensibility input as a search condition, and reduce the user's workload for learning the hierarchical neural network. Therefore, the user's preference can be strongly reflected even using a hierarchical neural network that has been pre-learned by an evaluator in advance.

Next, another embodiment of the present invention will be described in detail with reference to FIG.
FIG. 20 is a block diagram showing the configuration of another embodiment of the music search system according to the present invention.

  In another embodiment shown in FIG. 20, a music database 36, a music map storage unit 37, and a music search unit 38 having functions equivalent to those of the music database 15, music map storage unit 17 and music search unit 18 shown in FIG. The terminal device 30 is configured so that the terminal device 30 can search the music database 36 and search the music map stored in the music map storage unit 37. In another embodiment, the music search device 10 stores the music data input from the music data input unit 11 in the music database 15, the impression data converted by the impression data conversion unit 14 in the music database 15, The music map mapped by the mapping unit 16 is used as a music registration device that stores the music map in the music map storage unit 17.

  The contents stored in the music database 15 and the music map storage unit 17 of the music search device 10 are output to the terminal device 30 by the database output unit 26, and the database input unit 39 of the terminal device 30 receives the music database 15 and the music map storage unit 17. Is stored in the music database 36 and the music map storage unit 37. The search condition is input from the terminal operation unit 33 based on the display content of the terminal display unit 34.

  Note that the present invention is not limited to the above-described embodiments, and it is obvious that the embodiments can be appropriately changed within the scope of the technical idea of the present invention. In addition, the number, position, shape, and the like of the constituent members are not limited to the above-described embodiment, and can be set to a suitable number, position, shape, and the like in practicing the present invention. In each figure, the same numerals are given to the same component.

It is a block diagram which shows the structure of this Embodiment of the music search system which concerns on this invention. It is a block diagram which shows the structure of the terminal device shown in FIG. It is a block diagram which shows the structure of the neural network learning apparatus which learns in advance the hierarchical neural network and music map which are used for the music search apparatus shown in FIG. It is explanatory drawing for demonstrating the learning algorithm of the hierarchical neural network in the hierarchical neural network learning part shown in FIG. It is explanatory drawing for demonstrating the learning algorithm of the music map in the neural network learning apparatus shown in FIG. 4 is a flowchart for explaining a pre-learning operation of a hierarchical neural network and a music map learning operation in the neural network learning apparatus shown in FIG. 3. It is a figure which shows the memory content of the learning data memory | storage part shown in FIG. 3 is a flowchart for explaining a learning operation of the hierarchical neural network performed at the time of initial setting in the hierarchical neural network learning unit shown in FIG. 1. It is a figure which shows the example of an initial setting selection screen displayed on the PC display part shown in FIG. It is a figure which shows the example of an initial setting input screen displayed on the PC display part shown in FIG. It is a flowchart for demonstrating the music registration operation | movement in the music search apparatus shown in FIG. It is a flowchart for demonstrating the feature data extraction operation | movement in the feature data extraction part shown in FIG. It is a flowchart for demonstrating the music search operation | movement in the music search apparatus shown in FIG. It is a figure which shows the example of a search screen displayed on the PC display part shown in FIG. It is a figure which shows the example of a display of the search condition input area shown in FIG. It is a figure which shows the example of a display of the search result display area shown in FIG. It is a figure which shows the example of a display of the search result display area shown in FIG. It is a figure which shows the example of all the music list display areas displayed on the example of a search screen shown in FIG. It is a figure which shows the example of a keyword search area | region displayed on the example of a search screen shown in FIG. It is a block diagram which shows the structure of other embodiment of the music search system which concerns on this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Music search device 11 Music data input part 12 Compression processing part 13 Feature data extraction part 14 Impression degree data conversion part 15 Music database 16 Music mapping part 17 Music map memory | storage part 18 Music search part 19 PC operation part (Impression degree data input means) )
DESCRIPTION OF SYMBOLS 20 PC display part 21 Search result output part 22 Initial setting part 23 Learning data storage part 24 Speech output part 25 Hierarchical neural network learning part 26 Database output part 30 Terminal device 31 Search result input part 32 Search result storage part 33 Terminal operation part 34 terminal display unit 35 audio output unit 36 music database 37 music map storage unit 38 music search unit 39 database input unit 40 neural network learning device 41 music data input unit 42 audio output unit 43 feature data extraction unit 44 impression degree data input unit 45 Joint weight value learning section 46 Music map learning section 47 Joint weight value output section 48 Feature vector output section 49 Initial setting selection screen 50 Initial setting input screen 51 Impression degree data input area 52 Initial setting start button 53 Input confirmation button 60 Search screen 61 Music Display area 62 search condition input area 63 search result display area 64 all music list display area 65 keyword search area 491 personal information input field 492 genre input field 493 evaluator input field 494 selection execution button 611 keyword display 621 impression degree data input Area 622 Bibliographic data input area 623 Search execution button 631 Representative song search execution button 632 Output button 641 Representative song selection execution button 651 Keyword selection area 652 Set music display area 653 Automatic search button 654 Set music change button

Claims (7)

A music search system for storing a plurality of music data in a music database and searching for the desired music data from the plurality of music data stored in the music database,
Learning data storage means storing a plurality of hierarchical neural networks that have been pre-learned by an evaluator and that convert physical feature data of the music data into impression degree data determined by human sensitivity,
Initial setting means for selecting one of a plurality of hierarchical neural networks stored in the learning data storage means;
Audio output means for outputting the music data as audio;
Impression degree data input means for inputting the impression degree data corresponding to the music data output from the voice output means;
The hierarchical neural network selected by the initial setting means is trained by using the feature data included in the music data output from the audio output means and the impression degree data input from the impression degree data input means. Hierarchical neural network learning means;
Music data input means for inputting the music data;
Feature data extraction means for extracting the feature data from the music data input by the music data input means;
Impression degree data conversion means for converting the feature data extracted by the feature data extraction means into the impression degree data using a hierarchical neural network learned by the hierarchical neural network learning means;
Storage control means for storing the impression degree data converted by the impression degree data conversion means in the music database together with the music data input by the music data input means;
Music search means for searching the music database based on the impression degree data input as a search condition;
A music search system comprising: music data output means for outputting the music data searched by the music search means. Comprising personal information input means for inputting user personal information;
A plurality of hierarchical neural networks that have been pre-learned by different evaluators in the learning data storage means are stored together with the evaluator's personal information,
The initial setting unit selects one of a plurality of hierarchical neural networks stored in the learning data storage unit by comparing the personal information of the user and the personal information of the evaluator. The music search system according to claim 1. Comprising genre input means for inputting the genre of the music data;
The learning data storage means stores a plurality of hierarchical neural networks that have been pre-learned by music data of different genres,
3. The initial setting unit selects one of a plurality of hierarchical neural networks stored in the learning data storage unit based on the genre input by the genre input unit. The described music search system. A music search method for storing a plurality of music data in a music database and searching for the desired music data from a plurality of the music data stored in the music database,
A plurality of hierarchical neural networks for converting physical feature data of the music data, which has been pre-learned by the evaluator, into impression degree data determined by human sensitivity, are stored,
Select one of the stored hierarchical neural networks,
Audio output of the music data;
Receiving an input of the impression degree data corresponding to the music data output by voice;
The selected hierarchical neural network is learned using the feature data of the music data that has been output as audio and the received impression data,
Receiving input of the music data;
Extracting the feature data from the received music data;
Using the learned hierarchical neural network, the extracted feature data is converted into the impression data,
Storing the converted impression data together with the received music data in the music database;
Accept the input of impression degree data as a search condition,
Search the music database based on the received impression degree data,
A music search method comprising outputting the searched music data. Accepts user personal information input,
A plurality of hierarchical neural networks that have been pre-learned by the evaluator are stored together with the evaluator's personal information,
5. The music according to claim 4, wherein one of the plurality of stored hierarchical neural networks is selected by comparing the received personal information of the user and personal information of the evaluator. retrieval method. Accepts input of the genre of the music data,
Store multiple hierarchical neural networks that have been pre-learned with music data of different genres,
6. The music search method according to claim 4, wherein one of the plurality of stored hierarchical neural networks is selected based on the received genre. A music search program for causing a computer to execute the music search method according to claim 4.