JP2001060099A - Device and method for information processing, and medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide information relative to a music having the possibility of possessing a voice inputted melody. SOLUTION: A communication section 31 receives voice data and IDs and outputs them to a melody data generating section 32. A profile information storage section 34 stores the age of a user, information relative to his taste and reference items in which prescribed set values are set. A melody data generating section 32 generates melody data from the voice data of the section 31 employing the reference items, in which prescribed set values are set, stored in the section 34 and outputs the generated melody data to a retrieval section 33. A music information storage section 35 stores melody data for every music. The section 33 computes the degree of correlation between the melody data supplied from the section 32 and the melody data of the section 35 using the taste information stored in the section 34, retrieves music information of a candidate music and outputs the information to the section 31.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an information processing apparatus, an information processing method, and a medium.
The present invention relates to an information processing apparatus and method capable of searching for predetermined information, and a medium.

[0002]

2. Description of the Related Art When a user operates a keyboard and inputs predetermined search information such as lyrics and a singer's name, the user is provided with related information (for example, a song name) of a song specified by the search information. The system exists.

[0003]

However, if the user does not know the information that can be the search information, that is, if appropriate search information is not input, the system does not provide the user with the relevant information of the song. There were challenges.

[0004] Therefore, the user can search information, for example,
Instead of inputting via the keyboard, a system is conceivable in which a melody is input by voice, and the user is provided with relevant information of a song that may have the same melody as the melody. However, there was a problem that the specific contents of the processing in this system were not disclosed.

[0005] The present invention has been made in view of such circumstances, and specifically discloses a process of providing to a user information related to a song that may have a melody input by voice. is there.

[0006]

According to a first aspect of the present invention, there is provided an information processing apparatus comprising: first storage means for storing information on a plurality of music pieces and melody data of the plurality of music pieces; and a voice transmitted from a terminal. Receiving means for receiving data; generating means for generating melody data from the audio data received by the receiving means; melody data of a plurality of songs stored in the first storage means; Calculating means for calculating the degree of correlation with the melody data; determining a candidate song based on the degree of correlation calculated by the calculating means; and storing information on the determined candidate song in the first storage means. Characterized in that it comprises a search means for searching from information related to the song, and an output means for outputting the information searched by the search means to a terminal.

In the information processing apparatus according to a second aspect, the generating means uses the reference item in which the predetermined setting value is set,
On the voice data, a plurality of pitch calculation sections and pitch calculation sections are determined, and the length of the determined pitch calculation section is defined as a pitch,
The pitch can be calculated from the voice data of the determined pitch calculation section.

According to a third aspect of the present invention, in the information processing apparatus, the generating means calculates the pitch from the voice data of the first pitch calculation section according to the first calculation method, and calculates the first pitch. When the pitch calculated from the voice data of the section is larger than a predetermined value, the pitch is calculated from the voice data of the remaining pitch calculation section according to the first calculation method, and the pitch of the first pitch calculation section is calculated. When the pitch calculated from the voice data is smaller than the predetermined value, the pitch can be calculated from the voice data of the remaining pitch calculation section according to the second calculation method.

In the information processing apparatus according to a fourth aspect, the receiving means further receives the music ID transmitted from the terminal, and the generating means can update the set value set in the reference item.

[0010] An information processing apparatus according to claim 5 is a user
A second storage unit for storing profile information corresponding to the ID, the receiving unit further receiving the user ID transmitted from the terminal, and the calculating unit corresponding to the user ID received by the receiving unit Then, the calculated degree of correlation can be corrected based on the user information stored in the second storage means.

[0011] In the information processing apparatus according to claim 6, the receiving means further receives the music ID transmitted from the terminal, and the second storage means stores the music ID based on the music ID received by the receiving means. Profile information can be updated.

According to a seventh aspect of the present invention, there is provided an information processing method comprising: a storage step of storing information on a plurality of music pieces and melody data of the plurality of music pieces; a reception step of receiving voice data transmitted from the terminal; A generating step of generating melody data from the audio data received in the processing of the step; calculating a degree of correlation between the melody data of the plurality of songs stored in the processing of the storing step and the melody data generated in the generating step A calculation step, a search step of determining a candidate song based on the degree of correlation calculated in the process of the calculation step, and searching for information on the determined candidate song from information on a plurality of songs stored in the storage step process. And outputting the information retrieved in the process of the retrieval step to the terminal.

[0013] The medium according to claim 8 is an input step of inputting information on a plurality of music pieces and melody data of the plurality of music pieces, a generating step of generating melody data from audio data transmitted from a terminal, A calculating step of calculating a degree of correlation between the melody data of the plurality of songs input in the processing of the input step and the melody data generated in the processing of the generating step; and a calculating step based on the degree of correlation calculated in the processing of the calculating step. A search step of determining a candidate song, and searching for information on the determined candidate song from information on a plurality of songs input in the input step process, and outputting the information found in the search step process to a terminal; And steps.

An information processing apparatus according to claim 1, and claim 7
In the information processing method described in the above, and in the medium according to claim 8, melody data of a plurality of songs and melody data of the plurality of songs are input, and melody data is generated from audio data transmitted from the terminal. The degree of correlation between the input melody data of the plurality of songs and the generated melody data is calculated, a candidate song is determined based on the calculated degree of correlation, and information on the determined candidate song is stored in a plurality of songs. , And the searched information is output to the terminal.

According to a ninth aspect of the present invention, there is provided an information processing apparatus, comprising: input means for inputting voice; generating means for generating melody data from the voice input by the input means; and melody data generated by the generating means. And output means for outputting to

According to a tenth aspect of the present invention, in the information processing apparatus, the generation means uses the reference item in which the predetermined setting value is set to generate a plurality of pitch calculation sections and pitch calculation sections on the audio data. The pitch can be calculated from the voice data of the determined pitch calculation section, with the length of the determined pitch calculation section determined as the pitch.

In the information processing apparatus according to the eleventh aspect, the generating means calculates the pitch from the voice data of the first pitch calculation section according to the first calculation method, and calculates the first pitch. When the pitch calculated from the voice data of the section is larger than a predetermined value, the pitch is calculated from the voice data of the remaining pitch calculation section according to the first calculation method, and the pitch of the first pitch calculation section is calculated. When the pitch calculated from the voice data is smaller than the predetermined value, the pitch can be calculated from the voice data of the remaining pitch calculation section according to the second calculation method.

In the information processing apparatus according to the twelfth aspect, the input means inputs the music ID, and the generating means can update the set value set in the reference item.

According to a thirteenth aspect of the present invention, in the information processing method, an input step of inputting a voice, a generating step of generating melody data from the voice input in the processing of the input step, and a melody generated by the processing of the generating step. And outputting the data to a server.

According to a fourteenth aspect of the present invention, in the medium, an input step of inputting a voice, a generating step of generating melody data from the voice input in the processing of the input step, and a melody data generated in the processing of the generating step are performed. And an output step of outputting to the server.

An information processing apparatus according to claim 9, wherein
In the information processing method according to the third aspect and the medium according to the fourteenth aspect, a voice is input, melody data is generated from the input voice, and the generated melody data is output to the server.

According to a fifteenth aspect of the present invention, there is provided an information processing apparatus, comprising: receiving means for receiving melody data transmitted from a terminal; first storage means for storing information on a plurality of music pieces and melody data of the plurality of music pieces. Calculating means for calculating a degree of correlation between the melody data of a plurality of songs stored in the first storage means and the melody data received by the receiving means; and a degree of correlation calculated by the calculating means. , Determine a candidate song, and provide information about the determined candidate song,
It is characterized by comprising a search means for searching from information on a plurality of songs stored in the first storage means, and an output means for outputting the information searched by the search means to a terminal.

[0023] The information processing apparatus according to claim 16 further comprises a second storage means for storing profile information corresponding to the user ID, wherein the receiving means receives the user ID transmitted from the terminal, The calculating means can correct the calculated degree of correlation based on the user information stored in the second storage means, corresponding to the user ID received by the receiving means.

[0024] In the information processing apparatus according to the seventeenth aspect, the receiving means receives the music ID transmitted from the terminal,
Can update the profile information based on the music ID received by the receiving means.

An information processing method according to claim 18 is a receiving step of receiving melody data transmitted from a terminal, a storing step of storing information on a plurality of music pieces and a melody data of a plurality of music pieces, and a storage step. Based on the melody data of a plurality of songs stored in the processing of the step, the calculation step of calculating the degree of correlation with the melody data received in the processing of the receiving step, based on the degree of correlation calculated in the processing of the calculation step, A search step of determining a candidate song and searching for information on the determined candidate song from the information on a plurality of songs stored in the storage step process, and outputting the information found in the search step process to a terminal And a step.

According to a nineteenth aspect of the present invention, there is provided the medium, comprising: an input step of inputting information on a plurality of music pieces and melody data of the plurality of music pieces; a melody data of the plurality of music pieces input in the input step processing; A calculating step of calculating the degree of correlation with the transmitted melody data; and a candidate song is determined based on the degree of correlation calculated by the processing of the calculating step. It is characterized by comprising a search step of searching from information on a plurality of input songs, and an output step of outputting information searched by the processing of the search step to a terminal.

[0027] In the information processing apparatus according to the fifteenth aspect, the information processing method according to the eighteenth aspect, and the medium according to the nineteenth aspect, information on a plurality of music pieces and melody data of the plurality of music pieces are input. The degree of correlation between the input melody data of a plurality of songs and the melody data transmitted from the terminal is calculated, and based on the calculated degree of correlation, a candidate song is determined, and information on the determined candidate song is determined. Is searched from information on a plurality of songs, and the searched information is output to the terminal.

[0028]

FIG. 1 shows a configuration example of a music information search service system to which the present invention is applied. Terminal 1
Is connected to the network 2 by wire or wirelessly, and outputs a voice (for example, a melody of a certain song) pronounced by the user,
It is converted into an electronic signal (voice data) and transmitted to the search server 3. The terminal 1 also presents the information transmitted from the search server 3 to the user. The search server 3 receives the voice data transmitted from the terminal 1, searches for the corresponding information, and transmits the information to the terminal 1.

Next, the operation will be described. When the user pronounces one sound, for example, “La” and inputs a melody of a certain song to the terminal 1, the terminal 1 converts the user's voice (melody of the song) into voice data and transmits it to the search server 3. I do. The search server 3 searches for music information (song name, lyrics, singer name, etc.) of a song that may have a melody transmitted from the terminal 1 and transmits the music information to the terminal 1. Terminal 1
Presents the music information from the search server 3 to the user by voice or text. Thus, for example, the user inputs the melody of a song whose name is not known to the terminal 1 by voice, thereby receiving music information related to the song and knowing the song name.

FIG. 2 shows a configuration example of the terminal 1. CP
U11 executes various processes according to a program stored in ROM12. The RAM 13 appropriately stores data and the like necessary for the CPU 11 to execute various processes.

The voice input unit 14 is composed of a microphone or the like, and converts voice generated by the user into voice data. The operation unit 15 is appropriately operated when a command for a predetermined process is input to the CPU 11. The display unit 16 is, for example,
It is constituted by a liquid crystal display or the like, and displays information to be presented to the user. The audio output unit 17 includes a speaker or the like, and outputs information to be presented to the user by audio.

The communication unit 18 transmits data (for example, voice data and ID) to the network 2 and receives data (for example, music information) transmitted via the network 2. The interface 19 is a voice input unit 1
4 to the communication unit 18 and the CPU 11 to execute interface processing.

That is, the terminal 1 has a voice data generation function of generating voice data from a user's voice and a communication function of executing communication with the search server 3.

FIG. 3 shows a configuration example of the search server 3. The communication unit 31 receives data (for example, audio data or ID) transmitted via the network 2 and outputs the data to the melody data generation unit 32 or transmits music information supplied from the search unit 33 to the network 2. Via terminal 1
Send to

The profile information storage unit 34 stores, for each user, a predetermined set value in addition to the user's age, gender, preference information (for example, the genre of the song that the user likes), the use history of the music information search service, and the like. The stored reference items (described later) are stored. Profile information storage unit 34
Also stores information on the sales of songs.

The melody data generation unit 32 generates melody data based on the voice data from the communication unit 31 by using the reference item in which a predetermined set value is stored, stored in the profile information storage unit 34. Then, it outputs to the search unit 33. The melody data is data (hereinafter, referred to as a phoneme) including data indicating the length of one sound (hereinafter, referred to as a pitch) and data indicating the pitch of the sound (hereinafter, referred to as a pitch). It is a series. The melody data generation unit 32 also performs a process of changing the set value of the reference item stored in the profile information storage unit 34.

The music information storage unit 35 stores melody data, song title, singer name, ISRC (International Standard
Recording Code). In the case of this example, the melody data stored in the music information storage unit 35 is a MIDI (Musical Instrument Digital Interface) comprising time data and frequency data for one sound.
Data.

The search unit 33 stores the profile information storage unit 3
4 and the melody data supplied from the melody data generation unit 32 and the melody data of each song stored in the music information storage unit 35 using the preference information and the sales information of the song. The communication unit 31 calculates the degree of correlation and searches for music information of the candidate song (one or more songs).
Output to The search unit 33 also performs a process of updating the preference information stored in the profile information storage unit 34.

Next, an outline of the song name search processing will be described with reference to the flowchart of FIG. In step S1, the user operates the operation unit 15 of the terminal 1 and inputs information for identifying himself (in this example, a user ID). In step S <b> 2, the user generates one sound, for example, “la” from the voice input unit 14 of the terminal 1 and inputs a part or all of the melody of the music whose title is desired to be known.

In step S3, the terminal 1 transmits the user ID input in step S1 and the voice data of the voice input in step S2 to the search server 3 via the network 2.

In step S4, the search server 3
Melody data is generated based on the user ID and the voice data transmitted from the terminal 1, and in step S5, based on the generated melody data, the candidate tune (1
Song or multiple songs), along with the song title of the candidate song, lyrics,
Predetermined music information such as a singer's name and a genre is searched and transmitted to the terminal 1 in step S6. In addition,
Details of the processing in step S4 and step S5 will be described later.

In step S 7, the terminal 1 displays the music information of the candidate tune transmitted from the search server 3 in characters on the display unit 16 or outputs it in voice via the voice output unit 17 to the user. Present. In step S8,
The user searches for the title of the song he or she wants to know from the information presented via the display unit 16 or the audio output unit 17. In this way, the user can know the song title of the song for which the melody has been input in step S2. After searching for a song title, the user operates the operation unit 15 of the terminal 1 to select the song (song title).

In step S 9, the terminal 1 transmits the selection result (for example, the song ID of the song) to the search server 3 via the network 2. In step S10, the search server 3 updates profile information (for example, reference item setting values and preference information) based on the song ID transmitted from the terminal 1. Thereafter, the process ends. The details of the processing in step S10 will be described later.

Next, the details of the processing (melody data generation processing) in step S4 will be described with reference to the flowchart of FIG. The melody data generator 32 of the search server 3 performs low-pass filter processing with a small cutoff frequency after converting the audio data transmitted from the terminal 1 into an absolute value. The audio data transmitted from the terminal 1 is, for example, data (waveform) including positive and negative components including a high frequency part as shown in FIG.
After being subjected to absolute value conversion and low-pass filtering,
As shown in FIG. 7, the high-frequency portion is eliminated and converted into data (waveform) consisting of only positive components. In addition,
The low-pass filter processing here is a so-called digital filter (FIR (Finite Impulse Response) or IIR
(InfiniteImpulse Response) is used.

Next, in step S22, the melody data generation unit 32 detects the maximum value M of the frequency from the audio data processed in step S21. In step S23, the melody data generation unit 32 determines a set value of a reference item necessary for processing in step S24 described later.

In this example, α1, α2, β1, β2,
There are a total of five types of reference items, i.e., γ and γ. Among them, a predetermined set value is set for each user among a plurality of set values prepared in advance, and stored in the profile information storage unit 34. I have. That is, here, the melody data generation unit 32 transmits the user ID transmitted from the terminal 1.
The reference item having a predetermined set value stored corresponding to (Step S3) is stored in the profile information storage unit 3.
Read from # 4. In this example, it is assumed that the following setting values are set in the read reference items.

Α1 = 0.1, α2 = 1.5, β1 = 0.
73, β2 = 0.5, γ = 1.1 In step S24, the melody data generation unit 32
Determines a maximum point detection start point S, a maximum point P, a minimum point detection start point E, and a minimum point B necessary for determining phonemes (tone length and pitch). Details of the processing here are shown in FIG.
10 to FIG. In describing this process, an outline thereof will be described first, and then a more specific description will be given based on the example of FIG.

In step S41, the melody data generating section 32 sets sample points on the audio data (for example, FIG. 7) subjected to the processing in step S21 in FIG. 5, and calculates the total number T of the set sample points. get.

In step S42, the melody data generator 32 initializes various parameters used in the following processing as follows.

Lmax = 0, Lpmax = 0, Lmin = maximum value M, Lpmin = 0, counter i = 2, counter j = 0, flag = 0 In step S43, the melody data generator 32
Determines whether or not the value of the counter i is equal to the total number T of the sample points.
Go to 44.

In step S44, the melody data generator 32 outputs the (i-2) th sample point (hereinafter, referred to as the (i-2) th sample point).
Described as sample point (i-2). The same applies to other sample points) in the Y-axis direction (for example, the value in the Y-axis direction in FIG. 7) (hereinafter referred to as a value Y (i-2). Is the same as the above), the value Y of the sample point (i-1)
(i-1), the value Yi of the sample point i, and the value Y (i
+1) and whether the value Y (i + 2) of the sample point (i + 2) satisfies the relationship of the following equations (1) and (2) (equation (1)
And whether the equation (2) holds). That is, sample point (i-2), sample point (i-1), sample point
For example, it is determined whether or not i, the sample point (i + 1), and the sample point (i + 2) have a positional relationship that is convex upward in FIG.

Value Y (i-2) <Value Y (i-1) <Value Yi (1) Value Yi> Value Y (i + 1)> Value Y (i + 2) (2 If it is determined in step S44 that the expressions (1) and (2) are satisfied, that is, if the positional relationship is convex upward, the process proceeds to step S45, and the melody data generation unit 32 determines the expression (3) ) To determine whether
If it is determined that the condition holds, the process proceeds to step S46.

Value Yi> Lmax (3) In step S46, the melody data generator 32
Gives Lmax to the value Yi and Lpmax to the X of the sample point i.
A value in the axial direction (hereinafter referred to as a value Xi; the same applies to other values in the X-axis direction). Note that the value Xi
Is the value itself of the counter i, and indicates the number of the sample point.

Next, in step S47, the melody data generating section 32 determines whether or not the flag is set to 0. If it is determined that 0 is set, the process proceeds to step S48, where the formula (4) ) Is determined, and if it is determined that the condition is satisfied, the process proceeds to step S49.

Lmax> (maximum value M × α1 (= 0.1)) (4) In step S49, the melody data generation unit 32
Determines whether the value of the counter j is 0, and if it is not 0, proceeds to step S50, and determines whether or not Expression (5) is satisfied.

Lmax> (value YB × α2 (= 1.5) of the minimum point B detected immediately before) (5) In step S49, it was determined that the value of the counter j was 0 In this case, or when it is determined in step S50 that the expression (5) is satisfied, the process proceeds to step S51, where the melody data generating unit 32 sets the value Xi set at this time to Lpmax (the value set in step S46). The sample point of Xi) is determined as the maximum point detection start point S. At this time, the melody data generation unit 32
The sample point of the value Xi set to in (the value Xi set in step S63) is determined as the minimum point B.

Next, proceeding to step S52, the melody data generator 32 sets 1 to a flag, and proceeds to step S5.
In 3, the value of the counter j is incremented by 1, and the process proceeds to step S54.

If it is determined in step S45 that the expression (3) does not hold, in step S47,
When it is determined that 0 is not set in the flag, when it is determined in step S48 that expression (4) does not hold, or when it is determined in step S50 that expression (5) does not hold, Proceed to step S54.

In step S54, the melody data generator 32 increments the value of the counter i by one, and thereafter returns to step S43.

If it is determined in step S44 that the formulas (1) and (2) do not hold, the process proceeds to the step S44.
Proceeding to 55, the melody data generation unit 32 determines that the value Y (i−
2) Whether the values of the value Y (i-1), the value Yi, the value Y (i + 1), and the value Y (i + 2) satisfy the following equations (6) and (7): It is determined whether or not Expression (6) and Expression (7) hold. That is, the sample point (i-2) and the sample point (i-
1), sample point i, sample point (i + 1), and sample point
It is determined whether (i + 2) has a downwardly convex positional relationship.

Value Y (i-2)> Value Y (i-1)> Value Yi (6) Value Yi <Value Y (i + 1) <Value Y (i + 2) (7 If it is determined in step S55 that the expressions (6) and (7) are satisfied, that is, if there is a downwardly convex positional relationship, the process proceeds to step S56, and the melody data generation unit 32 sets the flag to 1 Is determined, and if it is determined that 1 is set, the process proceeds to step S57.

In step S57, the melody data generating section 32 determines whether or not the expressions (8) and (9) are satisfied.
Proceed to.

Value Yi> Lmax × β1 (= 0.73) (8) Value Yi> Maximum M × β2 (= 0.5) (9) In step S58, the melody data generation unit 32
Sets the sample point of the value Xi set at this time to Lpmax (the value Xi set in step S46) to the local maximum point P
And the sample point indicated by the value of the counter i at this time is determined as the minimum point detection start point E.

In step S59, the melody data generator 32 sets Lmin to the maximum value M, Lmax to the value obtained by multiplying the value Yi by γ (= 1.1), and 0 to the flag.

If it is determined in step S57 that the expressions (8) and (9) do not hold, the flow advances to step S57.
Proceeding to 60, the melody data generating unit 32 calculates
It is determined whether or not holds, and when it is determined that
Proceed to step S61.

Lmin <value Yi (10) In step S61, the melody data generator 32
Sets the value Yi to Lmin and the value Xi to Lpmin.

If it is determined in step S56 that 1 is not set in the flag, the process proceeds to step S62, where the melody data generation unit 32 determines whether or not the equations (11) and (12) hold. If it is determined that the above holds, the process proceeds to step S63.

Lmin> value Yi (11) Lmax> value Yi (12) In step S63, the melody data generator 32
Is the value Yi for Lmin, the value Xi for Lpmin, and Lmax
Is set to a value obtained by multiplying the value Yi by γ.

If it is determined in step S55 that equations (6) and (7) do not hold, step S55 is reached.
If it is determined in step S60 that the expression (10) does not hold, in step S61, when Lmin and Lpmin are set to predetermined values, in step S62,
When it is determined that the expressions (11) and (12) do not hold, or in step S63, Lmin, Lpmi
When predetermined values are set for n and Lmax, the process returns to step S54.

If it is determined in step S43 that the value of the counter i is equal to the total number T of sample points, the flow advances to step S64 to determine whether or not the last minimum point B has been determined. When the determination is made, the process proceeds to step S65, where the (value Xi-1) th sample point is determined as the last minimum point B.

In step S64, the last minimum point B
Is determined, or when the last minimum point B is determined in step S65, the process ends, and the process proceeds to step S25 in FIG.

Next, the above-described processing will be described again using the audio data of FIG. 7 as an example. In step S41, sample points are set, and the total number T is obtained.
FIG. 11 shows sample points set in the section L of the audio data in FIG. FIG. 11 shows sample points (n−2) to (n + 25).

When the parameters are set as described above in step S42, in step S43
It is determined whether or not the value of the counter i is equal to the total number T of sample points.
Go to 44. In this case, since the value of the counter i is n (<T) at this time, the process proceeds to step S44.

In the example of FIG. 11, the value Y of the sample point (n-2)
(n-2), value Y (n-1) of sample point (n-1), value Yn of sample point n, value Y (n + 1) of sample point (n + 1), and sample point
Since the value Y (n + 2) of (n + 2) has the following relationship, it is determined in step S44 that the equations (1) and (2) do not hold. (6)
And it is determined that Expression (7) does not hold, and step S
Proceed to 54.

Value Y (n−2) <Value Y (n−1) <Value Yn <Value Y (n + 1) <Value Y (n + 2) In step 54, the value of the counter i is incremented by one. After that, the process returns to step S43, and the subsequent processes are executed. In the example of FIG. 11, the value Y (n + 1), the value Y (n + 2), the value Y (n + 3), and the value Y (n + 4) and the value Y (n + 5)
Since the relations of the equations (1) and (2) are satisfied, that is, the sample point (n + 1), the sample point (n + 2), and the sample point (n
+3), the sample point (n + 4), and the sample point (n + 5) are in a positional relationship convex upward, so that the value of the counter i is (n
+3) until steps S43, S44, S55,
The processing in S54 is repeatedly performed.

When the value of the counter i is incremented to (n + 3) in step S54, it is determined in step S44 that the expressions (1) and (2) hold, and the flow advances to step S45. When the value of the counter i is (n + 3), since Lmax is set to 0 (since the value is initially set to 0 by the processing in step S42), the value Y (n + 3) becomes Lmax ( = 0), and the process proceeds to step S46.

In step S46, the value Y (n +
3) and to Lpmax the value X (n + 3) (ie (n +
3)) is set. Next, in step S47,
It is determined whether or not the flag is set to 0. If it is determined that 0 is set, the process proceeds to step S48. When the value of the counter i is (n + 3), the flag
Since 0 has been set (since the initial setting to 0 has been performed by the processing in step S42), step S
Go to 48.

In step S48, it is determined whether or not equation (4) holds, that is, Lmax is set to α1 (=
It is determined whether or not the value is larger than the value obtained by multiplying by 0.1). If it is determined that the value is larger, the process proceeds to step S49. When the value of the counter i is (n + 3), the value Y (n + 3) is set in Lmax. Therefore, when the value of the counter i is (n + 3), the value Y (n +3) is still set), the value Y (n + 3) becomes α
It is determined whether or not the value is larger than a value obtained by multiplying by one. In this example, since the value Y (n + 3) is larger than the value, the process proceeds to step S49.

In step S49, it is determined whether or not the value of the counter j is 0. If it is determined that the value is 0, the process proceeds to step S51. When the value of the counter i is (n +
At the time of 3), since the value of the counter j is set to 0 (since the value is initially set to 0 by the processing in step S42), the process proceeds to step S51.

In step S51, the sample point of the value Xi set to Lpmax is determined as the local maximum point detection start point S. When the value of the counter i is (n + 3), L
Since the value X (n + 3) is set in pmax (since the value X (n + 3) is still set by the processing in step S46 when the value of the counter i is (n + 3)), The (n + 3) th sample point (sample point (n + 3))
As shown in FIG. 7, the maximum point detection start point S1 of the first phoneme is determined. At this time, the sample point of the value Xi set in Lpmin is determined as the minimum point B. When the value of the counter i is (n + 3), Lpmin is set to:
Since 0 has been set (since it has been set to 0 by the processing in step S42), in this case, the minimum point B is not determined.

As described above, when the maximum point detection start point S1 is determined, the flag is set to 1 in step S52, the value of the counter j is incremented by 1 in step S53, and the process returns to step S54.

Next, in step S54, the value of the counter i is incremented by one, and then in step S43
And the subsequent processing is executed. In the case of the example of FIG. 11, the value Y (n + 3), the value Y (n + 4), the value Y (n + 5), and the value Y (n + 6 ),
And the value Y (n + 7) satisfies the relations of the equations (6) and (7), ie, the sample point (n + 3) and the sample point
Since (n + 4), the sample point (n + 5), the sample point (n + 6), and the sample point (n + 7) are in a positional relationship of being convex downward,
Until the value of the counter i becomes (n + 5), step S4
3. The processes in S44, S55, and S54 are repeatedly performed.

In step S54, when the value of the counter i is incremented to (n + 5), it is determined in step S55 that the equations (6) and (7) hold, the process proceeds to step S56, and 1 is set in the flag. It is determined whether or not is set. When the value of the counter i is (n + 5), 1 is set in the flag, so (Step S52 when the value of the counter i is (n + 3))
(1 remains set as a result of the above processing), and the process proceeds to step S57.

In step S57, it is determined whether or not the expressions (8) and (9) are satisfied. If it is determined that the expressions are not satisfied, the process proceeds to step S60. When the value of the counter i is (n + 5), the value Y (n + 3) is set in Lmax. Therefore, (the value Y (n) is obtained by the processing in step S46 when the value of the counter i is (n + 3). +3) is still set), in this case, the value Y (n + 5) becomes the value Y (n + 3)
Is determined to be greater than a value obtained by multiplying the maximum value M by β2 (= 0.5). In this example, the value Y (n +
5) is larger than the value obtained by multiplying the value Y (n + 3) by β1 (although the formula (8) holds), but smaller than the value obtained by multiplying the maximum value M by β2 (the formula (9) does not hold) Therefore, the process proceeds to step S60.

In step S60, it is determined whether or not the expression (10) is satisfied. If it is determined that the expression is satisfied, the process proceeds to step S61. When the value of the counter i is (n +
In the case of 5), since the maximum value M is set in Lmin (since the maximum value M is still set by the processing in step S42), the value Y (n + 5) is smaller than the maximum value M. The process proceeds to step S61.

In step S62, the value Y (n +
5), and the value X (n + 5) is set to Lpmin, and the process returns to step S54.

In step S54, after the value of the counter i is incremented by one, the process returns to step S43, and the subsequent processing is executed. In the case of the example of FIG. 11, the value (n + 7) and the value Y (n + 8), the value Y (n + 9), the value Y (n + 10), and the value Y (n + 11) satisfy the relationship of Expressions (1) and (2), that is, Point (n + 7), sample point (n +
8), the sample point (n + 9), the sample point (n + 10), and the sample point (n + 11) are in a positional relationship of being convex upward, so that the value of the counter i is (n + 9). Until step S4
3. The processes in S44, S55, and S54 are repeatedly performed.

When the value of the counter i is incremented to (n + 9) in step S54, it is determined in step S44 that the equations (1) and (2) hold, and the flow advances to step S45. When the value of the counter i is (n + 9), the value Y (n + 3) is set in Lmax. Therefore, (the value Y (n) is obtained by the process in step S46 when the value of the counter i is (n + 3). +3) is still set), it is determined whether the value Y (n + 9) is greater than the value Y (n + 3). In this example, the value Y (n + 9) is the value Y
Since it is larger than (n + 3), the process proceeds to step S46.

In step S46, the value Y is set to Lmax.
The value Y (n + 9) is substituted for (n + 3), and the value X (n +
A value X (n + 9) is set instead of 3). Next, step S
In 47, it is determined whether or not 0 is set in the flag. If it is determined that 0 is not set, the process returns to step S54. When the value of the counter i is (n + 9), 1 is set in the flag (since the value of the counter i is still set to 1 by the processing in step S52 when the value of the counter i is (n + 3)), the process proceeds to step S54. Return to

After the value of the counter i is incremented in step S54, the process returns to step S43, and the subsequent processing is executed. In the case of FIG.
(n + 11), the value Y (n + 12), the value Y (n + 13), the value Y (n + 14), and the value Y (n + 15) are expressed by the following equations (6) and (7). Satisfies the relationship, i.e., sample point (n + 11), sample point (n + 1
2) Since the sample point (n + 13), the sample point (n + 14), and the sample point (n + 15) have a downwardly convex positional relationship, the value of the counter i becomes (n + 13). Until step S4
The processing in 3, S44, S55, and S54 is repeatedly performed.

When the value of the counter i is incremented to (n + 13) in step S54, it is determined in step S55 that the equations (6) and (7) hold, and the flow advances to step S56. Step S5
In 6, it is determined whether or not 1 is set in the flag. If it is determined that 1 is set, the process proceeds to step S57. When the value of the counter i is (n + 13),
Since 1 is set in the flag (since 1 remains set by the processing in step S52 when the value of the counter i is (n + 3)), the flow proceeds to step S57.

In step S57, it is determined whether or not the expressions (8) and (9) are satisfied. If it is determined that the expressions are satisfied, the process proceeds to step S58. When the value of the counter i is (n + 13), the value Y (n + 9) is set in Lmax. Therefore, when the value of the counter i is (n + 9), the value Y (n +9) is still set), it is determined whether the value Y (n + 13) is greater than the value obtained by multiplying the value Y (n + 9) by β1, and β2 is set as the maximum value M. It is determined whether the value is larger than the multiplied value. In this example, the value Y (n + 13) is larger than a value obtained by multiplying the value Y (n + 9) by β1 (formula (8) holds), and the maximum value M is β
Since the value is larger than the value obtained by multiplying 2 (formula (9) is satisfied), the process proceeds to step S58.

In step S58, the sampling point of the value Xi set to Lpmax is determined as the maximum point P. When the value of the counter i is (n + 13), the value X (n + 9) is added to Lpmax. ) Is set (since the value X (n + 9) is still set by the process in step S46 when the value of the counter i is (n + 9)),
The sample point 9) (sample point (n + 9)) is determined as the maximum point P1 of the first phoneme as shown in FIG. The sample point indicated by (n + 13) of the value of the counter i, that is, the (n + 13) th sample point (sample point (n + 13)) is the minimum point detection start point E of the first sound.
Determined to be 1.

Next, the process proceeds to step S59, where Lmin is set to the maximum value M, Lmax is set to the value obtained by multiplying the value Y (n + 13) by γ, and the flag is set to 0, and the process returns to step S54.

In step S54, after the value of the counter i is incremented, the process returns to step S43, and the subsequent processing is executed. In the case of the example of FIG.
(n + 17), the value Y (n + 18), the value Y (n + 19), the value Y (n + 20), and the value Y (n + 21) are expressed by the following equations (6) and (7). Satisfies the relationship, i.e., sample point (n + 17), sample point (n + 1
8) Since the sample point (n + 19), the sample point (n + 20), and the sample point (n + 21) are in a positional relationship that becomes convex downward, the value of the counter i becomes (n + 19). Until step S4
The processing in 3, S44, S55, and S54 is repeatedly performed.

When the value of the counter i is incremented to (n + 19) in step S54, it is determined in step S55 that the equations (6) and (7) hold, and the flow advances to step S56. Step S5
In 6, it is determined whether or not 1 is set in the flag. If it is determined that 1 is not set, the process proceeds to step S62. When the value of the counter i is (n + 19), since 0 is set in the flag,
Is set to 0 by the processing in step S59 when the value of is (n + 13) (step S6).
Proceed to 2.

In step S62, it is determined whether or not the expressions (11) and (12) are satisfied. If it is determined that the expressions are satisfied, the process proceeds to step S63. When the value of the counter i is (n + 19), the maximum value M is set to Lmin (with the maximum value M being set by the processing in step S59 when the value of the counter i is (n + 13)), Lmax Is set to a value obtained by multiplying the value Y (n + 13) by γ. (By the processing in step S59 when the value of the counter i is (n + 13), γ is added to the value Y (n + 13). Since the multiplied value remains set), it is determined whether the value Y (n + 19) is smaller than the maximum value M and smaller than the value obtained by multiplying the value Y (n + 13) by γ. Is determined. In this example, the value Y (n + 19) is smaller than the maximum value M (formula (11) holds) and smaller than the value obtained by multiplying the value Y (n + 13) by γ (formula (12)). ) Is satisfied), and the process proceeds to step S63.

In step S63, the value Y (n +
19) is set, the value X (n + 19) is set to Lpmin, and Lmax
Is multiplied by the value Y (n + 19) by γ (= 1.1). Then, the process returns to step S54.

In step S54, the value of the counter i is incremented, and the process returns to step S43 to execute the subsequent processing. In the case of the example shown in FIG. 11, the value Y (n + 1
9), value Y (n + 20), value Y (n + 21), value Y (n + 22), and value Y
Since (n + 23) satisfies the relations of the equations (1) and (2), ie, the sample point (n + 19), the sample point (n + 20),
Since the sample point (n + 21), the sample point (n + 22), and the sample point (n + 23) have a positional relationship of being convex upward, step until the value of the counter i becomes (n + 21). S43,
The processing in S44, S55, and S54 is repeatedly performed.

When the value of the counter i is incremented to (n + 21) in step S54, it is determined in step S44 that the equations (1) and (2) hold, and the flow advances to step S45.

When the value of the counter i is (n + 21), L
Since max is set to a value obtained by multiplying the value Y (n + 19) by γ, the value of γ of the value Y (n + 19) is obtained by the processing in step S63 when the value of the counter i is (n + 19). Since the double value is still set), in step S45, the value Y
It is determined whether (n + 21) is greater than a value obtained by multiplying the value Y (n + 19) by γ. In this example, the value Y (n + 21) is the value Y (n
+19) is larger than the value obtained by multiplying by γ (because Expression (3) is satisfied), and the process proceeds to step S46.

In step S46, the value Y (n +
21), and the value X (n + 21) is set to Lpmax. Next, in step S47, it is determined whether or not the flag is set to 0. If it is determined that 0 is set, the process proceeds to step S48. When the value of the counter i is (n
+21), since 0 is set in the flag (step S5 when the value of the counter i is (n + 13))
Since the process at 9 keeps 0 set),
Proceed to step S48.

In step S48, it is determined whether or not the expression (4) is satisfied. If it is determined that the expression (4) is satisfied, the process proceeds to step S49, but the value of the counter i becomes (n + 21).
At this time, since the value Y (n + 21) is set in Lmax (step S4 when the value of the counter i is (n + 21))
6, the value Y (n + 21) is still set), so it is determined whether the value Y (n + 21) is larger than the maximum value M multiplied by α1. In this example, the value Y (n + 21)
Is larger than the value obtained by multiplying the maximum value M by α1, and the process proceeds to step S49.

In step S49, it is determined whether or not the value of the counter j is 0. If it is determined that the value is not 0, the process proceeds to step S50. When the value of the counter i is (n +
At 21, the value of the counter j is not 0 (since the value of the counter i is incremented by 1 by the processing in step S53 when the value of the counter i is (n + 3)), and the process proceeds to step S50.

In step S50, it is determined whether or not the equation (5) is satisfied. If it is determined that the equation (5) is satisfied, the flow advances to step S51. When the value of the counter i is (n + 21), the value Y (n + 21) is set in Lmax.
It is determined whether the value Y (n + 21) is greater than the value obtained by multiplying the value YB of the minimum point B by α2. In this case, since the minimum point B has not been determined yet, the value YB is set to 0. That is, it is determined that the equation (5) holds, and step S51 is performed.
Proceed to.

When the value of the counter i is (n + 21), L
The value X (n + 21) is set in pmax (since the value X (n + 21) is still set by the processing in step 46 when the value of the counter i is (n + 21)), and Lpm
Since the value X (n + 19) is set in in (counter i
Is (n + 19), the value X (n + 19) is still set by the processing in step 63). In step 51, the (n + 19) th sample point (sample point (n + 19) ) Is determined as the minimum point B1 of the first phoneme, and the (n + 21) -th sample point (sample point
(n + 21)) is determined as the maximum point detection start point S2 of the second phoneme.

As described above, the minimum point B1 of the first phoneme
When the maximum point detection start point S2 of the second phoneme is determined, the process proceeds to step S52, where 1 is set in the flag. In step S53, the value of the counter j is incremented by 1, and the process returns to step S54.

Thereafter, the processing in steps S43 to S63 is repeatedly executed. In the case of FIG. 7, the maximum point detection start point S1 to the maximum point detection start point S8, the maximum point P1 to the maximum point P8, and the minimum point detection start point E1 Or minimum point detection start point E
8, and the minimum points B1 to B7 are determined.

If it is determined in step S43 that the value of the counter i is equal to the total number T of sample points, the flow advances to step S64. If the value of the counter i is T, the minimum point B is determined in step S51. If it is determined that the minimum point B has not been determined, the process proceeds to step S65. In the case of the example in FIG. 7, when the value of the counter i is T, in step S51, the minimum point B
Is not determined, that is, since the user's voice is interrupted halfway, the process proceeds to step S65 and the (T-
1) The sample point (T-1) is set as the minimum point B8. As described above, when the last minimum point B8 is determined, the processing is terminated, and the process proceeds to step S25 in FIG.

As described above, the maximum point detection start point S,
The maximum point P, the minimum point detection start point E, and the minimum point B are determined.

As described above, in step S51,
After the maximum point detection start point S and the minimum point B have been determined, the flag is set to 1 in step S52, and if the flag is set to 1 in step S47, the process returns to step S54. If the sample point has a positional relationship of convex upward (step S44) and the value Yi is larger than Lmax (step S45), Lma
x and Lpmax are changed. That is, when 1 is set in the flag, the values Y and X of the sample points that can be the local maximum point P are set as needed in Lmax and Lpmax.

As described above, after the maximum point P and the minimum point detection start point E are determined in step S58, the flag is set to 0 in step S59, and the flag is set to 0 in step S56. If so, the process proceeds to step S62, where the formula (11) and the formula (1)
When 2) holds (step S62), step S6
In 3, since Lmin and Lpmin are changed, when the flag is set to 0, the values Y and X of the sample points that can be the minimum point B are set in Lmin and Lpmin as needed.

In the above, when judging whether the positional relationship between the sample points is convex upward or downward, the relationship between the values Y of the five sample points is used. It is also possible to use the relationship between the values Y of seven or seven sample points.

Next, returning to FIG. 5, the processing after step S25 will be described.

In step S25, the melody data generating section 32 determines a sound duration. Specifically, the value X (Sk) of the k-th (= 1, 2,...) -Phoneme maximum point detection start point Sk determined in step S24 and the (k + 1) -th phoneme maximum check The difference from the value X (S (k + 1)) of the output start point S (k + 1) is calculated, and the calculation result is determined as the pitch of the k-th phoneme. In this way, the duration of each phoneme is calculated and determined. Since this is a so-called absolute duration, in this example, the duration of the first phoneme is set to 1 (reference).
And the length of the second and subsequent phonemes is expressed as a ratio to the length of the first phoneme. That is, it is converted to a relative pitch.

Next, in step S26, the melody data generator 32 determines the pitch. Details of this processing are shown in the flowchart of FIG.

In step S81, the melody data generation unit 32 determines, for example, the section between the maximum point detection start point S1 of the first phoneme and the minimum point B1 of the first phoneme of the voice data in FIG. FFT processing is performed on the voice data of the section and the pitch calculation section). Specifically, first, the voice data of the pitch calculation section of the first phoneme includes M
Calculation points are set, and the set M calculation points are grouped by a predetermined number.
FFT processing is performed. FIG. 13 shows a processing result of the FFT processing performed on one group. That is, in the case of this example, FIG.
Processing results as shown in FIG. 3 are obtained for the number of groups. In grouping the operation points, a predetermined number of operation points are included in adjacent groups in an overlapping manner. As a result, components due to discontinuity are not included in the processing result. That is, a Hamming window function or a Hanning window function is used, and the FFT
Processing is performed.

Next, in step S82, the melody data generator 32 determines the pitch of the first phoneme from the processing result of the FFT processing in step S81. Specifically, for each group, the frequency value of each peak obtained from the processing result of the FFT processing (FIG. 13) is summed, and the frequency of the first peak of the processing result having the largest sum is the pitch ( Pitch frequency).

In step S83, the melody data generator 32 determines whether the pitch (pitch frequency) of the first phoneme determined in step S82 is higher than a predetermined frequency f0, and determines that the pitch is higher. In this case, the process proceeds to step S84.

In step S84, the melody data generator 32 performs the same FFT processing as in step S81 on the remaining pitch calculation sections. Next, in step S85, the melody data generation unit 32
The same processing as in step S82 is performed on the processing result of the FFT processing executed in step S84, and the pitch of each phoneme is determined. Thereby, the pitch of the remaining phonemes is determined.

If it is determined in step S83 that the pitch of the first phoneme (pitch frequency) is smaller than the frequency f0, the process proceeds to step S86.

In step S86, the melody data generator 32 performs the same processing as in step S84, and performs FFT processing on the remaining pitch calculation sections. Next, in step S87, the melody data generation unit 32 performs a cepstrum process on the processing result of the FFT process in step S86. That is, the logarithm of the spectrum is obtained for the processing result of the FFT processing, and the inverse FFT processing is performed on them.

Next, in step S88, the melody data generation unit 32 detects a quefrency having the maximum cepstrum in a predetermined area from the processing result of the cepstrum process in step S87, and determines a frequency (pitch) from the detected quefrency. ) Is calculated. In addition, if the cepstrum processing is used, two FFT processings are eventually executed, and the amount of calculation increases, but the pitch can be easily detected.

That is, in this example, when the user's voice is relatively high, the pitch is calculated using the FFT process, and when the user's voice is relatively low, the pitch is calculated using the cepstrum process.

When the pitch of the phoneme is calculated in step S85 or S88, the process proceeds to step S89, where the melody data generation unit 32 sets the pitch of the first phoneme as a reference, and Is expressed as a difference from the pitch of the first phoneme (expressed in cents with a semitone being 100). That is, the pitch is converted to a relative pitch. In this way, when the relative pitch is obtained, the processing is terminated,
It returns to step S5 of FIG.

As described above, the pitch and pitch are obtained (phonemes are obtained). As a result, when the melody data is generated, the melody data generating section 32 outputs the user ID together with the melody data. This is supplied to the search unit 33.

Note that, in the above description, the FF
The frequency value of each peak obtained from the processing result of the T processing (FIG. 13) is summed, and the frequency of the first peak of the processing result having the largest total value is determined as the pitch (pitch frequency). The processing result of the FFT processing obtained for each group may be added to each peak frequency obtained from the processing result of the processing, and the frequency having the largest addition result may be used as the pitch frequency.

Next, details of the music information search processing in step S5 in FIG. 4 will be described with reference to the flowchart in FIG.

At step S101, the search server 3
Search unit 33 reads out, from the profile information storage unit 34, the sales information and the like stored together with the user ID supplied from the melody data generation unit 32, along with the age, gender, and preference information of the user. .

In step S102, the search unit 33
Divides the melody data supplied from the melody data generation unit 32 for each predetermined data amount. Next, in step S103, the search unit 33 performs the DP (Dynamic Processing) matching in step S104, which will be described later, and performs the relative pitch and relative tone conversion based on the melody data (MIDI data) stored in the music information storage unit 35. Calculate the height.

In step S104, the search unit 33
Performs DP matching using the divided portion of the melody data supplied from the melody data generation unit 32 and the relative pitch and relative pitch calculated in step S103. By this process, a penalty value is calculated for each of the divided parts (for each DP matching process), and the total is used as the final processing result here. Note that the divided melody data includes at least several tens of sample points.

In step S105, the search unit 33
Calculates the degree of correlation between the melody data supplied from the melody data generating unit 32 based on the processing result of the DP matching processing executed in step S104 and the information read in step S101. For example, when the user prefers pops, a larger weighting value is added to the processing result of the DP matching processing with the music to be popped, and the degree of correlation is calculated. Also, songs that are not (for example,
A small value weighting value is added to the processing result of the DP matching processing with the enka) to calculate the degree of correlation. Also, for example, similarly, a large weighting value is added to the processing result of the DP matching process with a song that sells well, a correlation is calculated, and the processing result of the DP matching process with a song that does not sell well includes: A small weight value is added to calculate the degree of correlation.

At step S106, the search unit 33
Determines candidate songs based on the degree of correlation calculated in step S105. If only one candidate song is selected, the song with the highest correlation is selected, and if multiple songs are selected, songs of that number are selected in descending order of correlation and are determined as candidate songs. You. In step S107, the search unit 33
Searches the music information storage unit 35 for predetermined music information (song name, lyrics, etc., in addition to the song name) of the determined candidate song.
In this way, when the music information of the candidate music is searched, the process ends, and the process returns to step S6 in FIG.

Next, details of the profile information update processing in step S10 of FIG. 4 will be described with reference to the flowchart of FIG.

In step S9 of FIG. 4, the song ID transmitted from the terminal 1 (the song I of the song selected by the user)
When D) is received by the communication unit 31 and supplied to the melody data generation unit 32 and the search unit 33, step S
In 121, the search unit 33 is stored in the profile information storage unit 34 based on the supplied music ID.
For example, the preference information is updated. The song specified by the song ID is a song that the user likes, and for example, the genre of the song is set in the preference information.

Next, in step S122, the melody data generating section 32 checks the reference items (α1, α2, β1,
One of the combinations of the set values of β2 and γ) is determined, and in step S123, melody data is generated using the reference item. Since the processing here is the same as the processing in step S4 in FIG. 4, description thereof is omitted, but one piece of melody data is generated according to a combination of one set value of the reference item.

Next, in step S124, the search unit 33 determines in step S123 that the melody data generation unit 3
Then, the degree of correlation between the melody data generated in Step 2 and the melody data of the music selected by the user is calculated. Here, the same processing as that in steps S101 to S105 in FIG. 14 is performed, and the description thereof will be omitted. However, this processing calculates one degree of correlation.

In step S125, the melody data generation unit 32 determines whether all combinations of the set values to be set have been set. If it is determined that there are still remaining combinations, step S122.
And the subsequent processing is executed according to another combination. That is, the melody data is generated according to the set values of all the combinations to be set, and the processing of steps 122 to S125 is repeatedly executed until the correlation degree of each of the generated melody data is calculated.

If it is determined in step S125 that all combinations of the set values to be set have been set, the process proceeds to step S126, where the melody data generation unit 32 selects the most significant one of the correlation degrees calculated by the search unit 33. The combination of the set values when a high degree of correlation was calculated,
The profile information storage unit 34 stores the set value of the reference item in association with the user ID. As described above, when the setting value of the reference item is updated, the processing is ended.

In the above description, the case where the user pronounces one sound, for example, “la” and inputs the melody to the terminal 1 has been described as an example.
It is also possible to generate a plosive followed by a vowel (e.g., a turf or a chu) so that the separation between the sounds has a small amplitude. This makes it easier to detect the sound duration. In the above description, an example is given in which the maximum point detection start point S, the maximum point P, the minimum point detection start point E, and the minimum point B are set on the audio data, and the sound duration is determined using these. However, it is also possible to perform voice recognition processing on voice data, extract each sound, and determine the sound length. Furthermore, these methods can be used in combination.

In the above description, the case where the user pronounces one sound, for example, “la” and inputs the melody to the terminal 1 has been described as an example.
(In this example, the terminal 1 is further provided with an interface for this purpose), and the user plays the electronic musical instrument instead of generating words, and obtains the melody of a song for which music information is to be obtained. May be input to the terminal 1. In the case of this example, the pitch of each phoneme constituting the melody can be obtained from the interval between sounds input from the electronic musical instrument, and the pitch can be obtained from a code indicating the pitch from the electronic musical instrument. When only the sound length can be obtained as in the case of an electronic mat, it is also possible to calculate the degree of correlation using only the sound length and determine the candidate music.

In the above description, the case where the terminal 1 transmits the voice data of the input user's voice to the search server 3 has been described as an example. For example, the terminal 1 may transmit the melody data of the search server 3. It has the same function as the generation unit 32, and can itself generate the melody data and transmit it to the search server 3. Although not specified, the terminal 1 may store the input voice of the user and transmit the voice to the search server 3 later (after the input is completed). In this case, when the storage capacity of the memory (for example, the RAM 13) is small, a compact flash, a smart media, a memory stick, or the like may be connected to the terminal 1 to store audio. When the terminal 1 is a portable terminal, for example, in addition to a dedicated portable device, a portable MD reproducing device, a PDA (Personal) having a voice input function, a communication function, or a melody data generation function. Digital
Assistants) or portable televisions. If the terminal 1 does not have the communication unit 18 that communicates with the search server 3, another communication device that enables communication between the terminal 1 and the search server 3 is required.

In the above description, the case where the title, lyrics and singer name of the candidate tune are supplied from the search data 3 to the terminal 1 has been described as an example, but the melody data is transmitted to the terminal 1. To allow the user to view it. This not only makes the selection of the user more accurate, but also enables the user to re-enter the melody by voice after viewing, grasping the melody, and transmitting the melody to the search server 3, and thereby the search server 3 , The setting value of the reference item can be set more appropriately.

In the above description, a plurality of melody data is generated by each combination of the setting values of the reference items at the time of the profile information updating process. , A plurality of melody data can be generated, and a candidate tune can be determined using the melody data. Note that the reference item β2 depends on the user's singing style and is an important item. Therefore, a combination of the set values of other reference items can be provided based on the set value of β2.

In the above description, the music information storage unit 3
Although the case where the melody data stored in No. 5 is MIDI data has been described as an example, it is also possible to use music data stored in a CD or the like.

The series of processes described above can be executed by hardware, but can also be executed by software. When a series of processing is executed by software, a program constituting the software is executed by a computer incorporated in the melody data generation unit 32 or the search unit 33 as dedicated hardware, or a variety of programs. By being installed, it is installed in, for example, a general-purpose personal computer that can execute various functions.

Next, referring to FIG. 16, regarding a medium used to install a program for executing the above-described series of processing in a computer and to make the computer executable, the computer uses a general-purpose personal computer. The following is an example of the case.

As shown in FIG. 16A, a program is stored in a hard disk 302 or a semiconductor memory 3 as a recording medium built in a personal computer 301.
03 in advance.

Alternatively, the program is executed as shown in FIG.
As shown in (B), the floppy disk 311 and the CD-R
OM (Compact Disk-Read Only Memory) 312, MO (Mag
neto-Optical) Disc 313, DVD (Digital Versatile)
Disk) 314, magnetic disk 315, semiconductor memory 31
6, etc., temporarily or permanently stored in a recording medium,
It can be provided as package software.

Further, as shown in FIG. 16 (C), the program can be wirelessly transferred from download site 321 to personal computer 301 via artificial satellite 322 for digital satellite broadcasting, or can be connected to a local area network or the Internet. Network 331
, And can be transferred to the personal computer 301 by wire, and stored in the built-in hard disk 302 or the like in the personal computer 301.

[0151] The medium in the present specification means a broad concept including all these media.

For example, as shown in FIG. 17, the personal computer 301 has a CPU (Central Processing Unit).
it) 342. The CPU 342 has a bus 341
An input / output interface 345 is connected via the input / output interface 345. When a user inputs a command via the input / output interface 345 from an input unit 347 including a keyboard, a mouse, and the like, the CPU 342 responds accordingly. ROM (Read) corresponding to the semiconductor memory 303 of FIG.
(Only Memory) 343 is executed. Alternatively, the CPU 342 may transfer the program stored in the hard disk 302 in advance from the satellite 322 or the network 331 to the communication unit 348.
And the program installed on the hard disk 302 or the floppy disk 311, the CD-ROM 312, and the MO mounted on the drive 349.
Disk 313, DVD 314, or magnetic disk 3
15 and installed on the hard disk 302 in a random access memory (RAM).
y) Load to 344 and execute. Further, the CPU 342
Sends the processing result to, for example, the input / output interface 3
An output is provided to a display unit 346 made up of an LCD (Liquid Crystal Display) or the like, as necessary, via the LCD 45.

In this specification, the steps of describing a program provided by a medium are not limited to processing performed in chronological order in the order described, but are not necessarily performed in chronological order. It also includes processes that are executed individually or individually.

In this specification, the system is defined as:
It represents the entire device composed of a plurality of devices.

[0155]

According to the information processing apparatus according to the first aspect, the information processing method according to the seventh aspect, and the medium according to the eighth aspect, melody data is generated from audio data, and the degree of correlation is calculated. Thus, it is possible to determine a candidate song and search for information on the candidate song.

The information processing apparatus according to claim 9,
According to the information processing method of the third aspect and the medium of the fourteenth aspect, the melody data is generated from the voice and transmitted to the server, so that the information specified by the voice can be provided. .

According to the information processing apparatus described in claim 15, the information processing method described in claim 18, and the medium described in claim 19, the melody data is received and the degree of correlation is calculated. , A candidate song is determined, information on the candidate song is searched, and the search result can be provided to the terminal.

[Brief description of the drawings]

FIG. 1 is a diagram illustrating a configuration example of a music information search service system.

FIG. 2 is a diagram illustrating a configuration example of a terminal 1 in FIG. 1;

FIG. 3 is a diagram illustrating a configuration example of a search server 3 of FIG. 1;

FIG. 4 is a flowchart illustrating a song name search process.

FIG. 5 is a flowchart illustrating a melody data generation process.

FIG. 6 is a diagram illustrating an example of audio data.

FIG. 7 is a diagram illustrating an example of audio data that has been subjected to low-pass filter processing;

FIG. 8 is a flowchart illustrating a process of determining a maximum point detection start point, a maximum point, a minimum point detection start point, and a minimum point.

FIG. 9 is another flowchart illustrating a process of determining a maximum point detection start point, a maximum point, a minimum point detection start point, and a minimum point.

FIG. 10 is another flowchart illustrating a process of determining a maximum point detection start point, a maximum point, a minimum point detection start point, and a minimum point.

FIG. 11 is a diagram showing an example of sample points.

FIG. 12 is a flowchart illustrating a pitch determination process.

FIG. 13 is a diagram showing a processing result of the FFT processing.

FIG. 14 is a flowchart illustrating music information search processing.

FIG. 15 is a flowchart illustrating profile information update processing.

FIG. 16 is a diagram illustrating a medium.

17 is a diagram illustrating a configuration example of a personal computer 301 in FIG.

[Explanation of symbols]

1 terminal, 2 network, 3 search server,
31 communication unit, 32 melody data generation unit, 33
Search unit, 34 Profile information storage unit, 35 Music information storage unit

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat ゛ (Reference) G10L 11/04 G06F 15/403 350C G10L 9/00 B

Claims (19)

[Claims] 1. An information processing apparatus for exchanging data with a terminal via a network, wherein: first storage means for storing information about a plurality of songs and melody data of the plurality of songs; Receiving means for receiving received audio data; generating means for generating melody data from the audio data received by the receiving means; and the melody data of the plurality of songs stored in the first storage means And calculating means for calculating the degree of correlation with the melody data generated by the generating means, based on the degree of correlation calculated by the calculating means,
Determine a candidate song, information on the determined candidate song,
A search unit for searching from the information on the plurality of songs stored in the first storage unit; and an output unit for outputting the information searched by the search unit to the terminal. Information processing device. 2. The method according to claim 1, wherein the generating unit determines a plurality of pitch calculation sections and a plurality of pitch calculation sections on the voice data using a reference item in which a predetermined setting value is set. The information processing apparatus according to claim 1, wherein the length of the length calculation section is a pitch, and a pitch is calculated from the determined voice data of the pitch calculation section. 3. The method according to claim 1, wherein the generating means calculates a pitch from the voice data of the first pitch calculation section according to a first calculation method, and calculates the voice data of the first pitch calculation section. If the pitch calculated from is larger than a predetermined value, the pitch is calculated from the voice data of the remaining pitch calculation section according to the first calculation method, and the first pitch calculation section is calculated. 3. If the pitch calculated from the voice data is smaller than a predetermined value, a pitch is calculated from the voice data of the remaining pitch calculation section according to a second calculation method. An information processing apparatus according to claim 1. 4. The apparatus according to claim 2, wherein the receiving unit further receives a song ID transmitted from the terminal, and the generating unit updates the set value set in the reference item. An information processing apparatus according to claim 1. 5. The apparatus further comprising: a second storage unit that stores profile information corresponding to the user ID, wherein the receiving unit further receives the user ID transmitted from the terminal, The method according to claim 1, wherein the calculated degree of correlation is corrected based on the user information stored in the second storage unit, corresponding to the user ID received by the reception unit. An information processing apparatus according to claim 1. 6. The receiving means further receives a music ID transmitted from the terminal, and the second storage means stores the profile information based on the music ID received by the receiving means. The information processing apparatus according to claim 5, wherein the information is updated. 7. An information processing method of an information processing apparatus for transmitting and receiving data to and from a terminal via a network, comprising: a storage step of storing information on a plurality of music pieces and melody data of the plurality of music pieces; Receiving the received audio data, generating the melody data from the audio data received in the processing of the receiving step, the melody of the plurality of songs stored in the processing of the storage step A calculating step of calculating a degree of correlation between the data and the melody data generated in the generating step; determining a candidate song based on the degree of correlation calculated in the processing of the calculating step; For searching for information related to the plurality of songs stored in the processing of the storage step. Step a, the information searched by the processing in the search step, the information processing method characterized by comprising an output step of outputting to the terminal. 8. An information processing program for exchanging data with a terminal via a network, comprising: an input step of inputting information on a plurality of music pieces and melody data of the plurality of music pieces; A generation step of generating melody data from audio data transmitted from the melody data of the plurality of songs input in the processing of the input step, and the melody data generated in the processing of the generation step A calculating step of calculating a degree of correlation; determining a candidate song based on the degree of correlation calculated in the processing of the calculating step; and information on the determined candidate song, the plurality of pieces of information input in the processing of the input step. A search step for searching from the information on the song, and the information searched for in the processing of the search step; Medium to execute the program characterized by comprising the output step of outputting to the computer. 9. An information processing apparatus for exchanging data with a server via a network, wherein: input means for inputting voice; generating means for generating melody data from the voice input by the input means; The melody data generated by the means;
An information processing apparatus comprising: output means for outputting to the server. 10. The generation means determines a plurality of pitch calculation sections and a plurality of pitch calculation sections on the voice data using a reference item in which a predetermined set value is set, and determines the determined sound. The information processing apparatus according to claim 9, wherein a length of a length calculation section is set as a pitch, and a pitch is calculated from the determined voice data of the pitch calculation section. 11. The method according to claim 1, wherein the generating means calculates a pitch from the voice data of the first pitch calculation section according to a first calculation method, and calculates the voice data of the first pitch calculation section. If the pitch calculated from is larger than a predetermined value, the pitch is calculated from the voice data of the remaining pitch calculation section according to the first calculation method, and the first pitch calculation section is calculated. 10. If the pitch calculated from the voice data is smaller than a predetermined value, a pitch is calculated from voice data of the remaining pitch calculation section according to a second calculation method. An information processing apparatus according to claim 1. 12. The information processing apparatus according to claim 9, wherein the input unit inputs a song ID, and the generation unit updates the set value set in the reference item. 13. An information processing method of an information processing apparatus for exchanging data with a server via a network, wherein: an input step of inputting voice; and melody data is generated from the voice input in the processing of the input step. An information processing method, comprising: a generation step; and an output step of outputting the melody data generated in the processing of the generation step to the server. 14. An information processing program for exchanging data with a server via a network, comprising: an input step of inputting voice; and melody data from the voice input in the processing of the input step. A medium for causing a computer to execute a program, comprising: a generating step of generating; and an output step of outputting the melody data generated in the processing of the generating step to the server. 15. An information processing apparatus for exchanging data with a terminal via a network, a receiving unit for receiving melody data transmitted from the terminal, information on a plurality of songs, and a melody of the plurality of songs. A first storage unit for storing data; a calculation for calculating a degree of correlation between the melody data of the plurality of songs stored in the first storage unit and the melody data received by the reception unit Means, based on the degree of correlation calculated by the calculation means,
Determine a candidate song, information on the determined candidate song,
A search unit for searching from the information on the plurality of songs stored in the first storage unit; and an output unit for outputting the information searched by the search unit to the terminal. Information processing device. 16. The apparatus further comprising: second storage means for storing profile information corresponding to a user ID, wherein the receiving means receives the user ID transmitted from the terminal, and wherein the calculating means comprises The method according to claim 15, wherein the calculated degree of correlation is corrected based on the user information stored in the second storage unit, corresponding to the user ID received by the reception unit. Information processing device. 17. The receiving unit receives a song ID transmitted from the terminal, and the second storage unit updates the profile information based on the song ID received by the receiving unit. The information processing apparatus according to claim 16, wherein: 18. An information processing method of an information processing apparatus for exchanging data with a terminal via a network, comprising: a receiving step of receiving melody data transmitted from the terminal; A storing step of storing the melody data of the song; calculating a degree of correlation between the melody data of the plurality of songs stored in the processing of the storing step and the melody data received in the processing of the receiving step A calculating step; determining a candidate song based on the degree of correlation calculated in the processing of the calculating step; and information regarding the determined candidate song, the information related to the plurality of songs stored in the processing of the storing step. A search step for searching from information; and outputting the information searched for in the processing of the search step to the terminal. An information processing method which comprises the steps. 19. An information processing program for exchanging data with a terminal via a network, comprising: an input step of inputting information on a plurality of music pieces and melody data of the plurality of music pieces; A calculating step of calculating a degree of correlation between the melody data of the plurality of songs input in the processing of the step and the melody data transmitted from the terminal; and the degree of correlation calculated by the processing of the calculating step A search step of determining a candidate song based on the information, and searching for information on the determined candidate song from the information on the plurality of songs input in the processing of the input step; Said information
A medium for causing a computer to execute a program, comprising: an output step of outputting to the terminal.