JP5125956B2 - Range identification system, program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To specify a register in which a user naturally sings, in a register specifying system. <P>SOLUTION: In singing register specification processing, a voice duration period tpc(k) indicating the time for which voice is continued with a volume equal to or larger than a set value is derived on the basis of voice data in each corresponding period T(k) (S330). Furthermore, a total duration period tp(f) resulting from tabulating voice duration periods tpc(k) with respect to each of pitches f corresponding to component sounds in each corresponding period T(k) is derived (S340). A range from the lowest pitch to the highest pitch, wherein the total duration period tp(f) is equal to or longer than a prescribed value, is specified as a first singing register. That is, the total duration period tp(f) has a large value in the case of the pitch f at which voice can be continued with a volume equal to or larger than the set value, thereby a register in which the user can naturally sing is specified. <P>COPYRIGHT: (C)2010,JPO&amp;INPIT

Description

  The present invention relates to a sound range specifying system for specifying a sound range in which a user can sing, and a program for causing a computer to function as a sound range specifying system.

Conventionally, a karaoke apparatus for a user to enjoy singing along with accompaniment music is known.
Among this type of karaoke apparatus, there is a device that specifies a range in which the user can sing and informs the user of the specified range that can be sung.

For example, among voices input via a microphone (that is, voices uttered by the user even once), the highest pitch (highest pitch) is the highest singing sound, and the lowest pitch (lowest pitch) is the lowest singing sound. To do. And as a technique for specifying the range, while singing the music by the user, while detecting the highest singing sound and the lowest singing sound, the pitch range from the detected lowest pitch to the highest pitch (that is, It has been proposed that the user can sing a sound range) (see, for example, Patent Document 1).
JP 2002-73058 A

  However, in the technique described in Patent Document 1, even if the highest singing sound or the lowest singing sound is uttered by the user by force (for example, the pitch when the user performs utterance that may damage the vocal cords). However, since the range from the highest singing sound to the lowest singing sound is set as a singable range, it is difficult to say that the singable range is appropriately specified.

  That is, in the technique described in Patent Document 1, there is a problem that it is not possible to appropriately specify a singing range without the user's excessive effort. Even if the karaoke device proposes a song to the user based on the determination result, There was a possibility that the user could not enjoy karaoke.

  Therefore, an object of the present invention is to specify a sound range that can be sung by a user without overdoing in a sound range specifying system.

  In the range specifying system of the present invention made to achieve the above object, the audio signal acquisition means acquires the audio signal when the user sings the music, and the song data generation means acquires the audio acquired by the audio signal acquisition means. By analyzing the frequency of the signal, singing data representing the transition of the pitch of the singing is generated. Further, the music data acquisition means acquires music data representing the pitch and value of each of the constituent sounds constituting the music, and the singing section specifying means collates the song data generated by the song data generation means with the music data. By doing this, the singing section which is a section corresponding to each component sound is specified.

  Then, when the duration value deriving means derives a voice duration value that becomes larger as the time during which the user has uttered with a voice volume equal to or higher than the set value for each singing section specified by the singing section specifying means, The first sound range determination means totals the sound duration values that have been recorded for each pitch of the constituent sounds corresponding to each singing section, and the total voice duration value (hereinafter referred to as the total duration value) A range from the lowest pitch to the highest pitch that is equal to or higher than a specified value is determined as a first singing range that is a range in which the user can sing.

  In other words, in the range specifying system of the present invention, the range of the pitch that the user can stably sing with a voice volume equal to or higher than a predetermined level (that is, the voice duration value is equal to or higher than a specified value) That is, it is determined as the first singing range.

  And, in the state that the user uttered utterance, it is difficult to stabilize the utterance with a voice volume above a certain level, so according to the range identification system of the present invention, the user can sing without undue effort Can be identified.

  Therefore, in the karaoke system having the range specifying system of the present invention, if the music whose pitch changes mainly in the specified singing range is suggested to the user as a recommended song, the user can more enjoy karaoke. it can.

By the way, normally, a user who is good at singing can sing with a sound volume above a certain level in a wider sound range than a user who is poor at singing.
That is, in the range specifying system of the present invention, when the total duration value is normalized, the distribution of the total duration value is the summation result of the pitches of the constituent sounds corresponding to each singing section, as the user who is better at singing ( That is, the total accumulated value is close to the maximum value. On the other hand, in the distribution of the total duration value, the difference between the maximum value and the total result of the pitches that do not correspond to the maximum value (that is, the total duration value) increases as the user is poorer in singing.

  Therefore, as described in claim 2, if the specified value used in the sound range specifying means is a ratio defined in advance with respect to the maximum value of the normalized totaled continuous value, the user who is good at singing is wider. The range can be determined as the singing range. As a result, according to the sound range specifying system of the present invention, the user's singing sound range can be appropriately determined regardless of the user's level (skill).

In general, the longer the tone value of a constituent sound (that is, the time length of each constituent sound), the more difficult it is to maintain a voice volume above a certain level with respect to the total tone value of the constituent sounds.
For this reason, the first sound range determination means in the sound range identification system of the present invention attaches a greater weight to the sound duration value as the note value of the constituent sound corresponding to the singing section is longer, as described in claim 3. It is desirable to be configured to aggregate.

According to the sound range specifying system of the present invention configured as described above, it is possible to more reliably specify a sound range (singing sound range) that can be uttered by the user without overdoing it.
Then, the duration value deriving means in the sound range specifying system of the present invention, as claimed in claim 4, is based on the signal level of the audio signal acquired by the audio volume value acquiring means than the singing section. In addition to deriving a voice volume value representing the user's voice volume for each short defined unit section, the integrated value deriving means derives a value obtained by integrating the length of the unit section whose voice volume value is equal to or greater than a set value as a voice duration value. It may be configured to.

  According to the sound range specifying system configured as described above, since the sound duration value is derived according to the signal level of the sound signal, the sound range can be specified without increasing the number of signals input to the sound range specifying system.

  The voice volume value here may be a value obtained by averaging the signal level (that is, the absolute value of the amplitude) of the audio signal every unit time, or may be a value obtained by squaring the signal level (amplitude) of the audio signal. Alternatively, a value obtained by averaging the signal level (amplitude) of the audio signal may be used.

  Further, in the range specifying system of the present invention, as described in claim 5, the stability deriving means derives the singing stability indicating the degree of stability of the vibrato in each singing section, and the derived singing stability Are summed up for each pitch of the constituent sounds corresponding to each singing section, and the second range determining means has the summed singing stability from the lowest pitch to the highest pitch that is equal to or higher than a predetermined value. You may be comprised so that the range to high may be determined as a 2nd song sound range which is a sound range which a user can sing.

  In the range specifying system of the present invention configured as described above, in addition to the first singing range that can be stably sung with a voice volume equal to or higher than the set value, the range that can be stably sung by the utterance method by vibrato is used as the second singing range. It is determined as a range.

That is, according to the range specifying system of the present invention, it is possible to determine the singing range based on two different parameters such as the voice duration value and the singing stability.
For this reason, when the range identification system of the present invention is applied to a karaoke device to propose a recommended song based on the singing range to the user or to reflect the singing range in the scoring result, the two specified singing The range can be used properly according to the music. As a result, a more appropriate recommended song can be proposed or a more appropriate scoring can be made, and the user can be more entertained with karaoke.

  In this way, in the range specifying system of the present invention configured to determine the second singing range in addition to the first singing range, the user is based on these two ranges (first and second singing ranges). It is desirable to specify a sound range that can be sung more naturally (that is, a singing sound range).

  That is, the range specifying system of the present invention is configured such that the singing range specifying unit specifies at least one of the first singing range and the second singing range as the singing range as described in claim 6. It may be constituted, and it may be constituted so that the range which satisfies both the 1st singing range and the 2nd singing range may be specified as a singing range.

  In the former case, since a wide sound range is specified as the singing sound range, the user can be satisfied, and in the latter case, a sound range in which the user can surely sing can be specified as the singing sound range.

The present invention may be implemented as a program for causing a computer to execute each procedure, as described in claim 7.
However, the program of the present invention generates singing data representing the transition of the pitch of the singing by analyzing the frequency of the acquired audio signal and the audio signal acquisition procedure for acquiring the audio signal when the user sings the song. The song data generation procedure, the song data acquisition procedure for acquiring the song data representing the pitch and value of each of the constituent sounds constituting the song, and the song data generated in the song data generation procedure are collated with the song data. Thus, the time during which the user was uttering with a voice volume equal to or higher than a preset value for each specified singing section, and a singing section specifying procedure for specifying a singing section that is a section corresponding to each component sound The duration value derivation procedure for deriving the voice duration value, which becomes larger as the length increases, and the derived voice duration value are tabulated for each pitch of the constituent sounds corresponding to each singing section. The first sound range determination procedure for determining the range from the lowest pitch to the highest pitch whose total sound duration value is equal to or greater than a predetermined value as a first singing range that can be sung by the user. Need to contain.

By causing the computer to execute such a program of the present invention, it is possible to obtain the same effect as the sound range specifying system according to claim 1.
Further, according to the program of the present invention, for example, the program can be recorded on a computer-readable recording medium such as a DVD-ROM, a CD-ROM, and a hard disk, and loaded into the computer as necessary, and started. If necessary, it can be used by downloading (acquiring) to a computer via a communication line and starting it.

Embodiments of the present invention will be described below with reference to the drawings.
FIG. 1 is a block diagram showing a schematic configuration of a karaoke system to which the present invention is applied.
<Configuration of the entire karaoke system>
The karaoke system 1 is for a user to sing according to the performance of a song processed in advance for karaoke (hereinafter referred to as karaoke song).

  As shown in FIG. 1, the karaoke system 1 includes a karaoke apparatus 20 that reproduces karaoke music specified by a user, and a server 30 that distributes music data that is data necessary for reproducing karaoke music to the karaoke apparatus 20. The karaoke apparatus 20 and the server 30 are connected via a network (for example, a dedicated line, a WAN, or the like). That is, the karaoke system 1 is configured as a so-called communication karaoke system.

  In the present embodiment (FIG. 1), the karaoke system 1 is shown in which one karaoke device 20 and one server 30 are connected, but the karaoke system 1 is not limited to this, and a plurality of karaoke systems 1 The karaoke device 20 and one server 30 may be connected, or the karaoke device 20 and the plurality of servers 30 may be connected.

  Among these, the server 30 is a known micro having at least a processing program (hereinafter referred to as a karaoke processing program) necessary for performance of karaoke music and a storage device (not shown) for storing music data, and ROM, RAM, and CPU. This is a well-known karaoke service server device comprising an information processing device mainly composed of a computer (not shown).

  That is, the server 30 includes information on a user who uses the karaoke system 1 (more precisely, the karaoke device 20) (hereinafter referred to as user information), a karaoke scoring result transmitted from the karaoke device 20, and a user's use. The log data and the karaoke processing program are transmitted in response to a request from the karaoke apparatus 20 while managing the history and the like in a unified manner.

  By the way, the music data is prepared in advance for each karaoke music, and is, for example, well-known karaoke playback data described in the MIDI (Musical Instrument Digital Interface) format.

  Therefore, the music data is composed of music information that is data related to karaoke music, a guide melody that is data related to the melody that the user should sing, and lyrics information that is data related to the lyrics of the karaoke music.

The music information includes music number data for specifying the karaoke music, music name data indicating the music name, and time data indicating the performance time of the karaoke music.
The guide melody is composed of pitches and tone values of constituent sounds that form the melody of karaoke music. Specifically, in the guide melody of the present embodiment, the tone output start time and tone output end time of each component sound are shown together with the pitch of each component sound. However, the tone output start time here is the time from the start of the performance of the karaoke music that starts outputting the constituent sounds, and the music output end time is from the start of the performance of the karaoke music that ends the output of the constituent sounds. Is the time.
<About karaoke equipment>
Next, the configuration of the karaoke apparatus 20 will be described.

  The karaoke apparatus 20 includes a communication unit 22 for executing data communication with the server 30, a storage unit 21 for storing a karaoke processing program and music data acquired from the server 30 via the communication unit 22, and various types. A display unit 23 for displaying an image and an operation reception unit 24 for receiving an instruction from the user are provided. Further, the karaoke apparatus 20 includes a microphone 26 for inputting sound, a speaker 27 for outputting sound, a sound input / output unit 25 for controlling sound input / output through the microphone 26 and the speaker 27, and karaoke. A control unit 28 that controls each of the units 21, 22, 23, 24, and 25 constituting the apparatus 20 is provided.

  Among these, the communication unit 22 is a communication interface for connecting the karaoke apparatus 20 to a network (for example, a dedicated line or WAN) to communicate with the outside. The communication unit 22 communicates with the server 30 according to a control command from the control unit 28. While outputting various data, various data and processing programs are acquired from the server 30.

  The display unit 23 is a display device configured from, for example, a liquid crystal display, and the operation receiving unit 24 is input via, for example, an input device configured from a plurality of key switches or the like, or a known remote controller. It comprises a receiving device that accepts instructions.

  The voice input / output unit 25 is configured as an AD converter that converts voice (analog signal) input via the microphone 26 into a digital signal and inputs the digital signal to the control unit 28. At the same time, the audio input / output unit 25 is configured to control the output of audio from the speaker 27. In the following, the sound that is input via the microphone 26 and converted into a digital signal is referred to as sound data.

  Furthermore, the storage unit 21 is a storage device (for example, a hard disk drive) configured to retain stored contents even when the power is turned off and to be able to read and write the stored contents. The storage unit 21 includes a program storage area for storing a karaoke processing program, a music data storage area for storing music data, and a specifying data storage area for storing audio data. The data storage area for specification is a voice data storage area for storing voice data for a predetermined number of karaoke songs (for example, for five songs), and a predetermined number of users (for example, for five people). ) Is prepared for the minutes.

  Next, the control unit 28 stores the ROM 28a that stores programs and data that need to retain stored contents even when the power is turned off, the RAM 28b that temporarily stores programs and data, and the ROM 28a and RAM 28b. It is mainly configured by a known microcomputer having at least a CPU 28c that executes control and various operations for the respective units 21, 22, 23, 24, and 25 constituting the karaoke apparatus 20 according to programs and data.

  The karaoke processing program is read from the storage unit 21 into the RAM 28b, and the CPU 28c executes each process according to the karaoke processing program stored in the RAM 28b.

  Among these karaoke processing programs, the CPU 28c performs a well-known karaoke performance process for playing (reproducing) a specified karaoke piece according to an instruction input via the operation receiving unit 24 and displaying lyrics on the display unit 23. A karaoke performance processing program is included. In the karaoke processing program, the pitch and tempo of the singing extracted from the voice input through the microphone 26 are checked against a scoring standard (that is, a guide melody), and the degree of conformity is scored. A scoring program for the CPU 28c to execute the well-known scoring process as a result is included.

  Further, in the karaoke processing program, a user information processing program for the CPU 28c to execute user information processing for receiving user information (for example, name, sex, identification number (ID), age) via the operation receiving unit 24. It is included. In the karaoke processing program, data generated when the user sings karaoke music (for example, voice data) and a history when the karaoke apparatus 20 is used are stored in the storage unit 21 or the server 30. An accumulation process program for the CPU 28c to execute the accumulation process is included.

  Further, in the karaoke processing program, a singing range specifying process for specifying a range in which the user can sing (hereinafter referred to as a singing range) based on the audio data stored in the determination data storage area of the storage unit 21. Is included in the sound range identification processing program for the CPU 28c to execute. That is, when the CPU 28c executes the singing sound range specifying process, the karaoke apparatus 20 functions as the sound range specifying system of the present invention.

Next, the operation (operation) of the karaoke system 1 will be described.
When the karaoke system 1 is used, the karaoke device 20 executes a user information processing program, receives user information for the number of users who use the karaoke device 20, and sets each received user information as voice data. Associate with storage area.

  Furthermore, in the karaoke apparatus 20, by executing a karaoke performance processing program, the karaoke music designated by the user is played and the lyrics of the karaoke music are displayed on the display unit 23. However, the control unit 28 acquires user information via the operation reception unit 24 before each karaoke piece is played.

  Further, when the performance of the karaoke music is started, the user sings according to the karaoke music played at that time (hereinafter referred to as the corresponding karaoke music). Then, when the performance of the corresponding karaoke song is completed, in the control unit 28, the CPU 28c executes the accumulation processing program and associates the voice data generated from the user's singing voice with the song number data of the corresponding karaoke song. (Hereinafter, data in which audio data and song number data are associated with each other is also referred to as correspondence data) and is stored in the audio data storage area. However, the audio data storage area in which the correspondence data is stored is associated with the user information of the user who sang the corresponding karaoke song. Thereby, audio | voice data, music number data, and user information are matched.

Thereafter, the karaoke apparatus 20 repeats a series of cycles from the execution of the karaoke performance processing program to storing the voice data in the voice data storage area until the user finishes specifying the karaoke music.
<Singing range identification processing>
Next, the singing range specifying process executed by the CPU 28c will be described.

Here, FIG. 2 is a flowchart showing a processing procedure of the singing range identification processing.
This singing sound range identification process is started when a prescribed number of audio data is stored in the audio data storage area of the storage unit 21, that is, when a single user sings a prescribed number of karaoke songs. .

  As shown in FIG. 2, when the singing sound range specifying process is started, first, at S110, one piece of voice data is acquired from all the voice data stored in the voice data storage area of the storage unit 21. That is, as shown in FIG. 5A, an audio signal whose signal level changes along with the performance of karaoke music (ie, the progress of performance time) is acquired as audio data.

  Subsequently, in S120, the audio data acquired in S110 is subjected to frequency analysis (in this embodiment, FFT (Fast Fourier Transform)) to derive a frequency spectrum of the audio data.

Specifically, in S120 in the present embodiment, the audio data for one karaoke piece is extracted for each reference time defined in advance as a continuous time (for example, several tens of ms, ie, a predetermined number of samplings). (Hereinafter, the extracted voice data is referred to as target data). Then, frequency analysis (that is, FFT) is performed for each target data. Thereby, as a result of the frequency analysis, the frequency spectrum (that is, the distribution of frequency components) in each target data is generated by the number of all target data.
Furthermore, in S130, based on the result (frequency spectrum) of the frequency analysis in S120, the time transition of the fundamental frequency f0 when the user sings the corresponding karaoke music is represented, and the fundamental frequency transition corresponding to the song data of the present invention. f0v (t) is derived.

  Specifically, in S130 in the present embodiment, a harmonic structure model represented by a comb shape is prepared in advance, and the target data is obtained using a known technique for collating the harmonic structure model with each frequency spectrum. The fundamental frequency f0 is detected from the frequency components for. Then, the detected fundamental frequency f0 is collected as a fundamental frequency transition f0v (t) according to the time transition of each target data, that is, the progress of the performance time of the corresponding karaoke piece.

Thereby, the fundamental frequency transition f0v (t) represents the time change of the fundamental frequency f0 when the corresponding karaoke piece is sung, as shown in FIG. 5B.
In subsequent S140, a guide melody of the corresponding karaoke song is acquired from the storage unit 21 in accordance with the song number data associated with the voice data acquired in S110.

  In S150, based on the guide melody acquired in S140 and the fundamental frequency transition f0v (t) derived in S130, a time delay amount tl (k) representing the utterance delay time for each component sound is calculated. To do.

  Here, a method of calculating the time delay amount tl (k) in the present embodiment will be described in detail. However, in this embodiment, k represents the order of performance of the corresponding component sounds, and represents that the kth performance is performed from the start of performance of the corresponding karaoke piece. Accordingly, k is a natural number from 1 to the total number of constituent sounds.

  First, the time change of the pitch of all the constituent sounds represented by the guide melody (that is, the melody by the guide melody) is set as the guide melody pitch, and the timing at which the pitch is switched to the kth constituent sound (more precisely, the period ) Is the pitch switching timing.

  Then, as shown in FIG. 5C, by comparing the fundamental frequency transition f0v (t) with the guide melody pitch, the pitch switching timing in the fundamental frequency transition f0v (t) (hereinafter, switching at the time of singing) (Referred to as timing). As a method for collating the fundamental frequency transition f0v (t) with the guide melody pitch, a known method described in JP-A-2005-107330 may be used.

Further, the difference between the detected singing time switching timing and the pitch switching timing in the guide melody pitch is calculated as a time delay amount tl (k).
As described above, a series of flow from collation of the guide melody pitch and the fundamental frequency transition f0v (t) to calculation of the time delay amount tl (k) is performed according to the time progress of the karaoke music (that is, k is 1). Until k reaches a maximum, repeating k). Thereby, the time delay amount tl (k) for all the constituent sounds is obtained.

  That is, in S150, when singing the corresponding karaoke song, the timing at which the singing is switched is specified as the timing at which each constituent sound is actually started, and the singing from the pitch switching timing in the guide melody pitch. The delay in time switching timing is being sought.

  In subsequent S160, a period in which the user actually sang each constituent sound is defined as a corresponding period T (k), and a voice duration indicating a time during which the voice is uttered with a volume greater than or equal to a set value in each corresponding period T (k). The time tpc (k) is derived and the duration for deriving the voice duration value distribution, which is the result of totaling the voice duration tpc (k) for each pitch f corresponding to the constituent sounds in each corresponding period T (k) Execute value distribution derivation processing.

  In addition, as shown in FIG. 7, the sound duration value distribution in the present embodiment is a summed voice duration tpc (k) (hereinafter, a total duration tp) with the pitch f corresponding to the constituent sounds as a horizontal axis. (F)) is a map with the vertical axis.

  Further, in S170, the singing stability level vsi (k) representing the degree of stability of the utterance in the vibrato in the corresponding period T (k) is derived, and the pitch f corresponding to the constituent sound in each corresponding period T (k). A stability distribution derivation process for deriving a singing stability distribution, which is the result of totaling the singing stability vsi (k), is performed.

  Note that the singing stability distribution in the present embodiment, as shown in FIG. 8, uses the pitch f corresponding to the constituent sounds as the horizontal axis, and the total singing stability vsi (k) (hereinafter, total stability mvsi). (K)) is a map with the vertical axis.

  Further, in S180, it is determined whether or not the processing from S110 to S170 (herein referred to as “regulation processing”) has been executed for all audio data stored in the audio data storage area of the storage unit 21. .

  If the result of determination is that the prescribed process has not been executed for all audio data, the process returns to S110, and in S110, from all the audio data stored in the audio data storage area of the storage unit 21. The voice data for which the regulation process has not been executed is acquired, and the process proceeds to S120.

On the other hand, if the result of determination in S180 is that the defining process has been executed for all audio data, the process proceeds to S190.
In subsequent S190, the first singing range is specified in accordance with the voice duration value distribution derived in the duration value distribution derivation process (S160).

  Specifically, in the present embodiment, a range from the lowest pitch to the highest pitch where the total duration mtpc (k) is equal to or greater than a predefined value on the speech duration distribution as shown in FIG. That is, the frequency band from the lowest fundamental frequency to the highest fundamental frequency in which the total duration mtpc (f) is equal to or greater than a specified value is set as the first singing sound range.

  Note that the specified value in this embodiment is a ratio A (for example, half of the maximum value, 1/3, etc.) defined in advance with respect to the maximum value M of the total duration mtpc (k) on the sound duration value distribution. (Specified value = M × A).

In S200, the second singing sound range is specified according to the singing stability distribution derived in the stability distribution deriving process (S170).
Specifically, in the present embodiment, the total stability mvsi (f) on the singing stability distribution as shown in FIG. 8 is a range from the lowest pitch to the highest pitch that is equal to or higher than a predetermined value, That is, the frequency band from the lowest fundamental frequency to the highest fundamental frequency where the total stability mvsi (f) is a specific value or more is set as the second singing sound range.

  The specific value in the present embodiment is a ratio A (for example, half of the maximum value, 1/3, etc.) defined in advance with respect to the maximum value M of the total stability mvsi (f) on the singing stability distribution. (Specified value = M × A).

  In S210, based on the first singing range specified in S190 and the second singing range specified in S200, a range (frequency band) in which the first singing range and the second singing range overlap. , It is specified as a range where the user can reliably sing (hereinafter referred to as a determined singing range).

  In S220, the first singing sound range specified in S190, the second singing sound range specified in S200, and the entire singing sound range specified in S210 are notified to the user via the display unit 23 and the speaker 27. Then, a specific range use process for storing the specified entire singing range in the storage unit 21 and the server 30 is executed. Furthermore, in the specific range use processing of the present embodiment, the karaoke music in which both the lowest pitch and the highest pitch in all the constituent sounds are within the determined singing range is detected from all the karaoke music, and the detected karaoke Suggest a song to the user as a recommended song.

Then, the present singing range specifying process is terminated.
<Sustained value distribution derivation process>
Next, the continuous value distribution derivation process will be described.

Here, FIG. 3 is a flowchart showing a processing procedure of the continuous value distribution derivation processing.
As shown in FIG. 3, when the continuous value distribution derivation process is started in S160 of the singing range specification process, first, in S310, the voice data in each corresponding period T (k) is converted into the singing voice section vo1 ( Extract as t).

Specifically, in the present embodiment, first, each corresponding period T (k) is derived based on the singing time switching timing for each component sound.
Note that the corresponding period T (k) is the singing time switching timing for the k-th component sound (ie, the musical sound output start time st (k) + time delay amount tl (k)) of the corresponding period T (k). Start timing. Furthermore, the time length tt (k) of the corresponding period T (k) is obtained as a difference between the singing time switching timing for the kth constituent sound and the singing time switching timing for the k + 1th constituent sound, and is 0 [s]. To the k-th component sound value (ie, the time during which the component sound is output) len (k) [s]. That is, for the kth component sound, if the user can sing for the entire period, the time length tt (k) becomes the note value len (k) [s] of the component sound, and if the user cannot sing for the entire period, the time length tt (k) is 0 [s].

  And the audio | voice data in each derived | led-out corresponding period T (k) are extracted according to the following (1) Formula as singing voice area vo1 (t). However, st (k) + tl (k) + tt (k) in the equation (1) corresponds to the corresponding period T (k).

As shown in FIG. 6 (A), the extracted singing voice section vo1 (t) takes into account the delay in the utterance time with respect to the constituent sounds, and the timing at which the user actually sang each constituent sound. And the audio data corresponding to the time.

  Subsequently, in S320, based on each of the singing voice sections vo1 (t) extracted in S310, the average value of voice power for each unit time (hereinafter referred to as a time window) shorter than the corresponding corresponding period T (k) ( An example of the voice volume value of the present invention, hereinafter referred to as average voice power) Pw (m) is derived.

  Specifically, in S320 of the present embodiment, the average voice power Pw (m) in each time window is derived according to the following equation (2).

However, in the expression (2), m is a time window index number (that is, a code for identifying the number of the corresponding time window from the start of the corresponding period T (k), hereinafter the time window number. Tw is a unit time defined as a time window (in this embodiment, it is several tens of ms).

  That is, in S320, first, as shown in FIG. 5B, a power value obtained by squaring each sampling value included in the singing voice section vo1 (t) is calculated. Next, each singing voice section vo1 (t) is divided for each time window, and the power value is integrated for each time window (that is, the sum of all power values in each time window is derived). Then, by dividing each integrated power value by a unit time TW defined as a time window, an average voice power Pw (m) is derived.

  Subsequently, in S330, based on each of the average voice powers Pw (m) derived in S320, the voice duration tpc (k) for each corresponding period T (k) (that is, corresponding to the voice duration value of the present invention). Is derived.

  Specifically, in S330 of the present embodiment, first, among the average voice power Pw (m) in each time window, the maximum voice power Pwmax whose value is equal to or greater than a prescribed power that is prescribed in advance and is the maximum. The time window number corresponding to the maximum sound power Pwmax is detected (hereinafter, the time window number is denoted by the symbol m0).

  Next, in the time progress of each time window, it corresponds to the average voice power Pw (m) after the time window number m0 and the value of the average voice power Pw (m) attenuated to half of the maximum voice power Pwmax. Time window number (hereinafter, the time window number is given a code mp, that is, the time window number mp is equal to or greater than the time window number m0).

  Furthermore, based on the specified time window number mp, the voice duration tpc (k) is derived by the following equation (3).

That is, in S330, as shown in FIG. 5C, from the start time (that is, start timing) of each corresponding period T (k), the value of the average voice power Pw (m) is half of the maximum voice power Pwmax. The time required for attenuation is derived as the voice duration tpc (k).

Further, in S340, a voice duration value distribution is generated based on each voice duration tpc (k) derived in S330.
Specifically, in the present embodiment, each of the voice durations tpc (k) is tabulated for each pitch f of the constituent sounds corresponding to the corresponding period T (k) using the following equation (4), and the tabulation duration is continued. The time tp (f) is derived.

However, in the equation (4), f is the pitch of the constituent sound, and k ′ is a set of identification numbers (corresponding to the k-th and k + 1-th mentioned above) representing the constituent sound equal to the pitch f, K0. Is the total number of constituent sounds having the same pitch f.

  That is, the total duration tp (f) is such that the total speech duration tpc (k) for the constituent sound equal to the pitch f becomes a larger value as the sound value len (k) of the corresponding constituent sound is longer. Weighted and aggregated, and further divided by the total number K0 of constituent sounds equal to the pitch f, that is, normalized.

  The total duration tp (f) is derived with respect to the whole pitch f appearing in the corresponding karaoke music piece. Then, the distribution of the derived total duration tp (f) with respect to the pitch f is derived as a speech duration value distribution.

  However, in the present embodiment, when the number of times of proceeding to S340 is the second time or later, the total duration tp (f) derived in that cycle (that is, current S340) is set to the previous cycle (that is, that is, Integration is performed on the calculated total duration tp (f) (before the previous S340). That is, the total duration tp (f) derived from each piece of voice data is integrated in the voice duration value distribution in this embodiment.

After that, the process returns to S170 of the singing range identification process.
<Stability distribution derivation process>
Next, the stability distribution derivation process will be described.

Here, FIG. 4 is a flowchart showing a processing procedure of the stability distribution derivation processing.
As shown in FIG. 4, when the stability distribution deriving process is started in S170 of the singing range specifying process, first, in S410, the fundamental frequency transition f0v (t) in each corresponding period T (k) is obtained. Extracted as a singing section f0vn (t).

  In S420, the singing section f0vn (t) extracted in S410 (that is, time transition of the fundamental frequency) is subjected to frequency analysis (in this embodiment, FFT), and a vibrato spectrum representing the result is derived.

  That is, the vibrato spectrum derived in S420 is the vibrato frequency component when the user sings each corresponding period T (k) corresponding to the singing section f0vn (t) (hereinafter, each of the frequency components is vibrato vibration). A distribution of several g).

  In S430, based on the vibrato spectrum derived in S420 (that is, the result of frequency analysis), the degree of utterance stability by vibrato in the corresponding period T (k) corresponding to the singing section f0vn (t), that is, Then, singing stability vsi (k) is derived.

  Specifically, in the present embodiment, the frequency peak in the vibrato spectrum is detected, and the frequency peak at which the amplitude value P0 (g) is the maximum value P0m is detected from the detected frequency peaks. Detect as.

  Then, a value obtained by multiplying the amplitude value P0 (g) (that is, the maximum value P0m) of the maximum frequency peak by 1/2 is an effective value, and the frequency component of the maximum frequency peak (hereinafter referred to as peak vibrato frequency g0). ) And the vibrato frequency band (that is, the width) in which the amplitude value P0 (g) is an effective value is obtained as the effective band gw.

  Further, by dividing the peak vibrato frequency g0 by the effective band gw, the sharpness Q (Q = gm / gw) is obtained as the singing stability vsi (k) for the constituent sound corresponding to the corresponding period T (k). To derive.

  If the song is sung by a stable (ideal) vibrato, there is only one frequency peak detected on the vibrato spectrum, and the effective band gw is small, that is, the maximum frequency peak is sharp. Become. For this reason, if the song is stably sung by vibrato, the singing stability vsi (k) becomes a large value.

In S440, the total stability mvsi (f) is derived based on each of the singing stability vsi (k) derived in S430, and a singing stability distribution is generated.
Specifically, the total stability mvsi (f) in the present embodiment is the total singing stability vsi (k) for the constituent sound equal to the pitch f, and the longer the note value len (k) of the corresponding constituent sound is, the longer it is. Weighted so as to be a large value, tabulated and further divided by the total number K0 of constituent sounds equal to the pitch f, that is, normalized.

  The total stability mvsi (f) is derived for the whole pitch f appearing in the corresponding karaoke music piece. Then, the singing stability distribution is derived by associating each derived total stability mvsi (f) with the corresponding pitch f.

  However, in the present embodiment, when the number of times of advance to S440 is the second or later, the total stability mvsi (f) derived in the cycle is replaced with the total stability mvsi (f) derived in the previous cycle. Is accumulated. That is, the total stability mvsi (f) derived from each audio data is integrated in the singing stability distribution in the present embodiment.

Then, the process proceeds to S180 of the singing range identification process.
That is, in the singing range identification processing of the present embodiment, the voice duration time tpc (k) representing the time during which the voice was continuously spoken with a voice volume equal to or greater than the set value based on the voice data in each corresponding period T (k). To derive. Furthermore, the total duration tp (f) obtained by summing up the voice duration tpc (k) for each pitch f corresponding to the constituent sounds in each corresponding period T (k) is derived.

From this, the total duration tp (f) increases if the pitch f can be continuously uttered with a volume greater than or equal to the set value, and the sound cannot be continuously uttered with the volume greater than or equal to the set value. If it is high f, the value becomes small.
[Effect of the embodiment]
As described above, in the singing sound range specifying process of the present embodiment, the range of the pitch that can be stably sung by the user with a voice volume equal to or higher than a preset setting value is determined as the first singing sound range.

  And, in a state where the user utters utterance, it is difficult to stabilize the utterance with a volume higher than a certain level, so according to the karaoke system 1 of the present invention, the user can sing without undue effort. Can be identified.

  Further, in the singing range identification processing of the present embodiment, the total duration tp (f) is derived as a normalized value. That is, as the user who is good at singing, the total duration tp (f) for each pitch of the constituent sounds becomes a value close to the maximum value, and as the user who is good at singing, the maximum value and the pitch not corresponding to the maximum value are obtained. The difference in value from the total duration tp (f) for each increases.

  Further, in the singing range identification processing of the present embodiment, weighting is performed such that the longer the note value len (k) of the component sound in the corresponding corresponding period T (k), the larger the value, and the total duration tp (f ). For this reason, in the singing sound range specifying process of the present embodiment, the longer the tone value of the constituent sound (that is, the length of time for outputting the constituent sound), the longer the utterance is sustained with a voice volume equal to or higher than the set value for the constituent sound. The total duration tp (f) is derived in consideration of the fact that it becomes difficult.

  As a result, according to the singing range specification processing of the present embodiment, it is possible to appropriately determine the user's singing range regardless of the user's singing level (skill), in other words, the user can speak without overdoing it. A sound range (that is, a singing sound range) can be specified more reliably.

  Therefore, in the karaoke system 1 of the present embodiment, if a song whose pitch changes mainly within the specified singing sound range is proposed to the user as a recommended song, the user can be more enjoyed with karaoke.

  In the singing range specification processing of the present embodiment, the second singing range is specified based on the singing stability vsi (k) in each corresponding period T (k), the second singing range and the first singing range. Is identified as the determined singing range.

For this reason, according to the singing sound range specification process of this embodiment, the sound range in which the user can sing more reliably can be specified as the determined singing sound range.
[Other Embodiments]
As mentioned above, although embodiment of this invention was described, this invention is not limited to the said embodiment, In the range which does not deviate from the summary of this invention, it is possible to implement in various aspects.

  For example, in the above-described embodiment, the activation timing of the singing sound range specifying process is set as a case where one user sings a predetermined number of karaoke songs, but the starting timing of the singing sound range specifying process is not limited to the above embodiment. Instead, it may be a case where an activation command is received from the user via the operation reception unit 24, or may be a time when singing is completed for one karaoke piece. In other words, the activation timing of the singing sound range identification process may be any timing as long as the audio data to be processed exists.

  In S320 in the continuous value distribution derivation process of the above embodiment, the average voice power Pw (m) is derived by averaging a value obtained by squaring the sampling value (signal level of the voice signal) for each time window. However, the method of deriving the average voice power Pw (m) is not limited to this, and may be a value obtained by averaging the signal level (that is, the absolute value of the amplitude) of the voice signal within the time window, for example.

  Furthermore, in S330 in the sustained value distribution derivation process of the above embodiment, the time required from the start time of each corresponding period T (k) until the value of the average voice power Pw (m) attenuates to half of the maximum voice power Pwmax. Although the voice duration time tpc (k) is used as the actual duration time (hereinafter referred to as the actual duration time), the voice duration time tpc (k) is not limited to this. For example, the voice duration tpc (k) may be obtained by dividing the actual duration by the value of the corresponding constituent sound (that is, the time length of the constituent sound). That is, the ratio of the actual duration to the tone value of the constituent sounds may be the voice duration tpc (k).

  By the way, in S340 in the continuous value distribution derivation process of the above embodiment, as a normalization method for deriving the total duration tp (f), the result of totaling the voice duration tpc (k) is used as the total number of constituent sounds. However, the normalization method is not limited to this. For example, the normalization method may be divided by the total performance time of the constituent sounds for the same pitch, or may be other methods.

Furthermore, in S340, normalization may not be performed when deriving the total duration tp (f).
In S340 in the continuous value distribution derivation process of the above embodiment, when deriving the total duration tp (f), the longer the tone value of the constituent sounds, the greater the weight is given and the total voice duration tpc (k) is totaled. However, weighting may not be performed.

  In S340 in the continuous value distribution derivation process of the above embodiment, the total duration tp (f) is derived by totaling the voice durations tpc (k) for each pitch of the constituent sounds. The method for deriving the duration tp (f) is not limited to this, and may be derived based on the following equation (5), for example.

However, k ′ in the equation (5) represents a constituent sound equal to the pitch f and a value of the constituent sound (that is, a time length for outputting the sound) is equal to or longer than a preset set time. This is a set of sound identification numbers (corresponding to the k-th and k + 1-th mentioned above).

That is, the total duration tp (f) may be a total of the voice durations tpc (k) for each type of note (for example, a half note or a quarter note) corresponding to the constituent sounds.
In this case, it is possible to specify the tone value (time length) of the constituent sound that the user can continuously (stable) with a voice volume equal to or higher than the set value.

  In S210 of the singing sound range specifying process of the above embodiment, the sound range including both the first singing sound range and the second singing sound range is specified as the determined singing sound range. For example, a range including at least one of the first singing range and the second singing range may be specified as the determined singing range, or a range consisting only of the first singing range is specified as the determined singing range You may do it.

That is, in the latter case, the stability distribution derivation process in S160 and the determination of the second singing sound range in S200 in the singing sound range specifying process of the above embodiment may be omitted.
In the specific range use processing in the above embodiment, the specified singing range is notified to the user, or karaoke music based on the specified determined singing range is proposed to the user as a recommended song. The processing content executed in the processing is not limited to this. For example, it may be used for scoring so that the narrower the determined singing sound range is, the more severe the scoring is.

  Furthermore, in the singing range identification process of the above embodiment, the voice duration value distribution and the singing stability distribution are respectively set to the total duration tp (f) or the total stability mvsi (f) and the pitch corresponding to the constituent sounds. Although it was derived as an associated map, the voice duration value distribution or the singing stability distribution is not limited to the map, and the total duration tp (f) or the total stability mvsi (f) corresponds to the constituent sounds. Anything can be used as long as it is associated with the pitch to be played.

In the above embodiment, the singing range specifying process is executed by the control unit 28 of the karaoke apparatus 20, but the singing range specifying process may be executed by the microcomputer of the server 30. In this case, the audio data is preferably stored in the storage device of the server 30.
[Correspondence between the present invention and this embodiment]
By the way, the function obtained by executing S110 of the singing sound range specifying process corresponds to the audio signal acquiring means of the present invention, and the function obtained by executing S120 and S130 is the singing data generating means of the present invention. The function obtained by executing S140 corresponds to the music data acquisition unit of the present invention. Further, the function obtained by executing S150 of the singing range specifying process and S310 of the continuous value distribution deriving process corresponds to the singing section specifying means of the present invention, and executes S320 and S330 of the continuous value distribution deriving process. The function obtained by this corresponds to the sustained value deriving means of the present invention. Furthermore, the function obtained by executing S340 of the continuous value distribution deriving process and S190 of the singing range specifying process corresponds to the first range determining means of the present invention.

  Furthermore, the function obtained by executing S430 of the stability distribution deriving process corresponds to the stability deriving means of the present invention, and is obtained by executing S440 of the stability deriving process and S200 of the singing sound range specifying process. The function corresponds to the second singing range determination means of the present invention.

  The function obtained by executing S210 of the singing range specification process corresponds to the singing range specification unit of the present invention.

It is a block diagram which shows schematic structure of a karaoke system. It is a flowchart which shows the process sequence of a singing sound range specific process. It is a flowchart which shows the process sequence of a continuous value distribution derivation process. It is a flowchart which shows the process sequence of stability distribution derivation processing. It is the schematic diagram which showed typically the processing content of the song range identification process. It is the schematic diagram which showed typically the processing content of the continuous value distribution derivation processing. It is explanatory drawing for demonstrating the method of determining a 1st song sound range. It is explanatory drawing for demonstrating the method of determining a 2nd song sound range.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Karaoke system 20 ... Karaoke apparatus 21 ... Memory | storage part 22 ... Communication part 23 ... Display part 24 ... Operation reception part 25 ... Voice input / output part 26 ... Microphone 27 ... Speaker 28 ... Control part 28a ... ROM 28b ... RAM 28c ... CPU 30 ... Server

Claims (7)

An audio signal acquisition means for acquiring an audio signal at the time of singing a song by a user;
Singing data generating means for generating singing data representing the transition of the pitch of the singing by frequency analysis of the sound signal acquired by the sound signal acquiring means;
Music data acquisition means for acquiring music data representing the pitch and value of each of the constituent sounds constituting the music;
Singing data generated by the singing data generating means, by comparing with the music data, a singing section specifying means for specifying a singing section that is a section corresponding to each of the constituent sounds;
For each singing section specified by the singing section specifying means, a duration value derivation means for deriving a voice duration value that becomes a larger value as the time during which the user has spoken with a voice volume equal to or higher than a preset setting value is longer; ,
The voice duration value derived by the duration value derivation means is tabulated for each pitch of the constituent sounds corresponding to each singing section, and the tabulated voice duration value is set as a tabulation duration value, and the tabulation duration value is: First range determining means for determining a range from a minimum pitch to a maximum pitch that is equal to or higher than a predetermined value defined in advance as a first singing range that can be sung by the user. Range identification system. The first range determination means includes
Normalizing the aggregate duration value,
The range specifying system according to claim 1, wherein a predetermined ratio with respect to the normalized maximum value of the aggregated continuous value is set as the specified value. The first range determination means includes
The sound range identification system according to claim 1 or 2, wherein the sound duration value is tabulated with a greater weight as the sound value of the constituent sound corresponding to the singing section is longer. The sustained value deriving means includes
Based on the signal level of the audio signal acquired by the audio signal acquisition unit, a voice volume value deriving unit that derives a voice volume value representing the voice volume of the user for each unit section defined shorter than the singing section;
The integrated value deriving means for deriving, as the sound duration value, a value obtained by integrating the lengths of unit intervals in which the sound volume value derived by the sound volume value deriving means is greater than or equal to the set value. The sound range identification system according to any one of claims 1 to 3. Stability derivation means for deriving singing stability representing the degree of stability of vibrato in each singing section specified by the singing section specifying means;
The singing stability derived by the stability deriving means is aggregated for each pitch of the constituent sounds corresponding to each singing section, and the aggregated singing stability is a minimum that is equal to or higher than a predetermined value. 5. A second range determination unit that determines a range from a pitch to a maximum pitch as a second singing range that can be sung by the user. The range identification system described in. At least one of the first singing range determined by the first range determination unit and the second singing range determined by the second range determination unit is specified as a singing range that can be sung by the user. The tune range identification system according to claim 5, further comprising: On the computer,
An audio signal acquisition procedure for acquiring an audio signal when the user sings a song;
Singing data generation procedure for generating singing data representing the transition of the pitch of the singing by frequency analysis of the audio signal acquired in the audio signal acquisition procedure,
A music data acquisition procedure for acquiring music data representing the pitch and value of each of the constituent sounds constituting the music;
The singing section specifying procedure for specifying the singing section that is a section in the singing corresponding to each of the constituent sounds by comparing the singing data generated in the singing data generating procedure with the music data,
For each singing section specified in the singing section specifying procedure, a continuation value derivation procedure for deriving a voice continuation value that becomes a larger value as the time during which the user has spoken with a voice volume equal to or higher than a preset setting value is longer; ,
The voice duration value derived in the duration value derivation procedure is tabulated for each pitch of the constituent sounds corresponding to each singing section, and the tabulated voice duration value is set as a tabulation duration value, and the tabulation duration value is: And a first range determination procedure for determining a range from a minimum pitch to a maximum pitch that is equal to or higher than a predetermined value as a first singing range that can be sung by the user. Program to do.