JP2017111268A - Technique judgement device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a technique judgement device capable of judging technique of input sound.SOLUTION: A technique judgement device in an embodiment includes: an input sound acquisition part that acquires an input sound; a feature quantity detection section that time-serially detects a feature quantity of the input sound acquired by the input sound acquisition part; a flat portion detection section that detects a flat portion in a feature quantity based on the feature quantity acquired by the feature quantity detection part; and a technique determination part that judges the technique in the input sound based on the variation of the feature quantity in a predetermined period before and after a flat portion in the feature quantity.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a technique for determining an input sound technique.

  The karaoke apparatus has a function of analyzing and evaluating a singing voice. Various methods are used for singing evaluation. As one of the methods, for example, Patent Document 1 discloses a karaoke scoring device that detects and evaluates the screaming that is one of singing techniques.

JP 2004-102149 A

  However, in the technique disclosed in Patent Document 1, there are many false detections, and there is a problem that the singing technique cannot be accurately determined and evaluated.

  One of the objects of the present invention is to determine the technique of input sound.

  According to an embodiment of the present invention, an input sound acquisition unit that acquires an input sound, a feature amount detection unit that detects a feature amount of the input sound acquired by the input sound acquisition unit in time series, and the feature amount Based on the feature amount acquired by the detection unit, based on the flat portion detection unit that detects the flat portion of the feature amount, and the variation of the feature amount in a predetermined period before or after the flat portion of the feature amount And a technique determination unit that determines a technique of the input sound.

  The flat part detection unit may detect a period in which the time-series fluctuation of the feature amount is equal to or less than a predetermined fluctuation, and may detect the period as the flat part when the period is equal to or longer than a predetermined time.

  The feature amount may be a pitch or a volume.

  The technique determination apparatus may include a tempo estimation unit that estimates a tempo of the input sound based on the feature amount, and the predetermined time may be determined according to the tempo.

  When a plurality of flat portions are detected by the flat portion detection unit, the predetermined period may be a period between two flat portions adjacent to each other in time series.

  According to an embodiment of the present invention, a computer acquires an input sound, detects a feature amount of the input sound in time series, detects a flat portion of the feature amount based on the feature amount, and flattenes the feature amount. There is provided a program for executing the determination of the technique of the input sound based on the variation of the feature amount in a predetermined period before or after the part.

  According to one embodiment of the present invention, it is possible to determine the technique of the input sound.

It is a block diagram which shows the structure of the technique determination apparatus in one Embodiment of this invention. It is a block diagram which shows the structure of the technique determination function and evaluation function in one Embodiment of this invention. It is a figure for demonstrating the concept of the flat part detection of a volume in the flat part detection part of the technique determination function in one Embodiment of this invention. It is a figure for demonstrating the concept of the flat part detection of a pitch by the flat part detection part of the technique determination function in one Embodiment of this invention. It is a figure for demonstrating the concept of the screeching determination by the technique determination part of the technique determination function in one Embodiment of this invention. It is a figure for demonstrating the concept of the scoop down determination by the technique determination part of the technique determination function in one Embodiment of this invention. It is a figure for demonstrating the concept of the jump determination by the technique determination part of the technique determination function in one Embodiment of this invention. It is a figure for demonstrating the concept of the fall determination by the technique determination part of the technique determination function in one Embodiment of this invention. It is a figure for demonstrating the concept of the crescendo determination by the technique determination part of the technique determination function in one Embodiment of this invention. It is a figure for demonstrating the concept of the decrescendo determination by the technique determination part of the technique determination function in one Embodiment of this invention. It is a figure for demonstrating the concept of fist determination by the technique determination part of the technique determination function in one Embodiment of this invention. It is a figure for demonstrating the concept of forte piano determination by the technique determination part of the technique determination function in one Embodiment of this invention. It is a block diagram which shows the structure of the technique determination function and evaluation function in the modification of one Embodiment of this invention.

  Hereinafter, a technique determination apparatus according to an embodiment of the present invention will be described in detail with reference to the drawings. The following embodiments are examples of the embodiments of the present invention, and the present invention is not limited to these embodiments.

<First Embodiment>
A technique determination apparatus according to a first embodiment of the present invention will be described in detail with reference to the drawings. The technique determination apparatus which concerns on 1st Embodiment is an apparatus provided with the function which determines the song sound of the user who sings (it may be hereafter called a singer). This technique determination apparatus detects the pitch and volume of a singing sound in time series, and determines a specific technique based on a change in volume and a change in pitch.

[Hardware]
FIG. 1 is a block diagram showing a configuration of a technique determination apparatus 10 according to the first embodiment of the present invention. The technique determination apparatus 10 is a karaoke apparatus provided with a singing scoring function, for example. The technique determination apparatus 10 includes a control unit 11, a storage unit 13, an operation unit 15, a display unit 17, a communication unit 19, and a signal processing unit 21. A sound input unit (for example, a microphone) 23 and a sound output unit (for example, a speaker) 25 are connected to the signal processing unit 21. Each of these components is connected to each other via a bus.

  The control unit 11 includes an arithmetic processing circuit such as a CPU. The control unit 11 causes the CPU to execute the control program 13 a stored in the storage unit 13 and realizes various functions in the technique determination apparatus 10. The realized functions include a singing technique judging function. Further, the realized function may include a song evaluation function based on the technique determined by the technique determination.

  The storage unit 13 is a storage device such as a nonvolatile memory or a hard disk. The storage unit 13 stores a control program 13a for realizing the technique determination function. The control program 13a may include a song evaluation function. The control program 13a may be provided in a state stored in a computer-readable recording medium such as a magnetic recording medium, an optical recording medium, a magneto-optical recording medium, or a semiconductor memory. In this case, the technique determination apparatus 10 only needs to include a device that reads the recording medium. The control program 13a may be downloaded via a network such as the Internet.

  Moreover, the memory | storage part 13 memorize | stores the music data 13b and the song audio | voice data 13c as data regarding a song. The storage unit 13 may store evaluation reference data 13d. The music data 13b includes data related to a karaoke song, for example, guide melody data, accompaniment data, and lyrics data. The guide melody data is data indicating the melody of the song. Accompaniment data is data indicating the accompaniment of a song. The guide melody data and accompaniment data may be data expressed in the MIDI format. The lyric data is data for displaying the lyrics of the song and data indicating the timing for changing the color of the displayed lyrics telop. The singing voice data 13c is data corresponding to the singing voice input from the sound input unit 23 by the singer. In the present embodiment, the singing voice data 13c is stored in the storage unit 13 until the singing voice is determined by the technique determination function. The evaluation reference data 13d is information used as a reference for the evaluation of the singing voice by the evaluation function, and is preliminarily added to the music data indicating the singing song to be evaluated (the singing tune outputted when the singing voice is input). The associated reference sound data may be used.

  The operation unit 15 is a device such as an operation button, a keyboard, or a mouse provided on an operation panel and a remote controller, and outputs a signal corresponding to the input operation to the control unit 11. The display unit 17 is a display device such as a liquid crystal display or an organic EL display, and displays a screen based on control by the control unit 11. Note that a touch panel device in which the operation unit 15 and the display unit 17 are integrated may be used. Based on the control of the control unit 11, the communication unit 19 is connected to a communication line such as the Internet or a LAN and transmits / receives information to / from an external device such as a server. The function of the storage unit 13 may be realized by an external device that can communicate with the communication unit 19.

  The signal processing unit 21 includes a sound source that generates an audio signal from a MIDI format signal, an A / D converter, a D / A converter, and the like. The singing voice is converted into an electrical signal by the sound input unit 23 and input to the signal processing unit 21, and A / D converted by the signal processing unit 21 and output to the control unit 11. The singing voice is stored in the storage unit 13 as singing voice data 13c. The accompaniment data is read by the control unit 11, D / A converted by the signal processing unit 21, and output from the sound output unit 25 as an accompaniment of the song. At this time, a guide melody may also be output from the sound output unit 25.

[Technology judgment function]
A technique determination function realized by executing the control program 13a stored in the storage unit 13 by the control unit 11 of the technique determination apparatus 10 will be described. A part or all of the configuration for realizing the technique determination function described below may be realized by hardware.

  FIG. 2 is a block diagram showing a configuration of the technique determination function 100 according to the first embodiment of the present invention. Referring to FIG. 2, the technique determination function 100 includes an input sound acquisition unit 103, a feature amount detection unit 105, a flat part detection unit 111, and a technique determination unit 113.

  The input sound acquisition unit 103 acquires singing voice data (input sound) corresponding to the singing voice input from the sound input unit 23. In addition, although the input sound acquisition part 103 acquires song voice data directly from the signal processing part 21, you may make it acquire the song voice data once memorize | stored in the memory | storage part 13. FIG. The input sound acquisition unit 103 is not limited to acquiring singing voice data indicating the input sound to the sound input unit 23, and the singing voice data indicating the input sound to the external device is transmitted by the communication unit 19 via the network. You may get it. In the present embodiment, the input sound acquisition unit 103 sequentially outputs the singing voice data sequentially input during the reproduction of the music data to the feature amount detection unit 105.

  The feature amount detection unit 105 acquires singing voice data from the input sound acquisition unit 103. The feature amount detection unit 105 detects the feature amount of the singing sound including the volume and the pitch with respect to the acquired singing voice data. The feature amount detection unit 105 includes a volume detection unit 107 and a pitch detection unit 109.

  The volume detecting unit 107 detects the volume of the singing sound in time series from the singing voice data acquired by the input sound acquiring unit 103. That is, the volume detection unit 107 detects a temporal change in the volume of the singing sound based on the singing voice data. In the present embodiment, the volume detector 107 detects the volume based on the amplitude of the audio signal indicated by the singing audio data. The volume detection unit 107 outputs data (volume waveform) indicating the detected volume in time series to the flat part detection unit 111 and the technique determination unit 113 in time series.

  The pitch detection unit 109 detects the pitch of the singing sound in time series from the singing voice data acquired by the input sound acquisition unit 103. That is, the pitch detection unit 109 detects a zero cross when the waveform of the voice signal indicated by the singing voice data changes from negative to positive for each frame (data sample divided by a predetermined period), and sets the time interval of the zero cross. The pitch (frequency) of the singing sound is specified by measuring. At this time, a high-frequency component that becomes a noise component may be cut from the audio signal by a low-pass filter, or a DC component may be cut by a high-pass filter. Moreover, the pitch detection part 109 may identify a pitch from the spectrum obtained by giving FFT (Fast Fourier Transform) to song voice data. The pitch detection unit 109 outputs data (pitch waveform) indicating the pitches detected in this way in time series to the flat part detection unit 111 and the technique determination unit 113 in time series.

  The flat part detection unit 111 detects a flat part of the feature amount based on the time-series variation of the feature amount detected by the feature amount detection unit 105. The flat part detection unit 111 outputs data indicating the flat part of the detected feature amount to the technique determination unit 113.

  Specifically, the flat part detection unit 111 may detect a flat part in the data indicating the volume detected by the volume detection unit 107, that is, a period during which the volume is substantially constant as the flat part of the volume. . For example, the flat part detecting unit 111 has a volume fluctuation of a predetermined threshold ΔVth or less for each frame (data sample divided every predetermined time) with respect to the data indicating the volume detected by the volume detecting unit 107. It is determined whether or not. When frames whose fluctuation in volume is equal to or less than a predetermined threshold ΔVth are continuously detected for a predetermined number or more (for example, two or more frames), that is, a period in which fluctuation in volume is equal to or less than a predetermined threshold ΔVth When the predetermined time is longer than the predetermined time, the flat part detecting unit 111 may detect a frame whose volume fluctuation is equal to or less than a predetermined threshold ΔVth as a flat part of the volume.

FIG. 3 is a diagram for explaining an example of the concept of detecting the flat portion of the sound volume in the flat portion detecting unit 111. FIG. 3 is a volume waveform showing the volume of the singing sound in time series, with the vertical axis indicating volume (V) and the horizontal axis indicating time (T). In FIG. 3, frames f _{n−1 to} f _{n + 6} are shown. The length of the frame f is arbitrary. The start point detection unit 109 determines whether or not the change in volume in each of the frames f _{n−1 to} f _{n + 6} is equal to or less than a predetermined threshold value ΔVth. For example, volume fluctuations in the frames f _n−1 , f _n , f _{n + 1} , f _{n + 4} , f _{n + 5} , f _{n + 6} exceed a predetermined threshold ΔVth (ΔVn−1> ΔVth, ΔVn + 1> ΔVth, ΔVn + 4> ΔVth, ΔVn + 5> (ΔVth, ΔVn + 6> ΔVth), f _{n + 2} , f _{n + 3} , when the fluctuation in volume is equal to or less than a predetermined threshold ΔVth (ΔVn + 2 ≦ ΔVth, ΔVn + 3 ≦ ΔVth,), the flat part detecting unit 111 detects the frames f _{n + 2} and f _{n + 3} . You may detect as a flat part of a sound volume.

Further, the flat part detecting unit 111 may calculate an average value of the sound volume for each of a plurality of predetermined frames, and detect the flat part of the sound volume based on the fluctuation of the calculated average value of the sound volume. For example, the flat part 111 calculates the average value of the sound volume in the frame f _n−1 , the frame f _n and the frame f _{n + 1} , and then calculates the average value of the sound volume in the frame f _n , the frame f _{n + 1} and the frame f _{n + 2} . Then, the average value of the volume in the frame f _{n + 1} , the frame f _{n + 2} and the frame f _{n + 3} is calculated. As described above, the flat portion detection unit 111 calculates the average value of the volume while shifting the period for calculating the average value of the volume by a predetermined time (here, a time corresponding to one frame). The flat part detection unit 111 determines that the difference between the average values of the volume of a plurality of consecutive predetermined frames (for example, the frames f _{n−1 to} f _{n + 1} and the frames _{n to} f _{n + 2} ) is equal to or less than a predetermined threshold. A period corresponding to a plurality of predetermined frame periods (for example, frames f _{n−1 to} f _{n + 2} ) may be detected as a flat portion of the sound volume.

  For example, the flat part detection unit 111 calculates the absolute value of the slope of the data indicating the volume (volume waveform) for each frame of the data indicating the volume detected by the volume detection unit 107, and calculates the calculated slope. It may be determined whether or not the absolute value of is less than or equal to a predetermined value. When the number of frames in which the absolute value of the calculated inclination is equal to or less than a predetermined value is continuously detected for a predetermined number or more (for example, two or more frames), the flat part detection unit 111 detects the absolute value of the calculated inclination. You may detect the flame | frame from which a value is below a predetermined value as a flat part of a sound volume.

  Further, the flat part detecting unit 111 may detect a flat part in the data indicating the pitch detected by the pitch detecting unit 109, that is, a period in which the pitch is substantially constant, as the flat part of the pitch. For example, the flat part detection unit 111 calculates the average value of the pitch for each frame (data sample divided every predetermined time) with respect to the data indicating the pitch detected by the pitch detection unit 109, and calculates the calculated pitch. Round the 10th place of the average value and apply it to the grid every 100 cents, and the number of frames on the same grid with the rounded pitch value is a predetermined number (for example, 2 frames or more) continuously. When detected, that is, when the rounded pitch value is on the same grid for a predetermined time or more, the flat part detecting unit 111 has the same rounded pitch value. A frame on the grid may be detected as a flat part of the pitch.

FIG. 4 is a diagram for explaining an example of the concept of the flat part detection of the pitch in the flat part detection unit 111. FIG. 4 is a pitch waveform showing the pitch of the singing sound in time series, the vertical axis indicates the pitch (cent), and the horizontal axis indicates time (T). In FIG. 4, frames f _{n−1 to} f _{n + 6} are shown. The length of the frame f is arbitrary. The flat part detection part 111 calculates the average value of the pitch in each frame fn _{-1 to} fn _{+ 6} . In FIG. 4, the average value of pitches in the frames f _{n−1 to} f _{n + 6} is indicated by black circles (●). The flat part detection unit 111 rounds the calculated average value of the pitch to the 10th place and applies it to the grid every 100 cents, and sets the pitch value corresponding to the applied grid in each frame f _{n−1 to} f _{n + 6} . The pitch.

For example, in FIG. 4, the pitch corresponding to the grid corresponding to the value after rounding to the 10th of the average value of the pitch of the frame f _n−1 is 100 * (m + 3) cents. Pitch corresponding to grid true value after rounding digit of 10 of the average value of the pitch of the frame f _n is set to 100 * (m + 4) cents. The pitch corresponding to the grid corresponding to the value after rounding off to the 10th place of the average value of the pitch of the frame f _{n + 1} is 100 * (m + 4) cents. The pitch corresponding to the grid corresponding to the value after rounding to the tenth of the average value of the pitches of the frame f _{n + 2} is 100 * (m + 4) cents. The pitch corresponding to the grid corresponding to the value after rounding to the tenth of the average value of the pitch of the frame f _{n + 3} is 100 * (m + 3) cents. The pitch corresponding to the grid corresponding to the value after rounding to the 10th place of the average value of the pitch of the frame f _{n + 4} is 100 * (m + 2) cents. The pitch corresponding to the grid corresponding to the value after rounding to the tenth of the average value of the pitch of the frame f _{n + 5} is 100 * (m + 1) cents. The pitch corresponding to the grid corresponding to the value after rounding to the tenth of the average value of the pitches of the frame f _{n + 6} is 100 * (m + 2) cents. The flat part detecting unit 111 detects a frame having the same pitch as a flat part of the pitch when frames having the same pitch are continuously detected by a predetermined number or more (for example, two or more frames). Also good. In FIG. 4, the pitch corresponding to the grid that corresponds to the average value of the rounded pitches of frames f _n , f _{n + 1} , and f _{n + 2} is 100 * (m + 4) cents. Therefore, the flat part detection unit 111 detects the frames f _{n to} f _{n + 2} as a flat part of the pitch.

  In addition, in order to detect the flat part of the pitch more accurately, the flat part detection unit 111 is a predetermined number of frames on the grid in which the value after rounding to the 10th digit of the average value of the pitch is on the same grid. When the above-described (for example, two or more frames) are detected continuously, if the temporal variation of the pitch in the detected frame is within a predetermined range, the detected frame is You may detect as a flat part.

  Further, for example, the flat part detection unit 111 calculates the absolute value of the inclination of the data indicating the pitch (pitch waveform) for each frame with respect to the data indicating the pitch detected by the pitch detection unit 109, and calculates the calculated inclination It may be determined whether or not the absolute value of is less than or equal to a predetermined value. When the number of frames in which the absolute value of the calculated inclination is equal to or less than a predetermined value is continuously detected for a predetermined number or more (for example, two or more frames), the flat part detection unit 111 detects the absolute value of the calculated inclination. You may detect the flame | frame from which a value becomes below a predetermined value as a flat part of a pitch. In addition, when the number of frames in which the calculated absolute value of the slope is equal to or less than a predetermined value is continuously detected for a predetermined number or more (for example, two or more frames), the flat part detection unit 111 detects the detected frames. If the temporal variation of the pitch at is a variation within a predetermined width, the detected frame may be detected as a flat portion of the pitch.

  In addition, for example, if the difference between the maximum and minimum pitch values in a plurality of predetermined frames is within a predetermined value, the flat portion detection unit 111 detects the predetermined plurality of frames as a flat portion of the pitch. May be.

  The technique determination unit 113 is based on the change in volume before and after the flat part (volume flat part or pitch flat part) of the feature amount of the singing sound detected by the flat part detection unit 111, and the fluctuation of the pitch. Determine the singing voice technique. For example, the technique determination unit 113 may determine, as a singing technique, inflections such as scooping up, scooping down, jumping up, fall and fist, crescendo, decrescendo, and forte piano.

FIG. 5 is a diagram for explaining the concept of scooping determination in the technique determination unit 113. Scribbing is a technique that raises the pitch from bottom to top before the pitch stabilizes. FIG. 5 is a pitch waveform of the singing sound. It is assumed that the flatness detection unit 111 detects frames f _{n + 1 to} f _{n + 3} in FIG. 5 as flat portions of the frame. As shown in FIG. 5, the pitch is increased in the frame (frames f _n−1 , f _n ) before the flat portion. In this case, the technique determination unit 113 determines that scooping is included before the flat portion. Here, the technique determination unit 113 continuously performs a predetermined number or more (for example, two or more frames) of frames in which the slope of the data indicating the pitch (pitch waveform) is a predetermined value or more before the flat portion. If it is, it may be determined that scooping is included before the flat portion.

FIG. 6 is a diagram for explaining the concept of scrambling determination in the technique determination unit 113. Squeaking down is a technique that lowers the pitch from top to bottom before the pitch stabilizes. FIG. 6 is a pitch waveform of the singing sound. It is assumed that the flatness detection unit 111 detects the frames f _{n + 1 to} f _{n + 2} in FIG. 6 as flat portions of the frame. As shown in FIG. 6, the pitch is lowered in the frame (frames f _n−1 , f _n ) in front of the flat portion. In this case, the technique determination unit 113 determines that scrambling is included before the flat portion. Here, the technique determination unit 113 continuously has a predetermined number or more (for example, two or more frames) of frames in which the slope of the data indicating the pitch (pitch waveform) is equal to or less than a predetermined value before the flat portion. If it is, it may be determined that a scooping down is included before the flat portion.

FIG. 7 is a diagram for explaining the concept of the flip-up determination in the technique determination unit 113. FIG. 7 is a pitch waveform of the singing sound. Bounce-up is a technique that raises the pitch from bottom to top mainly after the pitch is stabilized. FIG. 7 is a pitch waveform of the singing sound. It is assumed that the flatness detection unit 111 detects the frames f _{n−1 to} f _{n + 1} in FIG. 7 as flat portions of the frame. As shown in FIG. 7, the pitch is increased in the frames (frames f _{n + 2} , f _{n + 3} , f _{n + 4} ) after the flat portion. In this case, the technique determination unit 113 determines that the flip-up is included after the flat portion. Here, after the flat portion, the technique determination unit 113 continuously has a predetermined number or more (for example, two or more frames) of frames in which the slope of the data indicating the pitch (pitch waveform) becomes a predetermined value or more. If it is, it may be determined that the flip-up is included after the flat portion.

FIG. 8 is a diagram for explaining the concept of fall determination in the technique determination unit 113. Fall is a technique that lowers the pitch from top to bottom after the pitch is stabilized. FIG. 8 is a pitch waveform of the singing sound. It is assumed that the flatness detection unit 111 detects frames f _{n + 1 to} f _{n + 3} in FIG. 8 as flat portions of the frame. As shown in FIG. 8, the pitch is lowered in the frames (frames f _{n + 4} and f _{n + 5} ) after the flat portion. In this case, the technique determination unit 113 determines that a fall is included behind the flat portion. Here, after the flat portion, the technique determination unit 113 continuously has a predetermined number or more (for example, two or more frames) of frames in which the slope of the data indicating the pitch (pitch waveform) is a predetermined value or less. If it is, it may be determined that a fall is included after the flat portion.

FIG. 9 is a diagram for explaining the concept of crescendo determination in the technique determination unit 113. FIG. 9 is a volume waveform of the singing sound. It is assumed that the flatness detection unit 111 detects the frames f _{n−1 to} f _{n + 1} in FIG. 9 as the flat part of the frame. As shown in FIG. 9, the volume increases in the frames after the flat portion (frames f _{n + 2} , f _{n + 3} , f _{n + 4} ). In this case, the technique determination unit 113 determines that the crescendo is included after the flat portion. Here, after the flat portion, the technique determination unit 113 continuously has a predetermined number or more (for example, two or more frames) of frames in which the slope of the data indicating the volume (volume waveform) is a predetermined value or more. If it is, it may be determined that the crescendo is included after the flat portion.

FIG. 10 is a diagram for explaining the concept of the decrescendo determination in the technique determination unit 113. FIG. 10 is a volume waveform of the singing sound. It is assumed that the flatness detection unit 111 detects the frames f _{n to} f _{n + 2} in FIG. 10 as flat portions of the frame. As shown in FIG. 10, the volume decreases in the frames after the flat part (frames f _{n + 3} , f _{n + 4} ). In this case, the technique determination unit 113 determines that the decrescendo is included after the flat portion. Here, after the flat portion, the technique determination unit 113 continuously has a predetermined number or more (for example, two or more frames) of frames in which the slope of the data (volume waveform) indicating the volume is equal to or less than a predetermined value. If it is, it may be determined that the crescendo is included after the flat portion.

FIG. 11 is a diagram for explaining the concept of fist determination in the technique determination unit 113. Fist is a technique in which the pitch is mainly changed from a reference to a predetermined pitch (for example, about 100 cents or more) up or down within a predetermined time range, and quickly returned to the reference pitch. FIG. 11 is a pitch waveform of the singing sound. It is assumed that the flatness detection unit 111 detects frames f _{n−1 to} f _{n + 1} in FIG. 11 as first flat portions and detects frames f _{n−4 to} f _{n + 6} as second flat portions. Here, it is assumed that the average value of the pitch in the first flat portion and the average value of the pitch in the second flat portion are substantially equal. Here, the average pitch is substantially equal means that the average value of the pitch in the first flat part and the second flat part is calculated, and the calculated average value of the pitch is rounded off and applied to the grid every 100 cents. In this case, the average value of the rounded pitch in the first flat portion and the average value of the rounded pitch in the second flat portion may be on the same grid. As shown in FIG. 11, the pitch vibrates up and down in the frame (frames f _{n + 2} and f _{n + 3} ) between the first flat portion and the second flat portion. In this case, the technique determination unit 113 determines that a fist is included between the first flat portion and the second flat portion.

FIG. 12 is a diagram for explaining the concept of forte piano determination in the technique determination unit 113. The forte piano is a performance technique that is reduced immediately after the volume is increased before the volume is stabilized. FIG. 12 is a volume waveform of the singing sound. The flatness detection unit 111 detects frames f _{n + 5 to} f _{n + 6} in FIG. 12 as flat portions of the frame. As shown in FIG. 12, the sound volume decreases in the frame immediately after becoming strong near the frame f _{n + 1 in} front of the flat portion (frames f _{n + 2} , f _{n + 3} , f _{n + 4} ). In this case, the technique determination unit 113 determines that a forte piano is included before the flat portion. Here, the technique determination unit 113 continuously has a predetermined number or more (for example, two or more frames) of frames in which the slope of the data (volume waveform) indicating the volume is equal to or less than a predetermined value before the flat portion. If it is, it may be determined that the forte piano is included in front of the flat portion.

  The technique determination function 100 may include an accompaniment output unit 101 that reads accompaniment data corresponding to a song designated by the singer and outputs an accompaniment sound from the sound output unit 25 via the signal processing unit 21. Good. In this case, the input sound to the sound input unit 23 during the period in which the accompaniment sound is output is recognized as the determination target singing voice.

  As described above, the technique determination apparatus 10 according to the first embodiment detects feature amounts (pitch and volume) from input singing voice data in time series, and features feature flat portions (pitch flat portions and volume levels). A flat part) is detected, and a specific technique is determined based on a change in volume (change in volume) and a change in pitch before and after the flat part of the feature amount (flat part of the pitch or flat part of the volume). Since a series of processes from pitch and volume detection to technique determination can be executed with a small amount of calculation for each predetermined frame, singing voice data accumulation, machine learning, and reference data are unnecessary. This makes it possible to accurately determine a specific technique in real time while suppressing the amount of calculation.

<Modification>
Although the embodiments of the present invention have been described above, the present invention is not limited to the above-described embodiments, and can be implemented in various other modes. An example of the following other aspect is shown.

(Modification 1)
The functions realized in the technique determination apparatus 10 may include a singing evaluation function based on the technique determined by the technique determination, in addition to the singing technique determination function 100 described above. Below, the evaluation function 200 implement | achieved when the control part 11 of the technique determination apparatus 10 runs the control program 13a memorize | stored in the memory | storage part 13 is demonstrated. Part or all of the configuration for realizing the evaluation function 200 may be realized by hardware.

  FIG. 2 also shows an evaluation function 200 that evaluates a song based on the technique determined by the technique determination function 100 together with the technique determination function 100. Referring to FIG. 2, the evaluation function 200 includes a technique acquisition unit 201, a pitch acquisition unit 203, a volume acquisition unit 205, a reference data acquisition unit 207, a comparison unit 209, and an evaluation unit 211.

  The technique acquisition unit 201 acquires data indicating the technique of the singing sound determined by the technique determination unit 111 in the technique determination function 100 and outputs the data to the comparison unit 209. The pitch acquisition unit 203 acquires data indicating the pitch detected by the pitch detection unit 105 in the technique determination function 100 in time series, and outputs the data to the comparison unit 209. The volume acquisition unit 205 acquires data indicating the volume of the singing sound detected by the volume detection unit 107 in the technique determination function 100 in time series, and outputs the data to the comparison unit. The reference data acquisition unit 207 reads out and acquires the evaluation reference data 13 d of the corresponding singing sound stored in the storage unit 13, and outputs it to the comparison unit 209. Note that the evaluation reference sound data 13d only needs to indicate a sound that serves as a reference for evaluation, and therefore does not necessarily indicate a voice that serves as a model for singing.

  The comparison unit 209 compares the data indicating the pitch of the acquired singing sound, the data indicating the volume of the singing sound, and the data indicating the technique of the singing sound with the corresponding evaluation reference data 13d of the singing sound. The comparison unit 209 may compare the acquired data indicating the pitch of the singing sound with the reference pitch data included in the evaluation reference data 13d in time series, and the data indicating the volume of the acquired singing sound and the evaluation reference data 13d. The reference volume data included in the singing sound may be compared in time series, or the acquired singing sound technique may be compared with the reference singing technique data included in the value reference data 13d. For example, the comparison unit 209 relates to a technique such as extraction or vibrato, etc., a frequency standard deviation, a frequency average value, a pitch amplitude average value, a pitch amplitude standard deviation, a slope of a linear approximation line of the pitch amplitude, and the like. The acquired singing sound technique may be compared with the reference singing technique included in the value reference data 13d. The comparison unit 209 outputs the comparison result to the evaluation unit 211.

  Based on the comparison result output from the comparison unit 209, the evaluation unit 211 calculates an evaluation value that serves as an index for evaluating the singing sound. The evaluation unit 211 evaluates the higher the degree of coincidence between the data indicating the pitch of the singing sound by the singer, the data indicating the volume of the singing sound, and the data indicating the technique of the singing sound and the evaluation reference data 13d of the corresponding singing sound. The value is calculated to be high, and the evaluation value is calculated to be lower as the mismatch degree is higher. Moreover, the evaluation part 211 may give a weighting value about the techniques with high difficulty levels, such as extraction and vibrato, when the coincidence of the singing sound by the singer and the evaluation reference data 13d of the singing sound is high. The evaluation result by the evaluation unit 211 may be displayed on the display unit 17.

(Modification 2)
In the above-described embodiment, in the technique determination function 100, the flat part detection unit 111 detects the flat part of the feature amount based on the time-series variation of the feature amount (pitch and volume) detected by the feature amount detection unit 105. Detected. In the first embodiment described above, the flatness detection unit 111 determines that the number of frames whose feature amount variation is within a predetermined threshold or a predetermined width continues for a predetermined number or more, If it is longer than the predetermined time, the flat part detection unit 111 detects the detected frame as a flat part of the feature amount. Here, the predetermined time determined as the flat portion may be determined by singing voice data (input sound).

  FIG. 13 is a block diagram illustrating a configuration of the technique determination function 100a according to the modification of the first embodiment of the present invention. The technique determination function 100a includes a tempo estimation unit 110 that detects the tempo of singing voice data (input sound) based on data (volume waveform) indicating the volume detected by the volume detection unit 107 in time series, and includes a flat unit. 1st Embodiment of this invention except the detection part 111a determining the predetermined time required in order to determine the flat part of a feature-value based on the tempo of the song audio | voice data detected by the tempo estimation part 110 This is the same as the technique determination function 100.

  The tempo estimation unit 110 acquires data (volume waveform) indicating the volume of the singing voice data in time series from the volume detection unit 107. The tempo estimation unit 110 detects the tempo of the singing voice data (input sound) based on the change in volume (volume level). The tempo estimation unit 110 transmits the detected tempo of the singing voice data (input sound) to the flat part detection unit 111a. The flat part detection unit 111a determines the length of a predetermined time necessary to determine as a flat part of the feature amount based on the tempo of the singing voice data detected by the tempo estimation unit 110.

  Thus, according to the input singing sound, the precision of the detection of a flat part is improved by determining the length of the predetermined time required to determine as a flat part of a feature-value according to a singing sound. be able to.

  When the technique determination function 100a includes the accompaniment output unit 101, the tempo estimation unit 110 acquires time-series data (volume waveform) of the accompaniment sound volume from the accompaniment output unit 101, and based on the accompaniment sound volume. Thus, the tempo of the corresponding song may be detected.

  In the technique determination functions 100 and 100a described above, the sound indicated by the singing voice data acquired by the input sound acquisition unit 103 is not limited to the voice by the singer, but may be voice by singing synthesis or instrument sound. It may be. If it is a musical instrument sound, it is desirable to be a single note performance. In the case of instrument sounds, there is no concept of consonants and vowels, but depending on the performance method, there is a tendency similar to singing at the starting point of pronunciation of each sound. Therefore, the same determination may be made for musical instrument sounds.

  Based on the configuration described as the embodiment of the present invention, those in which a person skilled in the art appropriately added, deleted, or changed the design of the component, or added, omitted, or changed conditions of the process are also included in the present invention. As long as the gist of the present invention is provided, the scope of the present invention is included.

  Of course, other operational effects that are different from the operational effects brought about by the above-described embodiment are obvious from the description of the present specification or can be easily predicted by those skilled in the art. It is understood that this is brought about by the present invention.

DESCRIPTION OF SYMBOLS 10 ... Technique determination apparatus, 11 ... Control part, 13 ... Memory | storage part, 15 ... Operation part, 17 ... Display part, 19 ... Communication part, 21 ... Signal processing part, 23 ... Sound input part, 25 ... Sound output part, 100 , 100a ... technique determination function, 101 ... accompaniment output unit, 103 ... input sound acquisition unit, 105 ... feature amount detection unit, 107 ... volume detection unit, 109 ... pitch detection unit, 110 ... tempo detection unit, 111, 111a ... flat Part detection unit 113 ... technique determination part 200 ... evaluation function 201 ... technique acquisition part 203 ... pitch acquisition part 205 ... volume acquisition part 207 ... reference data acquisition part 209 ... comparison part 211 ... evaluation part

Claims (5)

An input sound acquisition unit for acquiring the input sound;
A feature amount detection unit that detects the feature amount of the input sound acquired by the input sound acquisition unit in time series; and
A flat part detection unit that detects a flat part of the feature quantity based on the feature quantity acquired by the feature quantity detection unit;
A technique determination unit that determines a technique of the input sound based on a variation in the feature amount in a predetermined period before or after the flat portion of the feature amount;
A technique determination apparatus comprising: The flat part detection part is
Detecting a period in which the time-series variation of the feature amount is equal to or less than a predetermined variation;
The technique determination apparatus according to claim 1, wherein when the period is equal to or longer than a predetermined time, the period is detected as the flat portion.   The technique determination apparatus according to claim 1, wherein the feature amount is pitch or volume. A tempo estimation unit that estimates the tempo of the input sound based on the feature amount;
The technique determination apparatus according to claim 2, wherein the predetermined time is determined according to the tempo. When a plurality of flat portions are detected by the flat portion detection unit,
The technique determination apparatus according to claim 1, wherein the predetermined period is a period between two flat portions adjacent to each other in time series.

Images