JP6627482B2 - Technique determination device - Google Patents

Description

  The present invention relates to a technique for determining a technique of an input sound.

  The karaoke apparatus has a function of analyzing and evaluating a singing voice. Various methods are used to evaluate singing. As one of the methods, for example, Patent Literature 1 discloses a karaoke scoring device that detects and evaluates a hiccup that is one of singing techniques.

JP 2004-102149 A

  However, the technique disclosed in Patent Literature 1 has a problem that erroneous detection is often performed, and a singing technique cannot be accurately determined and evaluated.

  An object of the present invention is to determine a technique of an input sound.

  According to an embodiment of the present invention, an input sound acquisition unit that acquires an input sound, a feature amount detection unit that detects a feature amount of the input sound acquired by the input sound acquisition unit in a time series, and the feature amount A flat portion detection unit that detects a flat portion of the feature amount based on the feature amount obtained by the detection unit; and a change in the feature amount in a predetermined period before or after the flat portion of the feature amount. And a technique determining unit for determining a technique of the input sound.

  The flat portion detection unit may detect a period in which the time series variation of the feature amount is equal to or less than a predetermined variation, and may detect the period as the flat portion when the period is equal to or greater than a predetermined time.

  The feature amount may be a pitch or a volume.

  The technique determining device may include a tempo estimating unit that estimates a tempo of the input sound based on the feature amount, and the predetermined time may be determined according to the tempo.

  When a plurality of flat portions are detected by the flat portion detection section, the predetermined period may be a period between two flat portions that are adjacent to each other in time series.

  According to an embodiment of the present invention, a computer acquires an input sound, detects a feature amount of the input sound in time series, detects a flat portion of the feature amount based on the feature amount, and flattens the feature amount. A program is provided for executing the determination of the technique of the input sound based on the variation of the feature amount in a predetermined period before or after the unit.

  According to an embodiment of the present invention, it is possible to determine a technique of an input sound.

It is a block diagram showing composition of a technique judging device in one embodiment of the present invention. It is a block diagram showing composition of a technique judgment function and an evaluation function in one embodiment of the present invention. It is a figure for explaining the concept of flat part detection of a volume in the flat part detection part of the technique judging function in one embodiment of the present invention. It is a figure for explaining the concept of flat part detection of a pitch by the flat part detection part of the technique judging function in one embodiment of the present invention. It is a figure for explaining the concept of scooping-up judgment by the skill judging part of the skill judging function in one embodiment of the present invention. It is a figure for explaining the concept of a flicking judgment by the technique judgment part of the technique judgment function in one embodiment of the present invention. It is a figure for explaining the concept of jumping-up judgment by the technique judgment part of the technique judgment function in one embodiment of the present invention. It is a figure for explaining the concept of the fall judgment by the technique judgment part of the technique judgment function in one embodiment of the present invention. It is a figure for explaining the concept of crescendo judgment by the technique judgment part of the technique judgment function in one embodiment of the present invention. It is a figure for explaining the concept of the decrescendo judgment by the technique judgment part of the technique judgment function in one embodiment of the present invention. It is a figure for explaining the concept of fist judgment by the technique judgment part of the technique judgment function in one embodiment of the present invention. It is a figure for explaining the concept of forte piano judgment by the technique judgment part of the technique judgment function in one embodiment of the present invention. It is a block diagram showing composition of a technique judgment function and an evaluation function in a modification of one embodiment of the present invention.

  Hereinafter, a technique determining apparatus according to an embodiment of the present invention will be described in detail with reference to the drawings. The embodiments described below are examples of the embodiments of the present invention, and the present invention is not limited to these embodiments.

<First embodiment>
The technique determining apparatus according to the first embodiment of the present invention will be described in detail with reference to the drawings. The technique determining device according to the first embodiment is a device having a function of determining a singing sound of a singing user (hereinafter, also referred to as a singer). This technique determination device detects the pitch and volume of a singing sound in a time series, and determines a specific technique based on a change in volume and a change in pitch.

[Hardware]
FIG. 1 is a block diagram illustrating a configuration of a technique determining apparatus 10 according to the first embodiment of the present invention. The technique determination device 10 is, for example, a karaoke device having a singing score function. The technique determination device 10 includes a control unit 11, a storage unit 13, an operation unit 15, a display unit 17, a communication unit 19, and a signal processing unit 21. A signal input unit (for example, a microphone) 23 and a sound output unit (for example, a speaker) 25 are connected to the signal processing unit 21. These components are interconnected via a bus.

  The control unit 11 includes an arithmetic processing circuit such as a CPU. The control unit 11 causes the CPU to execute the control program 13 a stored in the storage unit 13 to implement various functions in the technique determination device 10. The realized functions include a singing technique determination function. In addition, the realized function may include a singing evaluation function based on the technique determined by the technique determination.

  The storage unit 13 is a storage device such as a nonvolatile memory and a hard disk. The storage unit 13 stores a control program 13a for realizing the technique determination function. The control program 13a may include a singing evaluation function. The control program 13a may be provided while being stored in a computer-readable recording medium such as a magnetic recording medium, an optical recording medium, a magneto-optical recording medium, and a semiconductor memory. In this case, the technique determination device 10 only needs to include a device that reads a recording medium. Further, the control program 13a may be downloaded via a network such as the Internet.

  Further, the storage unit 13 stores music data 13b and singing voice data 13c as singing data. The storage unit 13 may store the evaluation reference data 13d. The music data 13b includes data related to karaoke songs, for example, guide melody data, accompaniment data, lyrics data, and the like. The guide melody data is data indicating a melody of a song. The accompaniment data is data indicating the accompaniment of the song. The guide melody data and the accompaniment data may be data expressed in a MIDI format. The lyrics data is data for displaying the lyrics of the song and data indicating the timing of changing the color of the displayed lyrics telop. The singing voice data 13c is data corresponding to the singing voice input by the singer from the sound input unit 23. In the present embodiment, the singing voice data 13c is stored in the storage unit 13 until the singing voice is determined by the technique determining function. The evaluation criterion data 13d is information used as a criterion for singing voice evaluation by the evaluation function, and is previously stored in music data indicating a singing song to be evaluated (a singing song output when the singing voice is input). The associated reference sound data may be used.

  The operation unit 15 is a device such as an operation button, a keyboard, and a mouse provided on an operation panel, a remote controller, and the like, and outputs a signal corresponding to an input operation to the control unit 11. The display unit 17 is a display device such as a liquid crystal display and an organic EL display, and displays a screen based on control by the control unit 11. Note that a touch panel device in which the operation unit 15 and the display unit 17 are integrated may be used. The communication unit 19 transmits and receives information to and from an external device such as a server by connecting to a communication line such as the Internet or a LAN based on the control of the control unit 11. Note that the function of the storage unit 13 may be realized by an external device that can communicate with the communication unit 19.

  The signal processing unit 21 includes a sound source that generates an audio signal from a MIDI format signal, an A / D converter, a D / A converter, and the like. The singing voice is converted to an electric signal in the sound input unit 23 and input to the signal processing unit 21, A / D converted in the signal processing unit 21, and output to the control unit 11. The singing voice is stored in the storage unit 13 as singing voice data 13c. The accompaniment data is read out by the control unit 11, D / A-converted by the signal processing unit 21, and output from the sound output unit 25 as accompaniment of a singing song. At this time, the guide melody may also be output from the sound output unit 25.

[Technique judgment function]
A technique determination function realized by the control unit 11 of the technique determination apparatus 10 executing the control program 13a stored in the storage unit 13 will be described. A part or all of the configuration for realizing the technique determination function described below may be realized by hardware.

  FIG. 2 is a block diagram illustrating a configuration of the technique determining function 100 according to the first embodiment of the present invention. Referring to FIG. 2, the technique determination function 100 includes an input sound acquisition unit 103, a feature amount detection unit 105, a flat portion detection unit 111, and a technique determination unit 113.

  The input sound obtaining unit 103 obtains singing voice data (input sound) corresponding to the singing voice input from the sound input unit 23. Note that the input sound obtaining unit 103 directly obtains the singing voice data from the signal processing unit 21, but may obtain the singing voice data once stored in the storage unit 13. Further, the input sound acquisition unit 103 is not limited to acquiring the singing voice data indicating the input sound to the sound input unit 23, and the singing voice data indicating the input sound to the external device is transmitted by the communication unit 19 via the network. May be acquired. In the present embodiment, the input sound acquisition unit 103 sequentially outputs the singing voice data sequentially input during the reproduction of the music data to the feature amount detection unit 105.

  The feature amount detection unit 105 acquires the singing voice data from the input sound acquisition unit 103. The characteristic amount detection unit 105 detects the characteristic amount of the singing sound including the volume and the pitch of the acquired singing voice data. The feature amount detection unit 105 includes a volume detection unit 107 and a pitch detection unit 109.

  The volume detection unit 107 detects the volume of the singing sound in time series from the singing voice data acquired by the input sound acquisition unit 103. That is, the volume detection unit 107 detects a temporal change in the volume of the singing sound based on the singing voice data. In the present embodiment, the volume detector 107 detects the volume based on the amplitude of the voice signal indicated by the singing voice data. Volume detection section 107 outputs data (volume waveform) indicating the detected volume in time series to flat section detection section 111 and technique determination section 113 in time series.

  The pitch detection unit 109 detects the pitch of the singing sound in chronological order from the singing voice data acquired by the input sound acquisition unit 103. That is, the pitch detection unit 109 detects a zero cross when the waveform of the voice signal indicated by the singing voice data changes from negative to positive for each frame (data sample divided by a predetermined period), and determines the time interval of the zero cross. The pitch (frequency) of the singing sound is specified by measurement. At this time, a high-frequency component which becomes a noise component may be cut from the audio signal by a low-pass filter, or a DC component may be cut by a high-pass filter. Further, the pitch detection unit 109 may specify the pitch from a spectrum obtained by performing FFT (Fast Fourier Transform) on the singing voice data. The pitch detecting unit 109 outputs data (pitch waveform) indicating the detected pitch in a time series to the flat part detecting unit 111 and the technique determining unit 113 in a time series.

  The flat part detection unit 111 detects a flat part of the feature amount based on a time-series variation of the feature amount detected by the feature amount detection unit 105. The flat part detection unit 111 outputs data indicating the flat part of the detected feature amount to the technique determination unit 113.

  Specifically, the flat portion detection unit 111 may detect a flat portion in the data indicating the volume detected by the volume detection unit 107, that is, a period in which the volume is substantially constant as a flat portion of the volume. . For example, the flat part detection unit 111 determines that the fluctuation of the sound volume of the data indicating the sound volume detected by the sound volume detection unit 107 for each frame (data sample divided every predetermined time) is equal to or less than a predetermined threshold ΔVth. It is determined whether or not. If frames whose volume fluctuations are equal to or less than the predetermined threshold value ΔVth are detected continuously for a predetermined number or more (for example, two frames or more), that is, a period in which the volume fluctuations are equal to or less than the predetermined threshold value ΔVth is determined in advance. If the predetermined time is equal to or longer than the predetermined time, the flat part detection unit 111 may detect a frame whose volume fluctuation is equal to or less than the predetermined threshold ΔVth as a flat part of the volume.

FIG. 3 is a diagram for explaining an example of the concept of flat part detection of the sound volume in the flat part detection unit 111. FIG. 3 is a volume waveform showing the volume of the singing sound in a time series. The vertical axis indicates the volume (V), and the horizontal axis indicates the time (T). FIG. 3 illustrates frames f _{n−1 to} f _{n + 6} . The length of the frame f is arbitrary. The start point detection unit 109 determines whether or not the fluctuation of the sound volume in each of the frames f _{n−1 to} f _{n + 6} is equal to or smaller than a predetermined threshold ΔVth. For example, the fluctuation of the sound volume in the frames f _n−1 , f _n , f _{n + 1} , f _{n + 4} , f _{n + 5} , f _{n + 6} exceeds a predetermined threshold value ΔVth (ΔVn−1> ΔVth, ΔVn + 1> ΔVth, ΔVn + 4> ΔVth, ΔVn + 5> When the fluctuation of the sound volume at ΔVth, ΔVn + 6> ΔVth), f _{n + 2} , f _{n + 3} is equal to or less than a predetermined threshold value ΔVth (ΔVn + 2 ≦ ΔVth, ΔVn + 3 ≦ ΔVth), the flat part detection unit 111 detects the frames f _{n + 2} , f _{n + 3} . It may be detected as a flat part of the volume.

Further, the flat portion detection section 111 may calculate the average value of the volume for each of a plurality of predetermined frames, and may detect the flat portion of the volume based on the fluctuation of the calculated average value of the volume. For example, the flat part 111 calculates the average value of the volume in the frames f _n−1 , f _n and f _{n + 1} , and then calculates the average value of the volume in the frames f _n , f _{n + 1} and f _{n + 2} . Then, the average value of the sound volume in the frame f _{n + 1} , the frame f _{n + 2,} and the frame f _{n + 3} is calculated. As described above, the flat portion detection unit 111 calculates the average value of the volume while shifting the period for calculating the average value of the volume by a predetermined time (here, a time corresponding to one frame). When the difference between the average values of the sound volumes of a plurality of continuous predetermined frames (for example, frames f _{n−1 to} f _{n + 1} and frames _{n to} f _{n + 2} ) is equal to or smaller than a predetermined threshold, the flat portion detection unit 111 (For example, frames f _{n−1 to} f _{n + 2} ) may be detected as a flat part of the volume.

  Further, for example, the flat part detection unit 111 calculates, for each frame, the absolute value of the slope of the data indicating the volume (volume waveform) with respect to the data indicating the volume detected by the volume detection unit 107, and calculates the calculated gradient. May be determined whether or not the absolute value of is less than or equal to a predetermined value. When frames in which the calculated absolute value of the inclination is equal to or less than the predetermined value are continuously detected for a predetermined number or more (for example, two or more frames), the flat portion detection unit 111 determines the absolute value of the calculated inclination. A frame whose value is equal to or less than a predetermined value may be detected as a flat portion of the volume.

  Further, the flat portion detection section 111 may detect a flat portion in the data indicating the pitch detected by the pitch detection section 109, that is, a period in which the pitch is substantially constant as a flat portion of the pitch. For example, the flat part detection unit 111 calculates an average value of the pitch for each frame (data sample divided every predetermined time) for the data indicating the pitch detected by the pitch detection unit 109, and calculates the average of the calculated pitch. The tens place of the average value is rounded and applied to a grid of 100 cents, and the number of frames on the same grid having the same rounded pitch value is continuously equal to or greater than a predetermined number (for example, 2 frames or more). If detected, that is, if the value of the pitch that has been rounded on the same grid is equal to or longer than a predetermined time, the flat portion detection unit 111 determines that the value of the pitch that has been rounded is the same. A frame on the grid may be detected as a flat portion of the pitch.

FIG. 4 is a diagram for explaining an example of the concept of detecting a flat portion of a pitch in the flat portion detection section 111. FIG. 4 is a pitch waveform showing the pitch of the singing sound in a time series. The vertical axis indicates the pitch (cent), and the horizontal axis indicates the time (T). FIG. 4 shows frames f _{n−1 to} f _{n + 6} . The length of the frame f is arbitrary. The flat part detection unit 111 calculates the average value of the pitch in each of the frames f _{n−1 to} f _{n + 6} . In FIG. 4, the average value of the pitch in each frame f _{n-1 to} f _{n + 6} is indicated by a black circle (●). The flat part detection unit 111 rounds the tens place of the calculated average value of the pitch, applies the result to the grid every 100 cents, and calculates the pitch value corresponding to the applicable grid in each of the frames f _{n−1 to} f _{n + 6} . The pitch is used.

For example, in FIG. 4, the pitch corresponding to the grid corresponding to the rounded value of the tens digit of the average value of the pitch of the frame fn ₋₁ is 100 * (m + 3) cents. Pitch corresponding to grid true value after rounding digit of 10 of the average value of the pitch of the frame f _n is set to 100 * (m + 4) cents. The pitch corresponding to the grid corresponding to the rounded value of the tens digit of the average value of the pitch of the frame fn _{+ 1} is 100 * (m + 4) cents. The pitch corresponding to the grid corresponding to the rounded value of the tens digit of the average value of the pitch of the frame fn _{+ 2} is 100 * (m + 4) cents. The pitch corresponding to the grid corresponding to the rounded value of the tens digit of the average value of the pitch of the frame fn _{+ 3} is 100 * (m + 3) cents. The pitch corresponding to the grid corresponding to the rounded value of the tens digit of the average value of the pitch of the frame fn _{+ 4} is 100 * (m + 2) cents. The pitch corresponding to the grid corresponding to the rounded value of the tens digit of the average value of the pitch of the frame fn _{+ 5} is 100 * (m + 1) cents. The pitch corresponding to the grid corresponding to the rounded value of the tens digit of the average value of the pitch of the frame fn _{+ 6} is 100 * (m + 2) cents. When the frames having the same pitch are continuously detected for a predetermined number or more (for example, two frames or more), the flat portion detection unit 111 detects the frames having the same pitch as the flat portions of the pitch. Is also good. In FIG. 4, the pitch corresponding to the grid that fits the average of the rounded pitches of frames f _n , f _{n + 1} , f _{n + 2} is 100 * (m + 4) cents. Therefore, the flat part detection unit 111 detects the frames f _{n to} f _{n + 2} as flat parts with a pitch.

  In order to more accurately detect the flat portion of the pitch, the flat portion detecting section 111 determines a predetermined number of frames in which the value of the average value of the pitch on the same grid after rounding to the tenth is the same. In the case where the detection is continuously performed as described above (for example, two or more frames), if the temporal variation of the pitch in the detected frame is a variation within a predetermined width, the detected frame is determined by the pitch. It may be detected as a flat part.

  Further, for example, the flat portion detection unit 111 calculates, for each frame, the absolute value of the slope of the data indicating the pitch (pitch waveform) for the data indicating the pitch detected by the pitch detection unit 109, and calculates the calculated slope. May be determined whether or not the absolute value of is less than or equal to a predetermined value. When frames in which the calculated absolute value of the inclination is equal to or less than the predetermined value are continuously detected for a predetermined number or more (for example, two or more frames), the flat portion detection unit 111 determines the absolute value of the calculated inclination. A frame whose value is equal to or less than a predetermined value may be detected as a flat portion of the pitch. In addition, when the frames in which the calculated absolute value of the inclination is equal to or smaller than the predetermined value are continuously detected for a predetermined number or more (for example, two frames or more), the flat portion detection unit 111 outputs the detected frame. If the temporal variation of the pitch in the variation is within a predetermined width, the detected frame may be detected as a flat portion of the pitch.

  Further, for example, if the difference between the maximum value and the minimum value of the pitch in a predetermined plurality of frames is within a predetermined value, the flat portion detection unit 111 detects the predetermined plurality of frames as the flat portion of the pitch. You may.

  The technique determining unit 113 determines a change in volume before and after a flat portion (a flat portion of the volume or a flat portion of the pitch) of the characteristic amount of the singing sound detected by the flat portion detecting unit 111, and a change in the pitch. Determine the singing voice technique. For example, the technique determination unit 113 may determine, as the singing technique, hiccups, hiccups, jumps, falls and fists, crescendos, decrescendos, and intonations such as forte piano.

FIG. 5 is a diagram for explaining the concept of the judge-up determination in the technique determination unit 113. Scooping is a technique that mainly raises the pitch from bottom to top before the pitch stabilizes. FIG. 5 is a pitch waveform of a singing sound. It is assumed that the flatness detection unit 111 has detected the frames fn _{+ 1 to} fn _{+ 3} in FIG. 5 as flat portions of the frame. As shown in FIG. 5, in the frame (frames f _n−1 , f _n ) before the flat portion, the pitch has increased. In this case, the technique determining unit 113 determines that the flat portion includes the flip-up. Here, before the flat portion, the technique determination unit 113 continuously performs a predetermined number of frames (for example, two frames or more) in which the slope of the data indicating the pitch (pitch waveform) becomes a predetermined value or more. If so, it may be determined that the flat portion includes a hiccup.

FIG. 6 is a diagram for explaining the concept of the crouching-down determination in the technique determining unit 113. Scooping is a technique that lowers the pitch from top to bottom before the pitch stabilizes. FIG. 6 shows a pitch waveform of a singing sound. It is assumed that the flatness detection unit 111 has detected the frames fn _{+ 1 to} fn _{+ 2} in FIG. 6 as flat portions of the frame. As shown in FIG. 6, in the frame (frames f _n−1 , f _n ) before the flat portion, the pitch is falling. In this case, the technique determining unit 113 determines that the flat portion includes the downhill position. Here, the technique determination unit 113 continuously arranges, in front of the flat portion, a predetermined number of frames (for example, two frames or more) in which the slope of the data indicating the pitch (pitch waveform) is equal to or less than a predetermined value. If so, it may be determined that the flat portion includes the crouching.

FIG. 7 is a diagram for explaining the concept of the flip-up determination in the technique determination unit 113. FIG. 7 shows a pitch waveform of a singing sound. Bouncing is a technique for raising the pitch from bottom to top after the pitch has stabilized. FIG. 7 shows a pitch waveform of a singing sound. It is assumed that the flatness detection unit 111 has detected the frames f _{n−1 to} f _{n + 1} in FIG. 7 as flat portions of the frame. As shown in FIG. 7, in the frames after the flat portion (frames f _{n + 2} , f _{n + 3} , f _{n + 4} ), the pitch increases. In this case, the technique determining unit 113 determines that a jump is included after the flat portion. Here, after the flat portion, the technique determining unit 113 continuously performs a predetermined number of frames (for example, two frames or more) in which the slope of the data indicating the pitch (pitch waveform) becomes a predetermined value or more. If so, it may be determined that a jump is included after the flat portion.

FIG. 8 is a diagram for explaining the concept of the fall determination in the technique determination unit 113. Fall is a technique of lowering the pitch from top to bottom mainly after the pitch has stabilized. FIG. 8 shows a pitch waveform of a singing sound. It is assumed that the flatness detection unit 111 has detected the frames fn _{+ 1 to} fn _{+ 3} in FIG. 8 as flat portions of the frame. As shown in FIG. 8, in the frames (frames f _{n + 4} and f _{n + 5} ) after the flat portion, the pitch is lowered. In this case, the technique determining unit 113 determines that a fall is included behind the flat part. Here, after the flat portion, the technique determining unit 113 continuously performs a predetermined number of frames (for example, two frames or more) in which the slope of the data indicating the pitch (pitch waveform) is equal to or less than a predetermined value. If so, it may be determined that a fall is included after the flat portion.

FIG. 9 is a diagram for explaining the concept of crescendo determination in technique determination section 113. FIG. 9 shows a volume waveform of a singing sound. It is assumed that the flatness detection unit 111 has detected the frames f _{n−1 to} f _{n + 1} in FIG. 9 as flat portions of the frame. As shown in FIG. 9, in the frames after the flat portion (frames f _{n + 2} , f _{n + 3} , f _{n + 4} ), the volume is increased. In this case, the technique determining unit 113 determines that crescendo is included after the flat portion. Here, after the flat portion, the technique determination unit 113 continuously performs a predetermined number of (for example, two or more) frames in which the slope of the data (volume waveform) indicating the volume is equal to or more than a predetermined value. In such a case, it may be determined that crescendo is included after the flat portion.

FIG. 10 is a diagram for explaining the concept of decrescendo determination in technique determination section 113. FIG. 10 shows a volume waveform of the singing sound. Flat detector 111 is assumed to have detected the frame _{f n ~f} n _{+ 2} in FIG. 10 as a flat portion of the frame. As shown in FIG. 10, in the frames after the flat part (frames f _{n + 3} , f _{n + 4} ), the sound volume decreases. In this case, the technique determining unit 113 determines that a decrescendo is included after the flat portion. Here, after the flat portion, the technique determination unit 113 continuously performs a predetermined number of frames (for example, two frames or more) in which the slope of the data (volume waveform) indicating the volume is equal to or less than a predetermined value. If so, it may be determined that a decrescendo is included after the flat portion.

FIG. 11 is a diagram for explaining the concept of fist determination in technique determining section 113. The fist is a technique in which the pitch is changed up or down by a predetermined pitch (for example, about 100 cents or more) within a predetermined time range from a certain reference, and quickly returned to the reference pitch. FIG. 11 shows a pitch waveform of a singing sound. It is assumed that the flat detection unit 111 detects the frames f _{n−1 to} f _{n + 1} in FIG. 11 as the first flat portions, and detects the frames f _{n−4 to} f _{n + 6} as the second flat portions. Here, it is assumed that the average value of the pitch in the first flat portion is substantially equal to the average value of the pitch in the second flat portion. Here, that the averages of the pitches are substantially equal means that the average value of the pitches in the first flat portion and the second flat portion is calculated, the calculated average value of the pitches is rounded, and the average value is applied to a grid of 100 cents. In this case, the average of the rounded pitch in the first flat portion and the average of the rounded pitch in the second flat portion may be on the same grid. As shown in FIG. 11, in a frame (frames fn _{+ 2} and fn _{+ 3} ) between the first flat portion and the second flat portion, the pitch vibrates up and down. In this case, the technique determining unit 113 determines that a fist is included between the first flat portion and the second flat portion.

FIG. 12 is a diagram for explaining the concept of forte piano determination in technique determination section 113. The forte piano is a playing technique that mainly decreases the volume immediately after the volume is increased before the volume is stabilized. FIG. 12 shows a volume waveform of a singing sound. It is assumed that the flatness detection unit 111 has detected the frames fn _{+ 5 to} fn _{+ 6} in FIG. 12 as flat portions of the frame. As shown in FIG. 12, in the frames (frames f _{n + 2} , f _{n + 3} , f _{n + 4} ) immediately after becoming strong in the vicinity of the frame f _{n + 1} before the flat part, the sound volume decreases. In this case, the technique determining unit 113 determines that forte piano is included before the flat portion. Here, the technique determination unit 113 continuously performs a predetermined number of frames (for example, two frames or more) in which the slope of the data (volume waveform) indicating the volume is equal to or less than a predetermined value before the flat portion. If so, it may be determined that a forte piano is included before the flat portion.

  The technique determining function 100 may include an accompaniment output unit 101 that reads out accompaniment data corresponding to a singing song specified by the singer and outputs an accompaniment sound from the sound output unit 25 via the signal processing unit 21. Good. In this case, the input sound to the sound input unit 23 during the period in which the accompaniment sound is output is recognized as the singing voice to be determined.

  As described above, the technique determination apparatus 10 according to the first embodiment detects a feature amount (pitch and volume) from input singing voice data in a time series, and detects a flat portion of the feature amount (flat portion of the pitch and volume of the volume). A flat part) is detected, and a specific technique is determined based on the fluctuation of the sound volume (change of the sound volume) before and after the flat part of the characteristic amount (the flat part of the pitch or the flat part of the sound volume) and the fluctuation of the pitch. A series of processes from pitch and volume detection to technique determination can be executed with a small amount of calculation for each predetermined frame, so that singing voice data accumulation, machine learning, and reference data are unnecessary. This makes it possible to accurately determine a specific technique in real time while suppressing the amount of calculation.

<Modification>
Although the embodiments of the present invention have been described above, the present invention is not limited to the above-described embodiments, and can be implemented in other various aspects. The following shows an example of another embodiment.

(Modification 1)
The functions realized by the technique determination device 10 may include a singing evaluation function based on the technique determined by the technique determination in addition to the singing technique determination function 100 described above. Hereinafter, the evaluation function 200 realized by the control unit 11 of the technique determination apparatus 10 executing the control program 13a stored in the storage unit 13 will be described. Part or all of the configuration for implementing the evaluation function 200 may be implemented by hardware.

  FIG. 2 shows an evaluation function 200 that evaluates a singing based on the technique determined by the technique determination function 100, together with the technique determination function 100. Referring to FIG. 2, the evaluation function 200 includes a technique acquisition unit 201, a pitch acquisition unit 203, a volume acquisition unit 205, a reference data acquisition unit 207, a comparison unit 209, and an evaluation unit 211.

  The technique acquiring unit 201 acquires data indicating the technique of the singing sound determined by the technique determining unit 111 in the technique determining function 100, and outputs the data to the comparing unit 209. The pitch acquisition unit 203 acquires data indicating the pitch detected by the pitch detection unit 105 in the technique determination function 100 in a time series, and outputs the data to the comparison unit 209. The volume acquisition unit 205 acquires data indicating the volume of the singing sound detected by the volume detection unit 107 in the technique determination function 100 in a time series, and outputs the data to the comparison unit. The reference data acquisition unit 207 reads and acquires the corresponding singing sound evaluation reference data 13d stored in the storage unit 13 and outputs the data to the comparison unit 209. It should be noted that the evaluation reference sound data 13d only needs to indicate a sound serving as a reference for evaluation, and does not necessarily need to indicate a sound serving as a model of singing.

  The comparing unit 209 compares the acquired data indicating the pitch of the singing sound, the data indicating the volume of the singing sound, and the data indicating the technique of the singing sound with the corresponding singing sound evaluation reference data 13d. The comparing unit 209 may compare the acquired data indicating the pitch of the singing sound and the reference pitch data included in the evaluation reference data 13d in a time series, and may obtain the data indicating the volume of the obtained singing sound and the evaluation reference data 13d. May be compared in chronological order with reference sound volume data, or data indicating the acquired singing sound technique may be compared with reference singing technique data included in the value reference data 13d. For example, the comparison unit 209 relates to techniques such as punching and vibrato, for example, frequency standard deviation, frequency average value, pitch amplitude average value, pitch amplitude standard deviation, and slope of a linear approximation line of pitch amplitude. With respect to, the acquired singing sound technique may be compared with the reference singing technique included in the value criterion data 13d. The comparison unit 209 outputs the comparison result to the evaluation unit 211.

  The evaluation unit 211 calculates an evaluation value serving as an index of singing sound evaluation based on the comparison result output from the comparison unit 209. The evaluation unit 211 evaluates the higher the degree of coincidence between the data indicating the pitch of the singing sound of the singer, the data indicating the volume of the singing sound, and the data indicating the technique of the singing sound, and the corresponding singing sound evaluation reference data 13d. The value is calculated higher, and the higher the degree of mismatch, the lower the evaluation value. In addition, the evaluator 211 may assign a weight to a technique having a high degree of difficulty such as skipping or vibrato when the degree of coincidence between the singing sound of the singer and the singing sound evaluation reference data 13d is high. The evaluation result by the evaluation unit 211 may be displayed on the display unit 17.

(Modification 2)
In the embodiment described above, in the technique determination function 100, the flat portion detection unit 111 determines the flat portion of the feature amount based on the time-series variation of the feature amount (pitch and volume) detected by the feature amount detection unit 105. Detected. In the first embodiment described above, the flatness detection unit 111 determines that the number of frames in which the variation of the feature amount is within a predetermined threshold or a predetermined width is continuous for a predetermined number or more. If the determined time is equal to or longer than the predetermined time, the flat portion detection unit 111 detects the detected frame as a flat portion of the feature amount. Here, the predetermined time determined as a flat portion may be determined by singing voice data (input sound).

  FIG. 13 is a block diagram illustrating a configuration of the technique determining function 100a according to the modification of the first embodiment of the present invention. The technique determining function 100a includes a tempo estimating unit 110 that detects the tempo of the singing voice data (input sound) based on data (volume waveform) indicating the volume detected by the volume detecting unit 107 in time series. The first embodiment of the present invention, except that the detecting unit 111a determines a predetermined time required to determine a flat portion of the feature amount based on the tempo of the singing voice data detected by the tempo estimating unit 110. Is the same as the technique determination function 100 of FIG.

  Tempo estimating section 110 acquires data (volume waveform) indicating the volume of the singing voice data in time series from volume detecting section 107. The tempo estimating unit 110 detects the tempo of the singing voice data (input sound) based on the fluctuation of the volume (the level of the volume). The tempo estimating unit 110 transmits the detected tempo of the singing voice data (input sound) to the flat portion detecting unit 111a. The flat portion detection section 111a determines the length of a predetermined time required for determining a feature value as a flat portion based on the tempo of the singing voice data detected by the tempo estimation section 110.

  As described above, by determining the length of the predetermined time required to determine the feature amount as a flat portion in accordance with the input singing sound in accordance with the singing sound, the accuracy of detecting the flat portion is improved. be able to.

  When the technique determining function 100a has the accompaniment output unit 101, the tempo estimating unit 110 acquires data (volume waveform) indicating the volume of the accompaniment sound in time series from the accompaniment output unit 101, and based on the volume of the accompaniment sound. Then, the tempo of the corresponding music may be detected.

  In the technique determination functions 100 and 100a described above, the sound indicated by the singing voice data acquired by the input sound acquiring unit 103 is not limited to the voice of the singer, and may be a voice synthesized by singing, or a musical instrument sound. It may be. When the sound is a musical instrument sound, it is desirable that the sound is a single sound performance. In the case of a musical instrument sound, there is no concept of a consonant or a vowel, but depending on the playing method, the sound has a tendency similar to that of a singing at the start point of each sound. Therefore, the same determination may be made for instrument sounds.

  Based on the configuration described as an embodiment of the present invention, those skilled in the art may appropriately add, delete, or change the design of components, or add, omit, or change the conditions of the process. As long as the gist of the present invention is provided, it is included in the scope of the present invention.

  In addition, even if it is another operation effect different from the operation effect obtained by the aspect of the above-described embodiment, it is obvious that the operation effect is obvious from the description in this specification or can be easily predicted by those skilled in the art. It is understood that the present invention brings about the present invention.

DESCRIPTION OF SYMBOLS 10 ... Technology determination apparatus, 11 ... Control part, 13 ... Storage part, 15 ... Operation part, 17 ... Display part, 19 ... Communication part, 21 ... Signal processing part, 23 ... Sound input part, 25 ... Sound output part, 100 , 100a: Technique determination function, 101: Accompaniment output unit, 103: Input sound acquisition unit, 105: Feature amount detection unit, 107: Volume detection unit, 109: Pitch detection unit, 110: Tempo detection unit, 111, 111a: Flat Part detection unit, 113 technique determination unit, 200 evaluation function, 201 technique acquisition unit, 203 pitch acquisition unit, 205 volume acquisition unit, 207 reference data acquisition unit, 209 comparison unit, 211 evaluation unit

Claims (7)

An input sound acquisition unit that acquires an input sound;
A feature value detection unit that detects a feature value of the input sound acquired by the input sound acquisition unit in a time-series manner,
A flat portion detection unit that detects a flat portion of the feature value based on the feature value acquired by the feature value detection unit;
In a predetermined period before or after the flat portion of the feature amount, a technique determination unit that determines the technique of the input sound based on the variation of the feature amount,
A technique determining device comprising: The flat portion detector,
Detecting a period in which the time-series variation of the feature amount is equal to or less than a predetermined variation,
The technique determination device according to claim 1, wherein when the period is equal to or longer than a predetermined time, the period is detected as the flat portion. A tempo estimating unit that estimates a tempo of the input sound based on the feature amount;
The technique determining device according to claim 2, wherein the predetermined time is determined according to the tempo. The feature quantity is a pitch or volume techniques determination apparatus according to any one of claims 1 to 3. When a plurality of flat portions are detected by the flat portion detection unit,
The technique determining apparatus according to claim 1, wherein the predetermined period is a period between two flat portions that are adjacent to each other in a time series. Get input sound,
Detecting the feature amount of the input sound in time series,
Based on the feature, a flat portion of the feature is detected,
In a predetermined period before or after the flat portion of the feature amount, determining the technique of the input sound based on the variation of the feature amount,
A technique determination method including: On computer
Get input sound,
Detecting the feature amount of the input sound in time series,
Based on the feature, a flat portion of the feature is detected,
In a predetermined period before or after the flat portion of the feature amount, determining the technique of the input sound based on the variation of the feature amount,
The program to execute.

Images