JPH0515280B2 - - Google Patents

Description

[Detailed description of the invention]

(Industrial Field of Application) The present invention relates to an intonation display device that displays a pitch curve (hereinafter referred to as intonation) of a voice. (Prior art) Foreign language education in Japan, especially foreign language conversation education, is said to be inefficient compared to that in developed countries. If you live in a big city, you will have the opportunity to meet foreigners, hear the foreign language live, and even have foreigners listen to your pronunciation to correct your pronunciation. However, outside of big cities, people rarely have the opportunity to meet foreigners, and even if they can practice with records or tapes, it is difficult to correct their pronunciation on their own. The problem is that with the traditional method of listening and imitating the words, students rely solely on their own pitch sense to determine whether or not the pronunciation is correct, and the students believe they have heard and pronounced the correct pronunciation. Therefore, unless students' pitch sense is trained, they will end up misperceiving incorrect pronunciation as correct pronunciation. Therefore, for effective language practice, it is necessary to use objective physical quantities (specifically, the waveform of the sound) to judge whether pronunciation is correct or not, rather than relying solely on the student's subjective sense of pitch. It is more reasonable to use them together. For this,
If you display the teacher's pronunciation as a waveform on the CRT, leave this waveform as is on the CRT, and display the student's pronunciation waveform with a different screen each time the student pronounces it.
You can objectively judge whether your pronunciation is correct or incorrect by looking at the waveform, making it extremely effective for language practice. Conventionally, Pitch extractor SE-01 was used as a device to display the intonation of pronunciation as described above.
(Product name: manufactured by Rion Co., Ltd.). This intonation display device analyzes the frequency of voice input from a microphone or line input using a multi-channel bandpass filter, and displays the intonation in real time on a CRT. The range is approximately 50Hz to 300Hz for men;
For women, it is possible to switch between approximately 50Hz and 600Hz. In addition, the intonations of two people pronounced at different times can be displayed above and below the same CRT, and the screen on the CRT can be copied using a printer if necessary. It's summery. (Problem to be Solved by the Invention) However, in the above-mentioned conventional and inonation display devices, the interval between the center frequencies between each bandpass filter is wide, so vowels with even slight intonation are continuously produced. Then, the intonation displayed on the CRT will not be a continuous curve, but will be a discontinuous curve, making it difficult to distinguish the intonation. Furthermore, when a consonant accompanied by a plosive or a fricative is pronounced, the pitch appears to jump around the intonation, making it difficult to distinguish the intonation.
The cause of this is that the pitch extracted from the audio is
This is probably because it is displayed on a CRT. FIG. 21 shows an example of intonation displayed by the above-mentioned conventional intonation display device.
"take it in, I'm sure he'll see me." In addition, in FIG. 21, in order to make the pitch curve itself even clearer, ambiguous parts are added to the pitch curve and displayed thicker while being confirmed using a sound spectrograph. As is clear from this display example, it is actually displayed without any additions, so
It is extremely difficult for an amateur student to discover the pitch curve hidden on the screen from such a display.
For this reason, users tend to be experts, and the product is extremely expensive, making it less popular. SUMMARY OF THE INVENTION An object of the present invention is to provide an intonation display device which can display intonation clearly and is suitably constructed at low cost, solving the various problems mentioned above. (Means for Solving the Problems) The intonation display device of the present invention has a plurality of filters each having a different center frequency, and has a multi-channel display in which the frequency components of an audio input signal are divided by each filter and output. a bandpass filter; an A/D converter that converts analog output from the bandpass filter into a digital signal;
The output of this A/D converter is sequentially sampled, the maximum value and the channel having the maximum value are detected based on the sampled data, and the maximum value and the channel having the maximum value are detected. By performing the necessary calculations on the band filter, the fundamental frequency, or pitch, of the sampled data is detected by increasing the resolution of the band filter. a central processing unit that performs the calculation to determine whether or not the pitches detected in the sampling are irregular and extracts regular pitches; The present invention is characterized in that it includes a display device that displays the extracted pitch strings in chronological order and simultaneously displays a plurality of different pitch strings in different colors. (Operation) In the above configuration, the central processing unit performs necessary calculations based on the maximum value in sequential sampling, the channel having the maximum value, and the data of channels in the vicinity of the channel, so that the central processing unit can perform the necessary calculations on the hardware. Even if the resolution of the bandpass filter is low, it is possible to obtain the fundamental frequency, or pitch, with high resolution, and this allows the intonation to be displayed continuously by pronouncing vowels with intonation. . In addition, by extracting regular pitches based on the pitches obtained in this sampling and the pitches obtained in previous samplings, it is possible to Pitches that appear to jump around the intonation can be removed, the intonation can be displayed accurately and clearly as a whole, and different pitch rows can be color coded on the same display device. and can be displayed at the same time.
Moreover, a commercially available microcomputer can be used as the central processing unit, and an intonation display device can be constructed simply by connecting a bandpass filter and an A/D converter to this central processing unit, making it possible to reduce the cost. (Embodiment) FIG. 1 is a block diagram showing the structure of an example of an intonation display device of the present invention. In this example,
Line input terminal 1 and two microphone input terminals 2
a and 2b are selected by a changeover switch 3, and the audio signal from the selected input terminal is supplied to a bandpass filter (BPF) 4. BPF4 is the first
It has the center frequency shown in the table, and the passband is 1/3 each.
12 with 12 filters set in octaves
The audio signal is divided into 12 channels and supplied to the A/D converter 5. Figure 2 shows the measured characteristic curve of this BPF4.

【table】

[Processing A]

Y1=8 {MCH+(MCH+1)D/2MCHD+
OFFSET} [Processing B] Y1=8 {MCH−(MCH−1)D/2MCHD+
OFFSET}. In addition, in the above processing A and processing B, MCH is the channel with the maximum value,
MCHD indicates the value of that channel. Also,
OFFSET indicates the intonation position displayed on the CRT9. In this way, if the above processing A and processing B are selectively performed based on the value of D of each channel at a certain point in time, high resolution can be obtained even if the resolution on the hardware is low. For example, 1 channel ~
12 channels (each channel is 1/3 octave)
When using this data, a resolution of 100 channels (resolution 7Hz) can be obtained. FIG. 6 shows the channel relationship obtained as a result of performing the above processing A and B using the data of channels 1 to 12 for the frequency of 50 to 1000 Hz taken in from the microphone. Processing for irregular pitches When consonants with plosives, fricatives, etc. are pronounced, irregular raw pitches occur, and the pitches scatter around the intonation curve, making it difficult to distinguish intonation. . Therefore, in this example, the following processing is performed based on the pitches obtained by the above processes A and B, so that irregular pitches are not displayed. In other words, the pitch obtained by processing A or processing B at a certain point in time is Y1, and the six consecutive pitches obtained in the same way immediately before Y1 are
When Y0 is the average value of the seven values from Y2 to Y7, only Y1 that satisfies |Y1−Y0|<L is displayed, and Y1 with other values are displayed.
will not be displayed. In this example, the above L
Set the value to 6. By removing the display of irregular pitches in this manner, intonation curves can be easily identified. The operation of this embodiment will be explained below. FIG. 7 is a flowchart showing the overall operation of this embodiment. First, load the program stored in the FD 8 into the CPU 6, and press the "RUN" key on the keyboard 7 to start the program.
When the program is started, the CPT 9 first displays "Is the teacher a man or a woman?". Here, if the teacher is male, press the ``1'' key, if the teacher is female, press a key other than ``1'' to specify the gender. Next, the time interval for displaying intonation is displayed on the CRT 9. In this example, we set this time interval to 2.5
seconds, 4.5 seconds and 10 seconds, and by the "1" key
Specify a time interval of 2.5 seconds, 4.5 seconds with the "2" key, and 10 seconds with the "3" key, sample the audio signal 500 times at each time interval, and display the intonation. do. After the above processing, audio from the microphone or line is captured. This audio capture is performed according to the flowchart shown in Figure 8, and as a result, the BPF
The 12-bit data of each channel converted into a digital signal by the A/D converter 5 divided by 4 is sampled, the lowest 1 bit is discarded, and logarithmically compressed to 8-bit data is sent to the CPU.
6 memory. Here, the conversion speed in the A/D converter 5 is set to 1 KHz based on the designated intonation display time interval, for example, when the display time interval is 2.5 seconds. After the above processing is completed for the sampled 12 channel data, the logarithmically compressed 12
Based on the 8-bit data of the channel, processing for increasing the resolution is performed according to the flowchart shown in FIG. In this process, first the data of each channel that has been logarithmically compressed is taken in, and the reduction is amplified by calculating D=2×(data of each channel that is logarithmically compressed)/number of channels, and the calculation result is D Processing to remove noise is performed by determining whether or not D<8 and removing data where D<8. These reduction amplification processing and noise removal processing are performed on the number of channels corresponding to the specified gender, in this example, from 1 channel to 9 channels for men, and from 1 channel to 12 channels for women, and the maximum value is reached. The number of channels with , and its value D are saved. However, in the case of a male, the detection of the maximum value is performed from channel 1 to channel 8. When the above processing for the set channels is completed, determine whether all DNs of channels 1 to 8 for men and channels 1 to 12 for women are 0, and if they are not 0, the saved maximum Based on the number of channels with a value and their values, and the number of channels before and after them and their values, the following processing to increase the resolution of BRF4 is performed. In other words, this example uses data from channels 1 to 9 for men, and from channels 1 to 12 for women, so channel 1 has the maximum value for men. In this case, process A, the N channel among channels 2 to 8 has the maximum value, and the values of the channels before and after it D(N-1) and D(N+1) are D(N-1) ≥ D(N+1). When D(N-1)<D(N+1), process A is performed.For women, when channel 1 has the maximum value, process A, and channel 12 has the maximum value. In case of processing B, the N channel among channels 2 to 11 has the maximum value, and the values D(N-1) and D(N+1) of the channels before and after it are D(N-1) ≥ (N+1). If so, process B is performed, and if D(N-1)<D(N+1), process A is performed. When the processing to increase the resolution above is completed, or the pitch Y1 obtained in the relevant sampling and its Y1
The average value of the previous seven consecutive pitches including
Y0, the irregular pitches are processed according to the flowchart shown in FIG.
Only Y1 that satisfies |Y1−Y0|<6 is displayed on the CRT9, and Y1 in the current sampling and Y2 to Y7 in the previous sequential samplings are displayed on the CRT9.
Find the average value Y0 of the seven pitches. On the other hand, in Figure 9, when the values DN of each channel in the predetermined range according to gender are all 0, BPF4
The average value Y0 shown in FIG.
Perform the process to find . 500 within the specified display time interval
Repeat several times to capture the teacher's voice from the microphone or line and record the intonation on the CRT9.
For example, it is displayed in red at the top of the page. In addition, in the above processing, processing for irregular pitches is performed from the 7th sampling when data from Y1 to Y7 is complete in this example, and all pitches obtained in processing A or processing B are displayed in the sampling up to that point. do. Next, for the students who pronounce after the teacher,
Displays "Are you a man or a woman?" on the CRT9. Here, if the student is male, press the "1" key, and if the student is female, press a key other than "1" to specify the gender. After that, the students listen to the teacher's pronunciation with their ears, imitate the teacher's intonation on the CRT9, and perform the same process as that for the teacher described above. The audio is captured and its intonation is displayed in green, for example, at the bottom of the CRT 9. Note that the intonation display time interval at this time is the same as that for the teacher. After displaying the student's intonation, CRT9 displays "Do you want to repeat?" Here, if the same pronunciation is to be practiced repeatedly, by pressing the "Y" key, only the green intonation of the student is deleted so that the student can practice the same pronunciation again. Also, if you want to practice other pronunciations, press "N".
By pressing the key, the teacher's and student's intonations are erased and the initial state is restored. An example of a program that executes the above operations is shown below. Note that in the following programs, the programs written in Basic are written in assembler and use programs running in machine language as subroutines. Examples of intonation display according to this embodiment are shown in FIGS. 11 to 20 below. In addition, in FIGS. 11 to 20, the horizontal axis is time (seconds),
The time from the left edge to the right edge is all set to 2.5 seconds. The vertical axis is the frequency of the audio, and the upper channel is set to the teacher's channel, and the lower channel is set to the student's channel.On the CRT9, the teacher's intonation is red, the student's intonation is green, and the frame is is displayed in yellow. Figure 11 shows "1st" recorded on a cassette tape.
A man's English pronunciation of "day how to explain places and locations" is repeated twice on a cassette tape, and the intonation is displayed on the teacher and student channels. 11th
As is clear from the figure, even if the pronunciation is the same on the cassette tape, there are slight differences in the intonation displayed on the CRT9 due to differences in the timing of audio sampling. The intonation is displayed much more clearly than in . Figure 12 shows a man's English pronunciation of ``Simple greeting'' recorded on a cassette tape, the cassette tape repeated twice, and the intonation similarly displayed on the teacher and student channels. . In this display example,
Although the plosive part (ting) in the greeting is slightly distorted, the intonation can be recognized relatively clearly. Figure 13 shows a woman's English pronunciation of ``There's hardly a cloud in the sky'' recorded on a cassette tape, which is also repeated twice on the cassette tape and the intonation displayed on the teacher and student channels. The intonation is clearly displayed. Figure 14 shows a woman's English pronunciation of ``His business is good this year'', which was also recorded on a cassette tape, with the cassette tape repeated twice and the intonation displayed on the teacher and student channels. It is. The pronunciation in this case includes the plosive part of business (bus) and this.
There are fricative sounds, but these do not scatter,
clearly displayed. Figure 15 shows a man's English pronunciation of "Did you pay very much?" which was also recorded on a cassette tape, with the cassette tape repeated twice and the intonation displayed on the teacher and student channels. , the intonation is clearly displayed without the plosive sounds of pay. Figure 16 shows a man's Japanese pronunciation of ``Today is a nice weather'' from a microphone at normal and rapid speed, and the intonation of each pronunciation is displayed on the teacher and student channels. be. In general, it is said that it is difficult to display male intonation, but in this example, in the above pronunciation, the parts of "weathering" when pronounced at normal speed are only scattered, In either case, the intonation can be relatively clearly identified. Figure 17 shows a woman's Japanese pronunciation of ``Today is a nice weather'' from a microphone at a normal speed.
The intonation is displayed repeatedly on the teacher and student channels. This woman's pronunciation lacks intonation, but the intonation is clearly displayed. Figure 18 shows the Japanese pronunciation of ``Today is a sunny day'' by a man from a microphone at normal speed and fast speaking, and the intonation of each pronunciation is displayed on the teacher and student channels. be. Perhaps because the male voice in this case is a little high-pitched, the intonation is clearly displayed whether it is pronounced at normal speed or fast. Figure 19 shows a woman's Japanese pronunciation of ``Today is a sunny day'' from a microphone, repeated twice at normal speed, and the intonation displayed on the teacher's and student's channels. in this case,
"Sunny weather" with continuous fricatives and plosive sounds
Although the display of the part ``because of'' is unclear, the intonation can be relatively clearly identified. Figure 20 shows an example of a display in which the man who made the pronunciation in Figure 16 produced the pronunciation of "a" by varying the pitch from the microphone, and the changes in pitch are clearly displayed. . As is clear from the above display examples, according to this embodiment, even if there are plosives or fricatives, the intonation can always be displayed clearly so that the intonation can be clearly identified. Note that the present invention is not limited only to the embodiments described above, and numerous modifications and changes are possible. For example, the number of channels in BPF4 is not limited to 12 channels, and may be greater or less than that. Furthermore, the channel data to be used may not be uniquely set based on gender, but may be arbitrarily selected. Furthermore, A/D
Converters are not limited to 12 bits, but 8 bits,
16 bits etc. can be used, and
The A/D converted data can also be processed as is without being logarithmically compressed. Furthermore,
In the above embodiment, the audio frequency is reduced and enhanced, but this processing can be omitted if the overall characteristics of the audio input stage are of the reduction and enhancement type, and the overall characteristics of the audio input stage are of the low frequency range. If the signal is extremely amplified, high frequency enhancement processing may be performed. This high frequency enhancement processing is performed by, for example, 2×D/
This can be done by {(M+1)-N}.
Here, M is the final number of channels, which is 12 in the case of the embodiment described above, and N is the number of channels of data D. In addition, in the above embodiment, when increasing the resolution of BPF4, in the case of a male, the maximum value among channels 1 to 8 is detected using data from channels 1 to 9, and channel 1 detects the maximum value. When channel N out of channels 2 to 8 has the maximum value, process A or process A depending on the magnitude relationship of the values D(N-1) and D(N+1) of the channels before and after it. However, when the final channel of the designated channels to be used has the maximum value, it is also possible to directly perform process B without comparing the magnitude relationship of the channels before and after it. Furthermore, in the embodiment described above, processing for irregular pitches is performed from the 7th sampling onward;
In the sixth sampling, the average value of the pitch in the previous samplings can be obtained, however, in the second sampling, the pitch in the first sampling can be obtained and similar processing can be performed. (Effects of the Invention) As described above, according to the present invention, intonation can always be clearly displayed, and a plurality of different pitch rows can be displayed simultaneously on the same display device in different colors.
Moreover, it can be constructed easily and inexpensively by simply connecting the BPF and A/D converter to a commercially available microcomputer. Therefore, it can be easily used even by junior high and high school students, and can be widely used for language practice at school or at home.

[Brief explanation of the drawing]

FIG. 1 is a block diagram showing the configuration of an example of the intonation display device of the present invention, FIG. 2 is a diagram showing the characteristics of the BPF shown in FIG. 1, and FIG. 3 is a diagram showing the overall characteristics of the audio input stage of the BPF. Diagrams shown corresponding to each channel, Figure 4 is a diagram to explain low frequency enhancement processing, Figure 5 is a diagram showing how noise is generated, and Figure 6 is a diagram showing vibrations obtained by processing to increase resolution. 7 to 10 are flowcharts for explaining an example of the operation of the intonation display device shown in FIG. FIG. 21 is a diagram showing an example of display by the intonation display device shown in the figure, and FIG. 21 is a diagram showing an example of display by the conventional intonation display device. 1... Line input terminal, 2a, 2b... Microphone input terminal, 3... Selector switch, 4... BPF (band pass filter), 5... A/D converter,
6...CPU (Central Processing Unit), 7...Keyboard,
8...FD (floppy disk), 9...CRT (display device).

Claims (1)

[Claims] 1. A multichannel bandpass filter that has a plurality of filters each having a different center frequency and that divides and outputs the frequency component of an audio input signal by each filter, and an analog output from this bandpass filter. an A/D converter that converts the output into a digital signal, and sequentially samples the output of this A/D converter, detects the maximum value and the channel having the maximum value based on the sampled data,
By performing necessary calculations based on these maximum values, the channel, and the data of channels in the vicinity of the channel, the resolution of the bandpass filter is increased to detect the fundamental frequency, that is, the pitch, of the sampled data, and Based on the detected pitch and the pitch detected in the sampling before the sampling, necessary calculations are performed to determine whether the pitch detected in the sampling is irregular or not, and the pitch is determined to be regular. The present invention includes a central processing unit for extraction, and a display device that displays pitch columns selectively extracted from the central processing unit in sequential sampling in chronological order, and simultaneously displays a plurality of different pitch columns in different colors. An intonation display device characterized by the ability to