JP2000250566A - Sound and soundless deciding device and speech rate converting device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a sound/soundless deciding device capable of improving sound/soundless decisive precision by using movement of an image. SOLUTION: This device is the sound/soundless deciding device provided with a means making a video signal and a voice signal synthesized with the video signal an input and calculating the sound/soundless deciding data from the input voice signal and a decision means deciding whether an input voice is a sound section or a soundless section by comparing the obtained sound/ soundless deciding data with a sound/soundless deciding threshold value. In such a case, this device is provided with a movement detection means 1 detecting the movement of the image based on an input video signal and a threshold value control means 2 changing the sound/soundless deciding threshold value based on the detection result of the movement detection means 1.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a sound / non-speech determining device and a speech speed converting device.

[0002]

2. Description of the Related Art During high-speed reproduction of a VTR, of audio signals read from a video tape, an audio signal in a silent section is deleted or compressed on the time axis, and an audio signal in a sound section is expanded on the time axis. By doing so, a speech speed conversion device that outputs voice in a sound section at a speed lower than the playback speed is known.

[0003] Whether an audio signal read from a video tape is a silent section or a sound section is determined, for example, by comparing the audio signal power with a preset threshold value. That is, a section in which the audio signal power is equal to or greater than the threshold is determined as a sound section, and an interval in which the audio signal power is smaller than the threshold is determined as a silent section.

However, in this method, when there is a large background noise or when BGM is present, a section which should be a silent section may be determined as a sound section. Also,
A portion having a small voice line may be determined to be a silent section.

[0005]

In the case of programs such as dramas and news, it is important to output the voices of the performers. When the performer is talking, the performer's mouth moves. In a golf program, a tennis program, and the like, it is important to output a swing sound. In a scene where a swing sound is generated, a golf club, a racket, or the like is moving.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a sound / silence determination device which can improve the sound / silence determination accuracy by utilizing the movement of an image.

Further, the present invention provides a sound / voice having high determination accuracy.
An object of the present invention is to provide a speech speed conversion device provided with a silent section determination unit.

[0008]

A sound / non-speech determining apparatus according to the present invention receives a video signal and an audio signal synchronized with the video signal, and calculates voice / non-speech determining data from the input audio signal. By comparing the obtained voice / non-speech determination data with a voice / non-speech determination threshold, the voice / non-speech determination unit determines whether the input voice is in a voice section or a non-voice section. In the silence determination device, a motion detection unit that detects a motion of an image based on an input video signal, and a sound / noise detection unit based on a detection result of the motion detection unit.
It is characterized by comprising a threshold control means for changing a silence determination threshold.

As the motion detecting means, for example, a means for detecting a motion vector for each of a plurality of regions set in one screen is used. As the threshold control means, for example, the maximum value of the motion vector is equal to or more than a predetermined value. The threshold value for voice / non-speech determination is controlled so that the threshold value for voice / non-speech determination is reduced when the value is, and the threshold value for voice / non-voice determination is increased when the maximum value of the motion vector is smaller than a predetermined value. Is used.

The sound / non-speech determination data includes, for example, an average power value of the input audio signal for each predetermined period unit, a cumulative power value of the input audio signal for each predetermined period unit, and an input audio signal value for each predetermined period unit. The average amplitude value, the cumulative amplitude value of the input audio signal for each predetermined period unit, and the like are used.

According to the speech speed conversion device of the present invention, in a speech speed conversion device which receives a video signal and an audio signal synchronized with the video signal as input, and converts the input audio signal into a speech speed, whether the input audio signal is a voiced section or not. Section discriminating means for discriminating whether a section is a silent section, expanding means for expanding an input audio signal on a time axis in a sound section, and compressing or deleting the input audio signal on a time axis in a silent section A section discriminating means for calculating speech / silence determination data from the input voice signal, and comparing the obtained speech / silence determination data with a speech / silence determination threshold. By this means, a determination means for determining whether the input sound is a sound section or a silent section, a motion detection means for detecting a motion of an image based on an input video signal, and a detection result of the motion detection means Zui it, characterized in that it comprises a threshold control means for changing a voice or silence determination threshold.

As the motion detecting means, for example, a means for detecting a motion vector for each of a plurality of areas set in one screen is used. As the threshold control means, for example, the maximum value of the motion vector is equal to or more than a predetermined value. The threshold value for voice / non-speech determination is controlled so that the threshold value for voice / non-speech determination is reduced when the value is, and the threshold value for voice / non-voice determination is increased when the maximum value of the motion vector is smaller than a predetermined value. Is used.

The sound / non-speech determination data includes, for example, the average power of the input audio signal for each predetermined period unit, the cumulative power value of the input audio signal for each predetermined period unit, and the input audio signal for each predetermined period unit. The average amplitude value, the cumulative amplitude value of the input audio signal for each predetermined period unit, and the like are used.

[0014]

BEST MODE FOR CARRYING OUT THE INVENTION

An embodiment of the present invention will be described below with reference to the drawings.

FIG. 1 shows a configuration of a speech speed conversion device.

Here, a case will be described in which the audio signal in the audio section is output at a speed lower than the reproduction output in the double-speed reproduction during the double-speed reproduction of the VTR, and the audio signal in the silent section is deleted.

A video signal read from the VTR at a double speed reproduction speed is sent to a motion vector detecting section 1. As is well known, the motion vector detecting section 1 generates data for detecting a motion vector based on a representative point matching method.

The representative point matching method will be briefly described. As shown in FIG. 2, the video area 10 of each frame
A plurality of motion vector detection areas A0 to E7 are set in 0. The sizes of the motion vector detection areas A0 to E7 are the same. Also, each motion vector detection area A0-E
7 is further divided into a plurality of small areas e as shown in FIG. Then, as shown in FIG. 4, a plurality of sampling points S and one representative point R are set in each small area e.

The difference (correlation value at each sampling point) between the video signal level of the sampling point S in each small area e in the current frame and the video signal level of the representative point R of the corresponding small area e in the previous frame is It is obtained for each of the motion vector detection areas A0 to E7. Then, for each of the motion vector detection areas A0 to E7, correlation values of sampling points having the same deviation with respect to the representative point R are cumulatively added among all the small areas in the motion vector detection area. Therefore, for each of the motion vector detection areas A0 to E7, the number of correlation accumulated values corresponding to the number of sampling points in one small area e is obtained.

In each of the motion vector detection areas A0 to E7, the deviation of the point having the smallest accumulated correlation value, that is, the deviation of the point having the highest correlation is determined by the motion vector detection area A0 to E7.
The motion vectors are extracted as motion vectors 0 to E7 (movements of the subject).

The data generated by the motion vector detecting section 1 is sent to a voice / silence determination threshold control section 2 for determining a voice / silence determination threshold. The threshold for motion determination set by the threshold setting unit for motion determination 7 is also input to the threshold control unit 2 for sound / non-speech determination. The motion determination threshold setting unit 7 stores a motion determination threshold corresponding to the program mode, and sets the motion determination threshold corresponding to the program mode input by the user to the sound / non-sound determination threshold control unit 2. Sent to

Table 1 shows an example of the relative value of the motion determination threshold for each program mode.

[0024]

[Table 1]

For example, comparing the relative value of the threshold for motion determination with respect to a news program and the relative value of the threshold for motion determination with respect to a sports program (golf program), the relative value of the threshold for motion determination with respect to sports is set to be large. .
In the case of a news program, the presence / absence of the movement of the mouth of the caster is an important factor for the sound / non-sound determination. In the case of a golf program, the presence / absence of the movement of the golf club is an important factor for the sound / silence determination. The motion vector for the movement of the mouth is small, and the motion vector for the movement of the golf club is large. Therefore, in the case of a news program, the threshold value for motion determination for enabling the presence or absence of the movement of the mouth of the caster to be determined is set to a small value, and in the case of a golf program, small movements such as the movement of the mouth are ignored. Thus, the movement determination threshold is set to a large value so that the presence or absence of a large movement such as the movement of the golf club can be determined.

The sound / non-speech determination threshold control unit 2 compares the maximum value of the motion vectors detected in the respective motion vector detection areas A0 to E7 with the motion determination threshold. If the maximum value of the motion vectors detected in the respective motion vector detection areas A0 to E7 is equal to or larger than the threshold for motion determination, the threshold for voice / non-speech determination is reduced, and the maximum value of the motion vector If it is smaller than the judgment threshold,
Increase the silence determination threshold.

An input audio signal read from the VTR at a double speed reproduction speed is sent to a power calculator 3. The power calculator 3 calculates an average power value of the audio signal for a predetermined period unit, for example, one frame. The average power value calculated by the power calculation unit 3 is sent to the sound / non-sound section determination unit 4.

The voice / non-speech interval determination unit 4 calculates the average power value sent from the power calculation unit 3 and the voice / non-voice determination threshold value determined by the voice / non-voice determination threshold control unit 2. Be compared. If the average power value is equal to or greater than the voiced / silence determination threshold, the current frame is determined to be a voice section. If the average power value is smaller than the voice / silence determination threshold, the current frame is determined to be a silent section.

The speech speed control unit 5 includes an input voice signal and
The determination result of the silent section determination unit 4 is sent. The speech speed control unit 5 includes a silence removing unit 51 and a time axis extending unit 52. The time axis decompression unit 52 performs time axis decompression processing so that the audio signal determined to be a sound section is output at a speed lower than the reproduction output at the time of double speed reproduction. The audio signal determined to be a silent section is subjected to time axis compression processing by the silent section 51.

The audio signal of the sound section subjected to the time axis expansion processing by the time axis expansion section 52 is temporarily stored in the audio memory 6. The audio signals stored in the audio memory 6 are sequentially read and output.

In the above-described embodiment, the average power value of the audio signal for the predetermined period unit is calculated by the power calculation unit 3. However, the power accumulated value of the input audio signal for each predetermined period unit, Alternatively, the average amplitude value of the input audio signal or the cumulative amplitude value of the input audio signal for each predetermined period unit may be calculated.

In the above embodiment, the audio signal determined to be a silent section is deleted by the silent deletion section 51. However, the audio signal determined to be the silent section is subjected to the time axis compression processing. May be performed. In this case, the audio signal of the silent section subjected to the time axis compression processing is also temporarily stored in the audio memory 6.

In the above embodiment, when the maximum value of the motion vectors detected in the respective motion vector detection areas A0 to E7 is equal to or larger than the threshold for motion determination, the threshold for voice / non-voice determination is reduced. When the maximum value of the motion vector is smaller than the threshold for motion determination, the threshold for voice / non-voice determination is increased.

Therefore, for example, in a program such as a drama or a news, when a performer is speaking, the performer's mouth moves, so that the threshold for sound / non-speech determination is reduced. As a result, a section in which the performer is speaking in a low voice is more likely to be determined as a sound section, and the performer's voice is more easily heard.

In a golf program, a tennis program, and the like, when a swing sound is generated, a golf club, a racket, and the like move, so that the threshold for sound / non-sound determination is reduced. As a result, the section where the swing sound is generated is more likely to be determined as a sound section, and the swing sound is more easily heard.

[0036]

According to the present invention, a voiced / silent discriminating apparatus which can improve the voiced / silent discrimination accuracy by utilizing the motion of an image is realized. Further, according to the present invention, a speech speed conversion device provided with a voiced / silent section determination unit having high determination accuracy is realized.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration of a speech speed conversion device.

FIG. 2 is a schematic diagram showing a plurality of motion vector detection areas set in a video area of each frame.

FIG. 3 is a schematic diagram showing a small area e set in each motion vector detection area.

FIG. 4 is a schematic diagram showing a plurality of sampling points S and one representative point R set in each small area e.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Motion vector detection part 2 Threshold control part for sound / non-speech determination 3 Power calculation part 4 Speech / non-speech section judgment part 5 Speech speed control part 6 Voice memory 7 Threshold setting part for motion judgment 51 Silence deletion part 52 Time axis extension Department

Claims (6)

[Claims] 1. A means for receiving a video signal and an audio signal synchronized with the video signal as input, and calculating voice / non-speech determination data from the input voice signal, and converting the obtained voice / non-speech data into voice / non-speech. A sound / silence determination device having a determination unit that determines whether an input voice is a voiced section or a voiceless section by comparing with a determination threshold value. A sound / non-speech determining device, comprising: a motion detecting means for detecting; and a threshold control means for changing a sound / no-sound determining threshold based on a detection result of the motion detecting means. 2. The motion detecting means detects a motion vector for each of a plurality of areas set in one screen, and the threshold control means is provided when the maximum value of the motion vector is equal to or more than a predetermined value. And controlling the voice / silence determination threshold so that the voice / silence determination threshold is reduced and the voice / non-voice determination threshold is increased when the maximum value of the motion vector is smaller than a predetermined value. Item 1. The sound / non-sound determination device according to item 1. 3. The sound / non-speech determination data includes an average power value of the input audio signal for each predetermined period unit, an accumulated power value of the input audio signal for each predetermined period unit, and an amplitude of the input audio signal for each predetermined period unit. 3. The sound / non-speech determining device according to claim 1, wherein the sound / non-speech determining device is one arbitrarily selected from an average value and an amplitude cumulative value of the input audio signal for each predetermined period unit. 4. A speech speed conversion device which receives a video signal and an audio signal synchronized with the video signal and converts the input audio signal into a speech speed, determines whether the input audio signal is a voiced section or a silent section. Section determining means to perform, in a sound section, expansion means for expanding the input audio signal on the time axis, and in a silent section, means for compressing or deleting the input audio signal on the time axis, The section discriminating means includes calculating means for calculating voice / non-speech determination data from the input voice signal, and comparing the obtained voice / non-speech determination data with a voice / non-speech determination threshold to determine whether the input voice is valid. Determining means for determining whether the section is a sound section or a silent section; a motion detecting means for detecting a motion of an image based on an input video signal; and a sound / silence determining section based on a detection result of the motion detecting means. A speech speed conversion device comprising threshold value control means for changing a threshold value. 5. The motion detecting means detects a motion vector for each of a plurality of areas set in one screen, and the threshold control means is provided when the maximum value of the motion vector is equal to or more than a predetermined value. And controlling the voice / silence determination threshold so that the voice / silence determination threshold is reduced and the voice / non-voice determination threshold is increased when the maximum value of the motion vector is smaller than a predetermined value. Item 5. A speech speed conversion device according to item 4. 6. The sound / non-speech determination data includes an average power value of the input audio signal for each predetermined period unit, an accumulated power value of the input audio signal for each predetermined period unit, and an amplitude of the input audio signal for each predetermined period unit. 6. The speech speed conversion device according to claim 4, wherein the speech speed conversion device is one arbitrarily selected from an average value and an amplitude accumulation value of the input audio signal for each predetermined period unit.