JPH07191599A - Video apparatus - Google Patents

Abstract

PURPOSE:To obtain a video apparatus which enables even a person handicapped in hearing and speaking to understand contents of talking simply by viewing a screen by converting audio signals to character information and displaying the information on the screen. CONSTITUTION:An audio circuit 6 converts the intermediate frequencies of audio to FM detection and demodulation after amplification thereof, converts the frequencies to audio signals and supplies these signals via a character generating means 11 and a speech recognizing means 12 to a finger language forming means 13. The character forming means 11 converts the audio signals to character information and the audio recognizing means 12 recognizes the contents of meanings from the audio signals and converts the contents of the meanings recognized by the finger language image forming means 13 to animation images of the finger language. The images selected by a switching means 14 are so synthesized as to be superimposed on a part of the video signals corresponding to a basic screen by a superimposing display means 10 and supplies the prescribed signals to the electrodes of an image receiving tube 5.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a video device such as a television or a video which is suitable for people who are deaf.

[0002]

2. Description of the Related Art A television, which is a typical example of conventional video equipment of this type, supplies radio waves obtained by a receiving antenna 1 to a tuner 2 as shown in FIG. Channel selection according to 3. The video circuit 4 amplifies and detects the intermediate frequency, distributes the audio intermediate frequency, the color subcarrier, and the synchronization signal, and then amplifies the luminance signal, adds the color difference signal, and outputs a predetermined video signal to the electrode of the picture tube 5. Supply. The audio circuit 6 amplifies the audio intermediate frequency, FM-detects and demodulates it, converts it into an audio signal, and supplies it to the speaker 7. The synchronization / deflection circuit 8 separates the synchronization signal, synchronizes the vertical / horizontal oscillators, creates a sawtooth wave current, and supplies it to the deflection coil 9. Here, the media input unit for inputting the audio signal and the video signal is the receiving antenna 1, the tuner 2,
It is composed of a music selection remote controller 3, a video circuit 4, and an audio circuit 6.

By the way, as a television broadcast for the deaf person, a closed caption broadcast for displaying conversations and announcements in a program as subtitles on the television screen is performed in the United States. In Japan, only a limited number of programs are currently broadcasting subtitles and video with sign language corresponding to audio signals.

Further, in recent studies, voice recognition for unspecified speakers and animation image synthesis of sign language corresponding to semantic information have come to be considered (for example, Xu Army, Makoto Tanahashi, Yuji Sakamoto, Yoshinao Aoki: "Sign language image, hand gesture description and word dictionary construction method for intelligent communication"
76-A, 9, pp. 1332-1341 (1993-
9)).

Further, there is a part of the wired communication system in which contents equivalent to those of a voice service are converted into character information for a deaf person and superimpose-displayed on the screen (for example, JP-A-62-48890). .

[0006]

However, in the above-mentioned conventional configuration, unless the video media provider has previously added information corresponding to audio in advance such as closed caption broadcasting, subtitles, and broadcasting with sign language, etc.
There was a problem that deaf people could not understand the contents of utterances such as conversations and announcements at all.

The present invention is intended to solve the above problems, and an object of the present invention is to provide a video device which enables a deaf person or the like to understand the contents of speech just by looking at the screen.

[0008]

In order to solve the above-mentioned problems, the video equipment of the present invention comprises a media input section for inputting a voice signal and a video signal, and a voice signal received by the media input section converted into character information. And a display unit for displaying the character information generated by the character generation unit on the screen.

Further, a media input section for inputting a voice signal and a video signal, a voice recognizing means for recognizing the meaning content from the voice signal received by the media input section, and a meaning content recognized by the voice recognizing means for sign language. A sign language image generating means for converting into an animation image and a display means for displaying the image information generated by the sign language image generating means on a screen are provided.

Alternatively, a media input unit for inputting a voice signal and a video signal, voice recognition means for recognizing the meaning and content from the voice signal received by the media input unit, and the position of the speaker are identified from the voice signal or the video signal. Position identifying means, a sign language image generating means for converting the meaning content recognized by the voice recognizing means into an animation image of sign language, and the image information generated by the sign language image generating means is specified by the position identifying means. Display means for displaying on the screen corresponding to the position of the person.

Alternatively, a media input unit for inputting a voice signal and a video signal, voice recognition means for recognizing the meaning and content from the voice signal received by the media input unit, and a talk for identifying a speaker from the voice signal or the video signal. Person identification means,
Sign language image generation means for converting the meaning content recognized by the voice recognition means into a sign language animation image, and image information generated by this sign language image generation means for the individual speakers identified by the speaker identification means. And a display means for changing the type of the sign language animation and displaying it on the screen.

[0012]

According to the present invention, the contents of speech such as conversations and announcements in all video media such as television broadcasting are immediately displayed on the screen as an animation image of characters or sign language by the above-mentioned structure.

Particularly, a position specifying means for specifying the position of the speaker from the audio signal or the video signal is provided, and the display position of the animation image of the sign language is changed in accordance with the position of the speaker so that the original video signal is obtained. A realistic and easy-to-understand screen is created.

Alternatively, a speaker identifying means for identifying a speaker from an audio signal or a video signal is provided, and the type of the sign language animation is changed in accordance with each identified speaker, whereby a more realistic and easy-to-understand screen is displayed. Is built.

[0015]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A first embodiment of the present invention will be described below with reference to FIGS. In FIG. 1, it has the same function as that shown in the conventional example, the same number is given, and a part of the description is omitted. In the television, which is a typical example of the video equipment as in the conventional example, the radio wave obtained by the receiving antenna 1 is first supplied to the tuner 2, where it is amplified and frequency-converted to perform tuning according to the tuning remote controller 3. The video circuit 4 amplifies and detects the intermediate frequency, distributes the audio intermediate frequency, the color subcarrier, and the synchronization signal, then amplifies the luminance signal, adds the color difference signal, and superimposes the video signal corresponding to the basic screen. It is supplied to the display means 10.
The voice circuit 6 amplifies the voice intermediate frequency and then performs FM detection and demodulation,
It is converted into a voice signal and supplied to the sign language image generation means 13 via the speaker 7, the character generation means 11 and the voice recognition means 12. Here, the media input unit for inputting the audio signal and the video signal is composed of the receiving antenna 1, the tuner 2, the music selection remote controller 3, the video circuit 4, and the audio circuit 6.

The character generation means 11 converts the voice signal into character information, and the voice recognition means 12 recognizes the meaning content from the voice signal and then converts the meaning content recognized by the sign language image generation means 13 into a sign language animation image. To do. Switching means 14
Is a switching unit that can select either one of the characterized image generated by the character generation unit 11 and the sign language animation image generated by the sign language image generation unit 13 or neither of them. . The image selected by the switching unit 14 is the superimposed display unit 1.
At 0, the video signal is synthesized so as to be superimposed on a part of the video signal corresponding to the basic screen, and then a predetermined video signal is supplied to the electrode of the picture tube 5.

The structure of the character generation means 11 will be described with reference to FIG. The sound analysis unit 11a performs sound analysis on the sound input from the sound circuit 6 at a frame cycle for each unit time to obtain a sound power and an autocorrelation function. Based on this analysis data, the phoneme recognition unit 11b separates the voice input into vowels and consonants, and extracts a phoneme string for each phoneme by matching with a phoneme standard pattern stored in advance in the phoneme rule unit 11c. . The word recognition unit 11d predicts a word sequence using the word knowledge stored in advance in the word dictionary 11e, and extracts a word by matching this with a phoneme string. The syntactic analysis unit 11f further checks and corrects the consistency as a sentence by using the linguistic knowledge stored in advance in the grammar / syntax rule unit 11g. Here, the phoneme rule section 11c and the word dictionary 11e
The grammar / syntax rule part 11g cooperates with each other to form a knowledge database. Kanji / Kana mixed character converter 11
In h, the sentence extracted by the syntax analysis unit 11f is converted into a Japanese character string containing kanji and kana, and the subtitle image generation unit 11i converts this into an image and outputs it.

On the other hand, the structures of the voice recognition means 12 and the sign language image generation means 13 will be described with reference to FIG. Character generation means 11
Similarly, in the acoustic analysis unit 12a, the audio input from the audio circuit 6 is acoustically analyzed in a frame cycle for each unit time, and the audio power and the autocorrelation function are obtained. Based on this analysis data, the phoneme recognition unit 12b separates the voice input into vowels and consonants, and extracts a phoneme string for each phone by matching with the phoneme standard pattern stored in advance in the phoneme rule unit 12c. . The word recognition unit 12d extracts a word by predicting a word sequence using word knowledge stored in advance in the word dictionary 12e and matching this with a phoneme string. The syntactic analysis unit 12f further checks and corrects the consistency of the sentence by using the linguistic knowledge stored in advance in the grammar / syntax rule unit 12g. Here, the phoneme rule unit 12c, the word dictionary 12e, and the grammar / syntax rule unit 12g cooperate with each other to form a knowledge database. In the semantic reasoning 12h, the parsing unit 1
The meaning content corresponding to the sentence extracted in 2f is recognized at the language level, and is transmitted to the sign language image generation means 13 as meaningful sentence information divided for each basic unit.

In the sign language image generating means 13, the sign language description unit 13a extracts a necessary sign language action pattern from the sign language words stored in advance in the sign language word dictionary 13b and synthesizes it, corresponding to the sentence information. The sign language image generation unit 13c converts the sign language operation combined by the sign language description unit 13a into an animation image and outputs it.

In the above structure, the character generating means 12 converts the voice signal into the character information, and the superimposing display means 10 immediately displays the character information on the television screen, so that there is no voice output from the speaker 7. However, the effect is that you can understand the contents only on the TV screen. Alternatively, the voice recognition means 12 and the sign language image generation means 13 convert the voice signal into an animation image of sign language, and the superimposing display means 10 immediately displays this image information on the TV screen, so that it can be read only by the character display. It is possible to convey the necessary information visually and quickly and concisely even in tongue twisters that make it difficult. In particular, even when the screen size of the picture tube 5 is small, there is an effect that it is hard to get tired.

Next, a second embodiment of the present invention will be described with reference to FIGS. 4 is different from the first embodiment of the present invention in that there is no character generation means 11 or switching means 14,
This is because a position specifying means 15 for newly specifying the position of the speaker from the video signal and the audio signal and a superimposing display means 16 for displaying on the screen corresponding to the position of the speaker are newly provided. Here, it is assumed that audio stereo signals are output from the audio circuit 6 independently of the left and right channels.

Since the other construction is the same as that of the first embodiment, its explanation is omitted and only the construction of the position specifying means 15 will be explained with reference to FIG. The speech signal extraction unit 15a is a filter that extracts only the speech signal from the speech input from the speech circuit 6, detects the volume of the speech signal that has passed through the speech signal extraction unit 15a by the volume detection unit 15b, and the direction detection unit 15c. The direction of the speaker in the sound field is calculated by the left-right volume balance and its change. On the other hand, the human body position detecting unit 15d detects the human body position from the two-dimensional screen by image-processing the video input from the video circuit 4, and further the lip motion detecting unit 15e.
The position of the speaker on the screen is calculated by detecting the lip movement in the human body detected by. In the position determination unit 15f, the volume detection unit 15b, the orientation detection unit 15c, and the lip movement detection unit 15e.
The position of the speaker in the virtual three-dimensional space is estimated based on the input from, and this position information is transmitted to the superimposing display means 16.

The superimpose display means 16 changes the display position of the animation image of sign language to be superimposed on the screen according to the position information of the speaker specified by the position specifying means 15. The perspective is expressed by the size of the animation image in the superposed two-dimensional display area.

In the above configuration, the display position of the animation image of sign language is instantly changed corresponding to the position of the speaker specified by the position specifying means 15. For example, judging from the audio signal, when the speaker in the right hand starts talking, the sign language animation appears on the right side of the screen, and the sign language operation according to the content of the talk starts. If the speaker moves to the left while talking, the sign language animation also moves to the left on the screen. The same applies to the vertical direction or the depth direction. In other words, the position information of the speaker, which is lacking just by replacing the voice content with the sign language, can be included in the screen with a realistic feeling.

Next, a third embodiment of the present invention will be described with reference to FIGS. 6 is different from the second embodiment of the present invention in that the position specifying means 15 for specifying the position of the speaker is used.
In other words, there is newly provided speaker identifying means 17 for identifying a speaker from a video signal and display means 18 for changing the type of the sign language animation corresponding to the speaker and displaying it on the screen. The speaker identifying means 17 is configured to extract all persons from the video signal input from the receiving circuit 4 and identify the speakers from these persons to change the type of animation.

The structure of the speaker identifying means 17 will be described with reference to FIG. The speech signal extraction unit 17a is a filter that extracts only the speech signal from the speech input from the speech circuit 6, and for the speech signal that has passed through the speech signal extraction unit 17a, the volume detection unit 17b, the pitch detection unit 17c, the tone color detection unit 17d, and The speed detecting unit 17e detects the volume, pitch, tone color and speech speed of the speaker, respectively, of the speech signal. On the other hand, the human body shape detection unit 17f detects the human body shape from the two-dimensional screen by image-processing the video input from the video circuit 4, and the speaker obtained from the image by the lip motion detection unit 17g and the feature extraction unit 17h. The feature amount of is calculated. Speaker determination unit 17i
Then, these volume detecting section 17b, pitch detecting section 17c, timbre detecting section 17d, speed detecting section 17e, human body shape detecting section 17
Based on the output from f and the lip motion detection unit 17g, the speaker feature amount is classified into, for example, one of 20 patterns prepared in advance, and the classified information is transmitted to the superimpose display unit 18. The classification method uses a multivariate analysis method such as principal component analysis or a neural network method such as learning vector quantization, and realizes by drawing the individual feature vector in a multidimensional space. The superimpose display means 18 is configured to immediately switch the type of the sign language animation to be superposed and displayed corresponding to each speaker identified by the speaker identification means 17.

In the above configuration, the type of sign language animation is instantly changed corresponding to each speaker identified by the speaker identifying means 17. In other words, it is possible to incorporate the features of individual speakers, which would otherwise be lacking just by replacing the voice content into sign language, into the screen.

Although the first to third embodiments have been described using the television as an example, the present invention can be applied to any video equipment that outputs an audio signal and a video signal. It does not depend on the type of audio signal or video signal. As the display method on the screen, the superimposing display means 10, 16 and 18 are used, but they are divided into sub-screens and independently displayed, or newly generated character information and sign language animation images are displayed separately. It may be configured to be displayed on the screen. The screen does not need to use the picture tube 5 such as a CRT. Further, the display screen may be connected by a wired or wireless communication line and may be installed at a place apart from other components.

Further, the language targeted by the character generation means 11 is not limited to Japanese. The input signal to the position specifying means 15 or the speaker identifying means 17 may not be a combination of both audio signals and video signals. The types of speakers identified by the speaker identifying means 17 may be only two types such as sex. Conversely, the size, shape, facial expression, action, type, etc. of the animation may be adjusted steplessly according to the feature amount of the speaker. The feature amount of the voice may be converted into color information such as color and brightness to change the sign language animation expression.

[0030]

As described above, according to the video equipment of the present invention, the following effects can be obtained.

(1) For a deaf person or a person who wants to know the video content without voice, the utterance content can be understood only by looking at the screen. It can also be used as is for the vast amount of video media that has been accumulated so far and for highly urgent news and relay programs.

(2) When the utterance content is displayed as an animation image of sign language, the necessary information can be visually appealed quickly and concisely even to fast-mouthed words that would be difficult to read only by displaying characters. . Sign language is a communication method that is easy for people who have learned it to understand.
Even if the display screen size is small, it is effective in reducing fatigue.

(3) Further, by displaying the animation image of the sign language on the screen at the same time corresponding to the position of the specified speaker, a realistic and easy-to-understand screen can be constructed.

(4) Alternatively, a more realistic and easy-to-understand screen can be constructed by changing the type of the animation image of sign language corresponding to each speaker.

[Brief description of drawings]

FIG. 1 is a configuration diagram of a video device according to a first embodiment of the present invention.

FIG. 2 is a block diagram of a character generation means in the same embodiment.

FIG. 3 is a configuration diagram of a voice recognition means and a sign language image generation means in the same embodiment.

FIG. 4 is a configuration diagram of a video device according to a second embodiment of the present invention.

FIG. 5 is a configuration diagram of position specifying means in the embodiment.

FIG. 6 is a configuration diagram of a video device according to a third embodiment of the present invention.

FIG. 7 is a block diagram of speaker identification means in the same embodiment.

FIG. 8 is a block diagram of a conventional video device.

[Explanation of symbols]

 2 tuner 4 video circuit 6 audio circuit 10 superimpose display means 11 character generation means 12 voice recognition means 13 sign language image generation means 15 position identification means 17 speaker identification means

Claims (4)

[Claims] 1. A media input unit for inputting an audio signal and a video signal, a character generation unit for converting an audio signal received by the media input unit into character information, and the character information generated by the character generation unit. A video device having a display means for displaying on the screen. 2. A media input unit for inputting a voice signal and a video signal, a voice recognition unit for recognizing the meaning content from the voice signal received by the media input unit, and a sign language for the meaning content recognized by the voice recognition unit. A video device comprising: a sign language image generating means for converting into the animation image and a display means for displaying the image information generated by the sign language image generating means on the screen. 3. A media input unit for inputting a voice signal and a video signal, voice recognition means for recognizing the meaning and content from the voice signal received by the media input unit, and the position of the speaker from the voice signal or the video signal. The position specifying means for specifying, the sign language image generating means for converting the meaning content recognized by the voice recognition means into a sign language animation image, and the image information generated by the sign language image generating means are specified by the position specifying means. A video device having display means for displaying on the screen corresponding to the position of the speaker. 4. A media input unit for inputting a voice signal and a video signal, voice recognition means for recognizing the meaning and content from the voice signal received by the media input unit, and identifying a speaker from the voice signal or the video signal. A speaker identification means, a sign language image generation means for converting the meaning content recognized by the voice recognition means into a sign language animation image, and image information generated by the sign language image generation means are identified by the speaker identification means. And a display unit for changing the type of the sign language animation and displaying it on the screen corresponding to each speaker.