JP2006139138A - Information terminal and base station - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To realize a conversation in real time between a person with hearing difficulties and a hearing able-bodied person. <P>SOLUTION: The information terminal is equipped with a speech recognition section 22 which recognizes speech, a sign language image conversion section 23 which converts the recognized speech to a sign language image by referencing the correspondence table of the speech signal and the sign language image, an image display element 24 which displays the converted sign language image, an image photographing section 25 which photographs the image, a sign language recognition section 26 which recognizes the sign language information from the photographed image, a sign language-speech conversion section 27 which converts the recognized sign language information to speech by referencing the correspondence table of the sign language information and the speech information, and a communication controller 21 which communicatably outputs the converted speech. <P>COPYRIGHT: (C)2006,JPO&amp;NCIPI

Description

  The present invention relates to an information terminal such as a mobile communication terminal and a base station.

In the case of a hearing-impaired person, it is difficult to conduct a normal conversation, and therefore, the conversation is often performed using sign language that does not require hearing. However, there are many hearing-impaired people who have mastered sign language, but there are few people with normal hearing who have mastered sign language, so unless you are a special hearing person who has mastered sign language, It is difficult to talk using sign language. Therefore, for example, the technique disclosed in Patent Document 1 proposes to detect the movement of sign language using a TV camera, recognize the sign language content, and display the sign language content as characters on the computer screen. is doing.
Japanese Patent No. 3289304

  However, when a hearing-impaired person wants to talk using a communication means such as a cellular phone, for example, there is actually no desirable communication means. For example, when communicating between terminals such as mobile phones capable of videophone communication, a sign language image can be transferred. In this case as well, a normal hearing person who is a communication partner of a hearing impaired person You can't talk unless you master sign language. Moreover, although the sign language can be displayed as character information by using the technique of Patent Document 1, since only one-way information transmission can be performed, appropriate information cannot be transmitted from the normal hearing person to the hearing impaired person. For example, even if information can be transmitted as text from a normal hearing person to the hearing impaired person, one communication is in sign language and the other communication is in a form of writing, which is unnatural. Conversation close to is impossible.

  An object of the present invention is to realize a real-time conversation between a hearing impaired person and a healthy hearing person.

  The information terminal according to the present invention includes means for recognizing sound, means for converting the recognized sound into a sign language image with reference to a correspondence table between the sound signal and the sign language image, and means for displaying the converted sign language image. Prepare. Further, means for capturing an image, means for recognizing sign language information from the captured image, means for converting the recognized sign language information into speech with reference to a correspondence table between the sign language information and the sound signal, and converted speech Is provided. According to this configuration, since it is possible to convert between voice and sign language images with reference to the correspondence table between voice signals and sign language images, real-time conversation between a hearing impaired person and a normal hearing person is realized. be able to.

  The base station of the present invention is a base station for allowing a communication terminal to access a mobile communication network, and refers to a correspondence table between a means for recognizing voice transmitted from the communication terminal and a voice signal and a sign language image. A means for converting the recognized speech into a sign language image, a means for outputting the converted sign language image in a communicable manner, a means for recognizing a sign language image from an image transmitted from the communication terminal, and sign language information and an audio signal. Referring to the correspondence table, a means for converting the recognized sign language information into speech and a means for outputting the converted speech in a communicable manner are provided. According to this configuration, since the base station has a function of converting voice and sign language images, real-time conversation can be realized between the hearing impaired person and the healthy hearing person using the communication terminal.

  According to the present invention, since conversion between voice and a sign language image becomes possible, real-time conversation can be realized between a hearing impaired person and a normal hearing person.

(First embodiment)
FIG. 1 is a block diagram showing the configuration of a system in which the information terminal of the present invention is used. When communication is performed between a healthy hearing person 40 and a hearing impaired person 50 using a communication system as shown in FIG. Is assumed. That is, the healthy hearing person 40 uses the mobile communication terminal 10, and the hearing impaired person 50 uses the mobile communication terminal 20, between the mobile communication terminal 10 and the mobile communication terminal 20 via the mobile communication network 30. Communicate. In the example shown in FIG. 2, the mobile communication terminal 10 forms a radio line with the radio base station 31 that exists nearby, and the mobile communication terminal 20 communicates with the radio base station 32 that exists nearby. Form a line. The radio base station 31 and the radio base station 32 are connected to each other via a predetermined communication network 33. In this example, it is assumed that the hearing-impaired person 50 has a conversation using sign language, while the normal hearing person 40 is assumed to be a normal person who does not master sign language.

  As shown in FIG. 1, the mobile communication terminal 10 used by the healthy hearing person 40 includes a voice input unit 11, a voice output unit 12, an image display unit 13, and a communication control unit 14, as in a general mobile phone. Yes. In FIG. 1, only the minimum main elements are shown as the configuration of the mobile communication terminals 10 and 20, but actually various components not shown are mounted as in the case of a general mobile phone.

  A microphone or the like is used as the voice input unit 11. The voice input unit 11 detects the voice spoken by the healthy hearing person 40 and outputs the voice as an electrical signal. The audio electrical signal output from the audio input unit 11 is transmitted via the communication control unit 14 and is transmitted to the mobile communication terminal 20 via the mobile communication network 30. The audio electrical signal transmitted from the mobile communication terminal 20 is received by the communication control unit 14 of the mobile communication terminal 10 via the mobile communication network 30 and output from the audio output unit 12 as audio. A speaker or the like is used as the audio output unit 12. Also, images and text information can be transmitted between terminals. The mobile communication terminal 10 displays the received image and character information on the image display unit 13. As the image display unit 13, a liquid crystal display capable of displaying images and characters is used. Accordingly, a mobile phone having a general configuration can be used as the mobile communication terminal 10.

  On the other hand, the mobile communication terminal 20 used by the hearing impaired person 50 includes a communication control unit 21, a voice recognition unit 22, a sign language image conversion unit 23, an image display unit 24, an image photographing unit 25, a sign language recognition unit 26, and a sign language voice conversion. A unit 27 and a sign language communication dictionary 28 are provided. A voice signal transmitted from the mobile communication terminal 10 used by the healthy hearing person 40 is received by the communication control unit 21 via the mobile communication network 30 and input to the voice recognition unit 22. The voice recognition unit 22 employs a known voice recognition technique, recognizes the content of the input voice, and outputs the recognition result as character string information. In the case of voice recognition, the recognition dictionary held in the sign language communication dictionary 28 is referred to. The recognition result output by the voice recognition unit 22 is input to the sign language image conversion unit 23.

  The sign language image conversion unit 23 generates a sign language image corresponding to character string information representing the received voice. That is, image information representing an image that is similar to the case where a human expresses a word in sign language is generated. Specifically, the images or figures corresponding to both arms, hands, and all fingers are layered against the background of the human body and face, and the standard sign language expression includes the positional relationship, orientation, and bending of each part. A moving image that approaches is sequentially generated, and the content of the image is changed according to the input voice. When the sign language image conversion unit 23 generates a sign language image, dictionary information such as predetermined rules is acquired from the sign language communication dictionary 28 and collated with the contents of the character string information input from the speech recognition unit 22. The images are converted to images sequentially.

The image display unit 24 displays the image information generated by the sign language image conversion unit 23 on the screen as a moving image. Since the content of this moving image corresponds to the voice spoken by the healthy hearing person 40 transmitted from the mobile communication terminal 10, the hearing impaired person 50 using the mobile communication terminal 20 refers to the screen of the image display unit 24. The words spoken by the healthy hearing person 40 can be recognized from the image as sign language.
The image photographing unit 25 is a television camera built in the mobile communication terminal 20 and continuously performs photographing when making a call. When sign language communication is performed, the imaging direction of the image capturing unit 25 is directed toward the hearing impaired person 50 who is the user, and almost the entire upper body including both arms, both hands, and all fingers of the hearing impaired person 50 is captured. Adjust the field of view so that.

  The sign language recognition unit 26 recognizes the content of words expressed in sign language by sequentially performing image processing on images continuously output from the image capturing unit 25, that is, information on moving images in which sign language is captured. When the sign language recognition unit 26 recognizes the sign language, the sign language communication unit 28 performs processing with reference to the dictionary information held in the sign language communication dictionary 28. For the processing in the sign language recognition unit 26, for example, the technique disclosed in the cited document 1 may be employed. The sign language content, which is the recognition result of the sign language recognition unit 26, is input to the sign language voice conversion unit 27 as character string information representing the word. The sign language speech conversion unit 27 generates a speech signal corresponding to the input character string information using a known speech synthesis technique. The voice signal generated by the sign language voice conversion unit 27 is transmitted via the communication control unit 21 and reaches the mobile communication terminal 10 via the mobile communication network 30.

  Therefore, when a hearing impaired person 50 who is a user of the mobile communication terminal 20 expresses a word in sign language toward the image capturing unit 25, a sound corresponding to the content of the sign language is automatically generated, and the mobile communication is performed as an audio signal. It is transmitted to the terminal 10. That is, the healthy hearing person 40 who is the user of the mobile communication terminal 10 can hear the contents of the sign language expressed by the hearing impaired person 50 as a direct voice from the voice output unit 12, so that the healthy hearing person 40 has a conversation. As with the case, it is possible to have a conversation with the hearing impaired person 50.

  When the apparatus shown in FIG. 1 is used, the hearing impaired person 50 can talk with the communication partner in sign language for two-way transmission and reception. However, a sufficient effect can be expected even if only one of the transmission and reception communication is realized by sign language. For example, even if the functions of the image capturing unit 25, the sign language recognition unit 26, and the sign language voice conversion unit 27 are omitted from the mobile communication terminal 20, the hearing impaired person 50 is the content of the words spoken by the normal hearing person 40 as the communication partner. Can be recognized from the sign language image, and can respond to it and participate in the conversation. In this case, the response from the hearing impaired person 50 may be transmitted as text information using e-mail, for example. Conversely, if the functions of the speech recognition unit 22, the sign language image conversion unit 23, and the image display unit 24 are omitted from the mobile communication terminal 20 while leaving only the function of converting the sign language image into sound and transmitting it, the auditory sense Since the sign language content expressed by the handicapped person 50 is converted into voice and transmitted, the healthy hearing person 40 can understand the sign language content transmitted by the hearing handicapped person 50. In this case, the hearing impaired person 50 understands the transmission contents from the healthy hearing person 40 by transmitting character information from the mobile communication terminal 10 to the mobile communication terminal 20 by using a function such as e-mail. Therefore, two-way communication is realized.

(Second Embodiment)
FIG. 3 is a block diagram showing the configuration of a system in which the base station of the present invention is used. In FIG. 3, elements corresponding to those in the first embodiment are denoted by the same reference numerals. The changed part will be described below. In this embodiment, a function for generating a sign language image from voice and a function for generating voice from a sign language image are provided in the radio base station 60 that relays communication of the mobile communication terminal 20B. Therefore, it is not necessary to provide these functions in the mobile communication terminal 20B, and only the communication control unit 21, the image display unit 24, and the image capturing unit 25 are provided in the mobile communication terminal 20B.

  The speech recognition unit 22, the sign language image conversion unit 23, the sign language recognition unit 26, the sign language speech conversion unit 27, and the sign language communication dictionary 28 provided in the radio base station 60 are the same as those in the first embodiment. Fulfills the function. The communication control unit 61 controls communication between the radio base station 60 and the communication network 33, and the communication control unit 62 controls communication between the radio base station 60 and the mobile communication terminal 20B. Accordingly, the radio base station 60 converts the audio signal transmitted from the mobile communication terminal 10 into a sign language image by the audio recognition unit 22 and the sign language image conversion unit 23, and transmits it as image information to the mobile communication terminal 20B. Therefore, a sign language image corresponding to the sound from the healthy hearing person 40 is displayed on the image display unit 24 of the mobile communication terminal 20B used by the hearing impaired person 50.

  Further, since the mobile communication terminal 20B transmits a sign language image obtained by capturing the hearing impaired person 50 by the image capturing unit 25 to the wireless base station 60 as image information, the wireless base station 60 transmits from the mobile communication terminal 20B. The sign language image thus converted is converted into speech by the sign language recognition unit 26 and the sign language speech conversion unit 27 and transmitted to the mobile communication terminal 10 as a speech signal, so that the healthy hearing person 40 can hear the sign language content expressed by the hearing impaired person 50 as speech. Can be heard from the voice output unit 12.

  In this embodiment, since processing with a large load such as image processing and voice recognition is arranged on the radio base station 60, the configuration of the mobile communication terminal 20B can be simplified, and the mobile communication terminal 20B can be downsized. And power consumption can be reduced. Further, it can be used as the mobile communication terminal 20B with almost no change in the configuration and operation of a conventional mobile phone terminal.

  Since the information terminal and the base station of the present invention can convert voice and sign language images, the information terminal and the base station have an effect that a real-time conversation can be realized between a hearing impaired person and a normal hearing person. It is useful as an information terminal such as a body communication terminal and a base station.

The block diagram which shows the structure of 1st Embodiment Block diagram showing a configuration example of a communication system The block diagram which shows the structure of 2nd Embodiment

Explanation of symbols

DESCRIPTION OF SYMBOLS 10,20 Mobile communication terminal 11 Voice input part 12 Voice output part 13 Image display part 14,21 Communication control part 22 Voice recognition part 23 Sign language image conversion part 24 Image display part 25 Image photographing part 26 Sign language recognition part 27 Sign language voice conversion part 28 Dictionary for Sign Language Communication 30 Mobile Communication Network 31, 32, 60 Wireless Base Station 33 Communication Network 40 Healthy Hearing Person 50 Hearing Impaired Person 61, 62 Communication Control Unit

Claims (4)

A means of recognizing speech,
Means for converting the recognized voice into a sign language image with reference to a correspondence table between the voice signal and the sign language image;
Means for displaying the converted sign language image;
An information terminal comprising: Means for taking an image;
Means for recognizing sign language information from captured images;
Means for converting the recognized sign language information into speech with reference to the correspondence table between the sign language information and the audio signal;
Means for outputting the converted audio;
An information terminal comprising:   An information terminal equipped with the information terminal according to claim 1 and the information terminal according to claim 2. A base station for allowing a communication terminal to access a mobile communication network,
Means for recognizing the voice transmitted from the communication terminal;
Means for converting the recognized voice into a sign language image with reference to a correspondence table between the voice signal and the sign language image;
Means for outputting the converted sign language image in a communicable manner;
Means for recognizing a sign language image from an image transmitted from a communication terminal;
Means for converting the recognized sign language information into speech with reference to a correspondence table between the sign language information and the audio signal;
A means for outputting the converted voice in a communicable manner;
A base station comprising:

Images