JP2003188948A - Portable terminal device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a portable terminal device wherein communication is enabled between users handicapped in eyes and ears by enabling mutual conver sion between voice and text format. <P>SOLUTION: A cellular phone set 100 is provided with a transmission and reception part 2, a key input part 5, a display 8, a speaker 12, a microphone 16, a control part 4 or the like, in addition, a voice recognition part 15 which performs voice recognition of voice from the microphone 16 and produces text data, and a voice synthesis part 10 which converts received text data to voice by using voice synthesis and outputs the voice from the speaker 12. In the control part 4, control is executed in such a manner that the data in which the voice from the microphone 16 is converted to text by using the voice recognition part 15 are transmitted via the transmission and reception part 2 in accordance with setting of an originating call side or a receiving call side, and the text data received by the transmission and reception part 2 are displayed on the display 8 via a display control part 7, and processed by the voice synthesis part 10 and synthesized voice is outputted from the speaker 12. <P>COPYRIGHT: (C)2003,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a mobile terminal device, and more particularly, to a communication capable of understanding the contents between a normal person, a visually impaired person, and a deaf person. And a mobile terminal device.

[0002]

2. Description of the Related Art Mobile terminals include mobile phones and PHS.
There are (Personal Handyphone System) telephones, PDA (Personal Digital Assistant) devices with wireless communication functions, etc., and mobile phones and PHS telephones are becoming more and more capable of communicating by email, connecting to the Internet, etc. There is. Some PDA devices have the function of a mobile phone in addition to the original usage form of a personal computer. Such a mobile terminal device includes a plurality of keys (numerical keys, symbol keys, function keys, etc.) and a display made of liquid crystal or the like. Further, there are models that do not require keys by using a touch panel instead of keys (buttons) and models that have multiple functions such as a packet function, a chat function, a game function, and a schedule function.

As the users of such portable terminal devices, most of them are healthy persons who can be mass-produced. Therefore, the voice output is effective for the visually impaired person, but the display is useless, and the display is effective for the deaf person, but the voice output is useless.

FIG. 4 shows a usage pattern of the portable terminal device when the user is a healthy person and when the user is another person. Here, any mobile terminal device has a telephone function (telephone mode).
It is assumed that the target is a normal person who has a data communication function (text mode). Further, it is assumed that the visually handicapped person has no hearing loss and the deaf person has no hearing loss. People who have both eyes and ears will use a dedicated model equipped with Braille input / output means.

The portable terminal device 401 used by a healthy person is
Since the eyes and ears are free, either the telephone mode or the text mode can be arbitrarily selected and used.
The mobile terminal device 402 used by a visually handicapped person can use the telephone mode because the user can hear, but cannot use the text mode because he cannot see the display. Further, the mobile terminal device 403 used by a deaf person can use the text mode because he / she can see the display, but cannot use the telephone mode because he / she cannot hear the sound.

Therefore, in the connection between the mobile terminal device 401 and the mobile terminal device 402, a normal telephone call (transmission mode 404) by voice (or sound information) is possible between them. However, even if text is sent from the mobile terminal device 401 to the mobile terminal device 402, the mobile terminal device 402
It does not work for the visually handicapped (and vice versa). However, the mobile terminal device 401 is provided with a conversion means for converting into “text → speech” and transmitting (transmission form 405), or the text received by the mobile terminal device 402 from the mobile terminal device 401 is “text → speech”. With the conversion means for converting to, a visually handicapped person can understand the contents of the text.

On the other hand, in the connection between the mobile terminal device 401 and the mobile terminal device 403, since the mobile terminal device 403 only uses the text display, the text is directly transmitted (transmission mode 406) or the mobile terminal device 403 is used. Terminal device 401
Voices of users are converted to text and transmitted (Transmission form 4
07), it is possible to transmit the call contents to a deaf person. In this case, in order to convey the content of the call from the mobile terminal device 403 to the mobile terminal device 401, the mobile terminal device 403 side transmits it via the “text → speech” conversion means (transmission form 408), or the mobile terminal device 403. This can be done by converting "text to voice" on the device 401 side.

Further, in connection between the mobile terminal device 402 and the mobile terminal device 403, only one of voice and text is used. From the mobile terminal device 402 to the mobile terminal device 403, the user of the mobile terminal device 403 can use only the text.
From 2 the text is transmitted directly (transmission form 409) or the voice of the user of the mobile terminal device 402 is converted into text and transmitted (transmission form 410) to convey the call contents to the deaf person. be able to. Further, from the mobile terminal device 403 to the mobile terminal device 402, the mobile terminal device 40
Since the user of 2 uses only the voice, the mobile terminal device 403 transmits it through the conversion means of “text → voice” (transmission form 411), or the mobile terminal device 402 changes to “text → voice”. It is possible if converted.

Japanese Patent Laid-Open No. 9-116648 discloses a device which enables the above mutual communication. Figure 5
The communication form in this publication is shown. The portable information terminal device 501 and the portable information terminal device 502 compatible with multimedia are connected via a communication line 503 capable of transmitting a voice signal. The portable information terminal 501 corresponds to a PDA device, and the portable information terminal 502 corresponds to a personal computer. The portable information terminal 501 includes a voice synthesizing device for synthesizing input character data by voice, a voice converting device for converting a voice synthesizing signal from the voice synthesizing device into a voice signal, and a voice signal from the voice converting device to a line. It comprises a modem for outputting, a voice understanding device for recognizing voice signals from the other side, and a character conversion device for converting the recognition result by the voice understanding device into characters and displaying them on a display.

Since the transmission on the communication line 503 becomes an audio signal, the signal transmitted and received by any portable information terminal becomes an audio signal. However, the portable information terminal 501 and the portable information terminal 502 can input characters so that even a deaf person can use them, and the input characters are converted into voice information by a voice synthesizer and a voice converter. The information can be transmitted as information, and the voiced text information can be converted into character information by the voice understanding device and the character conversion device. For example, if the user of the portable information terminal 501 is a healthy person and the user of the portable information terminal 502 is a deaf person, the character information is converted into voice from the portable information terminal 501. The information can be obtained by converting the voice information transmitted and received by the portable information terminal 502 into text and displaying it. On the contrary, in the communication from the portable information terminal 502 to the portable information terminal 501, the portable information terminal 502 converts the character information into voice and transmits it.
Then, the information is transmitted by converting the received voice information into text and displaying it.

[0011]

However, according to the conventional portable terminal device, in the configuration of Japanese Patent Laid-Open No. 9-116648, not the real voice is transmitted but the text (character) is converted into voice and transmitted, and the receiving side. As long as the handheld device and the deaf person use the mobile terminal device, there is no problem, but the visually impaired person cannot see the text. So it cannot be used. In order to make it usable for the blind, the transmission modes 406, 407, 409, 4 shown in FIG.
You need to be able to use 10.

It is therefore an object of the present invention to provide a portable terminal device which enables mutual conversion between voice and text formats and enables communication even between persons with eyes and ears.

[0013]

In order to achieve the above object, the present invention provides character input means, a display capable of displaying the content input by the character input means and received text data, and voice input. In the mobile terminal device, a portable terminal device comprising means, voice output means, and wireless transmission / reception means for transmitting / receiving a voice by the voice input means and transmitting a text created using the character input means, A voice / text conversion means for recognizing voice to generate text data, a text data / voice conversion means for converting the received text data into a voice signal by voice synthesis, and the voice performed via the transmitting / receiving means. / If the text data is transmitted by the text conversion means depending on the setting contents of the calling side or the receiving side, , To provide a mobile terminal device, characterized in that it comprises a control means for outputting a sound by voice synthesis by the text data of the text data thus received has been received and simultaneously displayed on the Display the text data / voice converting means.

According to this structure, even if the user of the mobile terminal device on the receiving side has a problem, for example, an ear or an eye, the calling side responds to the text converted from the voice to the text. The data is transmitted, the received text data is displayed on the display on the called side, and at the same time, the text data / voice conversion means outputs the voice synthesized from the text data on the called side. Therefore, the deaf person can understand the communication content from the text displayed on the display, and the deaf person can understand the communication content from the output of the voice-synthesized voice. Communication between persons with disabilities as well as persons with disabilities in eyes and ears becomes possible, and persons in all positions can communicate through the mobile terminal device.

[0015]

DETAILED DESCRIPTION OF THE INVENTION

Embodiments of the present invention will be described below with reference to the drawings. In the following, it is assumed that the mobile terminal device is a mobile phone. FIG. 1 shows an embodiment of a mobile terminal device of the present invention. The mobile phone 100 includes an antenna 1, a transmission / reception unit 2 as a transmission / reception unit, a data conversion unit 3, a control unit 4 as a control unit, a key input unit 5 as a character input unit, a storage unit 6, a display control unit 7, and a display 8. A voice processor 9, a voice synthesizer 10 as a text data / voice converter, a switch 11, a speaker 12 as a voice output unit, a switch 13, a voice processor 14, a voice recognizer 15 as a voice / text converter, And a microphone (hereinafter, referred to as a microphone) 16 as a voice input unit.

The antenna 1 is a whip antenna which is attached to the main body in a stretchable manner. The transmission / reception unit 2 transmits / receives voice and data to / from a base station by a predetermined modulation method. The data conversion unit 3 decodes the received signal from the transmission / reception unit 2 and sends it to the voice processing unit 9 if it is voice data, and the control unit 4 or the voice synthesis unit 1 if it is other data.
0, voice processing units 9 and 14, voice recognition unit 1
The voice data from 5 and the data from the control circuit 4 are encoded and supplied to the transmitting / receiving unit 2. The control unit 4 includes a CPU 4a, a ROM and a RAM, and a key input unit 5
Each unit is controlled in accordance with various commands input from the computer and the setting contents stored in the data storage unit 24. The key input section 5 is
A plurality of keys (numeric keys, symbol keys, function keys, etc.) or buttons are provided, and information corresponding to the keys is input according to the user's pressing operation. The storage unit 6 includes a telephone book, a schedule book, and a semiconductor memory that stores various setting contents set by the user. The display control unit 7 controls the display of the display 8 under the control of the control unit 4. The display 8 is configured by using a liquid crystal display, and can display characters of, for example, 8 to 15 characters × 6 to 14 lines.

The audio processor 9 converts the audio data from the data converter 3 into an audio signal and outputs it. Speech synthesizer 1
0 is converted into voice PCM (pulse code modulation) data (text data) from the data converter 3, and this data is converted into a voice signal. The switch 12 connects either of the output terminals of the voice processing unit 9 or the voice synthesis unit 10 to the speaker 12 under the control of the control unit 4. Switch 13
Under the control of the control unit 4, either the output of the voice processing unit 14 or the voice recognition unit 15 is selected and supplied to the data conversion unit 3. The voice processing unit 14 converts the voice signal from the microphone 16 into digital data (voice data). The voice recognition unit 15 converts the voice signal from the microphone 16 into text data. In the telephone mode, the speaker 12 functions as a receiver and the microphone 16 functions as a transmitter.

FIG. 2 shows the processing when the mobile phone of FIG. 1 makes a call. In the following flowchart, the step is represented by "S". Here, it is assumed that both the calling side and the called side use the mobile phone having the configuration shown in FIG.
In addition to the conventional telephone mode and text mode, the mobile phone can use the third mode of the “text transmission request” according to the present invention. In the following, this third mode will be mainly described.

In the present invention, whether or not the mobile phone 100 used by a visually handicapped person or a deaf person is known unless the communication (call) partner knows in advance. Can not. Therefore, at the start of communication, a notification (text transmission request) can be made after the line is connected from the side of the mobile terminal device used by a visually handicapped person or a deaf person. The setting is performed by a key operation determined from the menu screen, or a dedicated key on the surface of the main body of the mobile terminal device (may be combined with other functions, for example,
If a “text transmission request” may be made when the button is pressed for 3 seconds or more), a notification will be made when the key is pressed. If the user of the mobile phone 100 on the calling side knows that the communication partner is a person with a disability from the beginning, the same function as the “text transmission request” is performed in advance by operating the designated key. "Text conversion mode" can be set.

In FIG. 2, when the power is on,
The user on the calling side can set the above-mentioned "text conversion mode" as necessary (S201).
Next, when the telephone mark button is pressed to go off-hook (S202), the control unit 4 operates the transmission / reception unit 2 and the data conversion unit 3 to call the base station. When the connection with the other party is completed via the base station (S203), the control unit 4 determines the called party (the other party).
Determines whether the "text transmission request" has come from the device or the "text conversion mode" has been set by the device itself (S203). When the "text transmission request" has not been received and the "text conversion mode" has not been set, the voice processing units 9 and 14 are used to execute a normal voice telephone mode (S211). In this process, if the call is terminated and the user goes on-hook (S212), the line is disconnected (S213).

When the "text transmission request" is received from the receiving side, the mobile phone 100 on the calling side displays a message such as "a text transmission request has been issued from the communication destination" on the display 8 (S205). ). The calling user who sees this message can know that the other party is a visually impaired person or a deaf person. When the control unit 4 detects a command corresponding to the "text transmission request" from the other party, the control unit 4 switches both the switches 11 and 13 to the b contact side. When the user on the calling side speaks into the microphone 16 (S206), the voice is recognized by the voice recognition unit 15 and text data is generated (S2).
07). This text data is displayed on the display 8 (S208). With this display, the user can confirm whether or not the contents uttered by him / her have been correctly converted into “voice → text”. If the contents are correct (S209), a previously designated button is pressed.
As a result, the text data generated by the voice recognition unit 15 is sent to the data conversion unit 3 via the switch 13,
It is converted into data for transmission (S210). As a result, the voice content input from the mobile phone 100 on the calling side is transmitted as text data, and this text data is received by the mobile phone (mobile terminal device) on the other side and displayed on the display 8 of the mobile phone. To be done. When the communication is completed and the on-hook is performed (S212), the line is disconnected (S213).

FIG. 3 shows a process when the mobile phone of FIG. 1 receives (receives) an incoming call. When the power is on, a person with a hearing disability may request “text transmission” so that the mobile phone 100 being used can be transmitted by text.
(The setting is performed by operating the key on the menu screen or operating the dedicated key as in the case of the above-mentioned calling) (S301). If the user of the mobile phone 100 is visually impaired, a voice-only transmission request is set (S301).

When the transmission / reception unit 2 receives a radio wave from the base station via the antenna 1 and an incoming call is detected (S302) and an off-hook is performed (S303), the control unit 4 sets the contents of S301. Is checked (S304),
If it is not the "text transmission request", the telephone mode is executed (S310). On the other hand, if "text transmission request" is set, the control unit 4 displays the text data sent from the calling side on the display 8 via the display control unit 7 (S305), and at the same time, the voice synthesis unit. The voice synthesized by 10 is switched to the speaker 1 via the switch 11.
Output from 2. As a result, the user of the mobile phone 100 can understand the information from the calling side regardless of the visually impaired person or the deaf person. When data is to be transmitted from the receiving side, characters can be input from the key input unit 5 and converted into data for transmission by the data conversion unit 3 and then transmitted as text data (S307). When the communication or the call ends and the on-hook is performed (S308), the line is disconnected (S309).

As described above, according to the present invention, in addition to communication between voices, the calling side can perform "voice-to-text" conversion and transmit to the line side (base station side), and the receiving side can. Since the text can be displayed and the voice can be output at the same time by voice synthesis, all of the eight transmission modes shown in FIG. 4 can be realized. As a result, the present invention enables mutual communication between a mobile phone used by a visually handicapped person and a mobile phone used by a deaf person, which could not be realized conventionally.

In the above embodiment, the "text transmission request" is notified from the called side to the calling side. However, it is possible to previously register only the text transmission in the telephone directory of the calling side mobile telephone. Alternatively, text transmission using voice recognition and voice synthesis may be automatically executed for a registered telephone number.

In the above embodiment, S20 of FIG.
7 and S208, voice output is performed simultaneously with the display of the text data, and in FIG. 3, the text data is displayed (S305) and the voice is output by voice synthesis (S30).
6) However, if the mobile terminal device (mobile phone) is a dedicated device intended only for visually impaired persons, the display processing of S208 and S305 may be omitted.
Similarly, if it is a special-purpose machine intended only for people who are deaf, the voice output process of S306 can be omitted.

[0028]

As is apparent from the above, according to the portable terminal device of the present invention, according to the setting of the calling side or the called side,
The text data converted from voice to text is sent from the calling side, the received text data is displayed on the display at the called side, and at the same time, the text data /
Since the voice conversion means outputs the voice synthesized from the text data, it is possible to communicate not only between the healthy person and the handicapped person but also those who have eyes and ears. People in all positions can communicate through the terminal device.

[Brief description of drawings]

FIG. 1 is a block diagram showing an embodiment of a mobile terminal device of the present invention.

FIG. 2 is a flowchart showing processing when the mobile terminal device of FIG. 1 makes a call.

FIG. 3 is a flowchart showing processing when the mobile phone of FIG. 1 receives (receives) an incoming call.

FIG. 4 is an explanatory diagram showing usage forms of the mobile terminal device when the user is a healthy person and when the user is another person.

FIG. 5 is an explanatory diagram showing a communication form in a conventional mobile terminal device.

[Explanation of symbols]

1 antenna 2 transceiver 3 data converter 4 control unit 5 Key input section 6 memory 7 Display control section 8 displays 9 Voice processing unit 10 Speech synthesizer 11,13 switch 12 speakers 14 Voice processing unit 15 Speech recognition unit 16 microphone

Claims (4)

[Claims] 1. A character input unit, a display capable of displaying the contents input by the character input unit and received text data, a voice input unit, a voice output unit, and a voice transmission / reception by the voice input unit. And a mobile terminal device comprising a wireless transmitting / receiving means for transmitting a text created using the character input means, a voice / text converting means for voice-recognizing the voice by the voice input means to generate text data. A text data / voice conversion means for converting the received text data into a voice signal by voice synthesis; and a transmission of text data by the voice / text conversion means via the transmission / reception means to a calling side or a receiving side. According to the setting contents of, and the received text data in the display A mobile terminal device, comprising: a control unit for outputting a voice obtained by voice-synthesizing the text data received at the same time as being displayed by the text data / voice conversion unit. 2. The mobile terminal device according to claim 1, wherein the control unit displays the text data generated by the voice / text conversion unit on the display. 3. The mobile terminal device according to claim 1, wherein the control means is a text transmission request from the receiving side for an outgoing call as the state of the receiving side. 4. The mobile terminal device according to claim 1, wherein the control means has a function that allows a user to set information that requests the calling side to send only text.