JP2003274004A - Conversion method from received voice information in mobile terminal to character image information and mobile terminal with function of executing the conversion method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To allow a user to surely confirm message contents without giving a sense of discomfort to surrounding people and feeling a delicacy about the surrounding people. <P>SOLUTION: When detecting the arrival of a call, a control section transmits a reply message for urging a caller to leave a message to the caller via a wireless transmission section. Then the control section checks whether or not the caller sends its message, allows a storage section to once store contents of the message and a voice recognition section to process voice recognition when the caller transmits the message. Upon the receipt of a result of recognition, from the voice recognition section, the control section once stores the result to the storage section and allows a main display section 9 to display a message denoting that a message memo is left. When the user confirming this message makes a prescribed reproduction operation, the control section allows the main display section 9 to display the result of recognition of the message memo. <P>COPYRIGHT: (C)2003,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method of converting received voice information into character image information in a mobile terminal having a call function and a mobile terminal having a function of executing the conversion method.

[0002]

2. Description of the Related Art Conventionally, for example, a mobile phone has a function of recording a message from a caller as a message memo when the user cannot answer the call when receiving a call. In a foldable mobile phone 101 having such a function, as shown in FIG. 20, a control unit 102 for controlling each component of the mobile phone main body, a processing program executed by the control unit 102, and a telephone directory. A storage unit 103 for storing various data such as information, message memo contents, and incoming call history information, and an antenna 104 for transmitting and receiving radio waves.
And modulates voice and data and transmits them as radio waves via the antenna 105.
5, a wireless communication unit 106 that is used for performing voice communication and data communication according to a predetermined protocol by receiving via 5 and demodulating into voice and data, an operation unit 107 including various operation keys, and a liquid crystal display device, for example. Display unit 108
And an incoming call notification lamp 109 composed of LEDs for notifying an incoming call, and each LED constituting the incoming call notification lamp 109
An LED drive unit 110 for driving a ringer generator, a ringer generator 111 for outputting music for incoming call notification, a ringer drive unit 112 for driving the ringer generator 111, a speaker 113 for outputting a received voice, A microphone 114 for inputting a speech voice is provided.

The user sets a message memo in advance, and a response message prompting the caller to send a message is transmitted when an incoming call arrives, and the message memo is recorded by the transmission of the message of the caller. The control unit 102 monitors the presence or absence of an incoming call, and when detecting the presence of an incoming call, the control unit 102 transmits, via the wireless transmission unit 106, a response message prompting the caller to make a message. Next, the control unit 102 checks whether or not the caller is sending a message, and if it is sending a message, temporarily stores the contents of the message memo in the storage unit 103. The control unit 102
A message indicating that the message memo is left is displayed on the display unit 10.
Display in 8. Next, the control unit 102 monitors whether or not a reproduction operation is performed by the user, and when the user who confirms this message performs a predetermined reproduction operation, the content of the message memo is output from the speaker 113.

[0004]

However, in the above-mentioned conventional mobile phone, the speaker 1 is used when the message memo is reproduced.
There is a problem in that the reproduced sound output from No. 13 is perceived as noise by the surrounding people and causes discomfort.
In addition, there is a problem in that the user cannot play the content and feels inconvenience, especially when the user wants to play the content in a hurry and wants to keep the contents secret from the surrounding people.

The present invention has been made in view of the above circumstances, and is capable of surely confirming the reproduced content without causing discomfort to the surrounding people and without hesitating to the surrounding people. An object of the present invention is to provide a method for converting received voice information in a terminal into character image information and a mobile terminal having a function of executing the conversion method.

[0006]

In order to solve the above-mentioned problems, the invention according to claim 1 relates to a method of converting voice information received by a portable terminal having a call function into corresponding character image information. It is characterized by including a voice information storing step of storing the voice information and a character image information outputting step of converting predetermined voice information into associated character image information and outputting the converted character image information.

The invention according to claim 2 relates to a method for converting received voice information into character image information in the portable terminal according to claim 1, wherein the character image information output step is the voice information storing step. It is characterized in that it is executed afterward, but before the reproduction operation of the stored contents stored by the execution of the voice information storing step.

The invention according to claim 3 relates to a method for converting received voice information into character image information in the portable terminal according to claim 1, wherein the character image information output step is stored by executing the voice information storage step. It is characterized in that it is executed after the reproduction operation of the stored content is performed.

The invention according to claim 4 relates to a method for converting received voice information into character image information in the portable terminal according to claim 1, 2 or 3, and relates to the character image information output in the character image information output step. Is displayed on the display means.

A fifth aspect of the present invention relates to a method for converting received voice information into character image information in the portable terminal according to the fourth aspect, wherein the portable terminal has a foldable housing and the display is provided. The means is characterized in that it is arranged on the inner side when folded.

A sixth aspect of the present invention relates to a method for converting received voice information into character image information in the portable terminal according to the fourth aspect, wherein the portable terminal has a foldable housing and the display is provided. The means is characterized in that it is arranged on the side that is on the outside when folded.

The invention according to claim 7 relates to a method for converting received voice information into character image information in the portable terminal according to any one of claims 1 to 6, which is output in the character image information output step. It is characterized by including a quoting step of quoting the character image information to edit a predetermined sentence.

An eighth aspect of the present invention relates to a method for converting received voice information into character image information in the portable terminal according to any one of the first to seventh aspects, wherein the character image information output step is unspecified. It is characterized by performing speaker voice recognition.

The invention according to claim 9 relates to a method of converting received voice information into character image information in the portable terminal according to any one of claims 1 to 7, wherein the sender is identified after receiving the voice information. And a caller identification step of obtaining caller information. In the character image information output step, specific speaker voice recognition is performed based on the caller information.

According to a tenth aspect of the present invention, there is provided a mobile terminal having a function of executing the method of converting received voice information into character image information, the communication means for receiving the voice information, and the character image corresponding to the received voice information. A conversion unit for converting the information into information, the conversion unit storing the received voice information, and a character image information output unit for converting predetermined voice information into associated character image information and outputting the converted character image information. It is characterized by having and.

The invention according to claim 11 relates to a portable terminal having a function of executing the method for converting received voice information into character image information according to claim 10, and relates to a character image output from the character image information output means. It is characterized in that display control means for displaying information on the display means is provided.

According to a twelfth aspect of the present invention, there is provided a portable terminal having a function of executing the method for converting the received voice information into the character image information according to the eleventh aspect, which comprises a foldable housing and displays the above. The means is characterized in that it is arranged on the inner side when folded.

According to a thirteenth aspect of the present invention, there is provided a portable terminal having a function of executing the method for converting the received voice information into the character image information according to the eleventh aspect, which has a foldable housing and has the above display. The means is characterized in that it is arranged on the side that is on the outside when folded.

The invention according to claim 14 relates to a portable terminal having a function of executing the method for converting received voice information into character image information according to any one of claims 10 to 13, wherein the character image information is output. It is characterized in that a quoting means for quoting the character image information output from the means to edit a predetermined sentence is provided.

The invention according to a fifteenth aspect relates to a portable terminal having a function of executing the method for converting the received voice information into the character image information according to any one of the tenth to fourteenth aspects. The means is characterized by performing unspecified speaker voice recognition.

The invention according to claim 16 relates to a portable terminal having a function of executing the method for converting received voice information into character image information according to any one of claims 10 to 14, and transmitting the voice information. The character image information output means is characterized by performing a specific speaker voice recognition based on the caller information.

[0022]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described below with reference to the drawings. The description will be specifically made using the embodiments. First Embodiment FIG. 1 is a perspective view showing a configuration of a folding type mobile phone according to a first embodiment of the present invention, in which a state in which the housing of the mobile phone is opened is viewed from the front side. FIG. 2 and FIG. 2 are perspective views showing a configuration of the mobile phone, showing a state in which the housing of the mobile phone is opened as seen from the rear side, and FIG. 3 is a perspective view showing the configuration of the mobile phone. FIG. 4 is a diagram showing a state in which the housing of the mobile phone is closed, FIG. 4 is a block diagram showing a configuration of the mobile phone, and FIG. 5 is a block diagram showing a configuration of a voice recognition unit of the mobile phone. FIG. 6 is a flow chart for explaining the operation of the mobile phone, and FIGS. 7 to 10 are diagrams showing screen display examples of the main display unit of the mobile phone.

As shown in FIGS. 1 to 3, the mobile phone 1 of this example is provided with a foldable housing 2, and in addition to the original call function, for example, it is connected to the Internet to browse a home page. A data communication function capable of sending and receiving electronic mail, and as shown in FIGS. 1 to 4, a control unit (a part of conversion means, a citation means, a display control means) for controlling each component of the main body of the mobile phone. ), A processing program executed by the control unit 3, a storage unit (a part of conversion unit, storage unit) 4 for storing various data such as telephone directory information, message memo contents, incoming call history information, etc. An antenna 5 that transmits and receives radio waves, modulates voice and data and transmits them as radio waves through the antenna 5, and receives radio waves through the antenna 5 and demodulates them into voice and data, according to a predetermined protocol. Obey A wireless communication unit (communication unit) 6 used for making a call or data communication, and a voice recognition unit (a part of conversion unit, character image information output) for converting received voice information of an unspecified speaker into, for example, character information. Means) 7, an operation section 8 including various operation keys, etc., and a main display section (display means) arranged on the inner side when folded and formed of a liquid crystal display device.
9, an auxiliary display unit 11 formed of a liquid crystal display device on the outer side when folded, and an LED for notifying an incoming call
Notification lamp 12 including an LED, and an LED drive unit 13 for driving each LED that constitutes the notification lamp 12
A ringer generator 14 that outputs a ring tone, a ringer drive unit 15 that drives the ringer generator 14, a speaker 16 that outputs a reception voice, a microphone 17 that inputs a transmission voice, a battery pack, and the like. And a power supply section 18 consisting of.

The housing 2 is composed of an upper housing 21 and a lower housing 22 which are connected to each other by a hinge portion 19.
The upper housing 21 and the lower housing 22 are rotatably coupled to each other to give the mobile phone 1 a foldable structure. The control unit 3 includes a voice recognition control program for controlling the main display unit 9 and the auxiliary display unit 11, a browser as a program for browsing a home page, and a mailer as a program for creating and sending / receiving electronic mail. Etc. execute various processing programs stored in the storage unit 4, etc., and use the various registers and flags secured in the storage unit 4 to control each constituent unit and perform voice recognition control processing and the like.

The storage unit 4 comprises a semiconductor memory such as a ROM and a RAM, and a program storage unit in which various processing programs such as a voice recognition control program executed by the control unit 3 are stored, and information in which various information is stored. In addition to having a storage unit, various registers and flags used by the control unit 3 during program execution are secured. For example, the voice recognition control program describes a procedure for causing the voice recognition unit to perform voice recognition processing after an incoming message is recorded and displaying the recognition result on the display unit by a predetermined playback operation. There is. The information storage unit is provided with a personal information storage area in which a name, a telephone number, an electronic mail address, etc. are stored, and a received voice information storage area. The personal information storage area stores a name, a telephone number, an electronic mail address, etc. as telephone directory information.

The voice recognition unit 7 classifies (clusters) the spectrum sequence of a large number of speakers for each word with the acoustic analysis processing unit 23 for obtaining a short-time spectrum of the voice, and sets it at the center of each cluster or each cluster. The classification processing unit 24 for obtaining the average spectrum sequence of the voices belonging to, the standard pattern storage unit 25 for storing the spectrum sequence obtained by the classification processing unit 24 as a standard pattern, and the matching between the input voice and the standard pattern are performed. The word recognition processing unit 26 calculates the degree of similarity and outputs the standard pattern word having the highest degree of similarity as a recognition result. After the high-frequency emphasis processing, the acoustic analysis processing unit 23 determines the short-time spectrum of the voice to be, for example, 10 [ms].
Obtain every ~ 20 [ms]. For the spectrum analysis, for example, a group of 15 to 20 band filters and a linear prediction model of the 8th to 12th order are usually used. As a spectral distance measure,
In the case of bandpass filter analysis, Euclidean distance between the feature vectors of the filter output is used, and in the case of linear prediction analysis, various LPC spectral distance measures are used.

The word recognition processing unit 26 expands and contracts the time axis of the input voice and the standard pattern in a non-linear manner in order to deal with the fluctuation of the generation speed, and matches the distance between the corresponding short-time spectra over the entire word. And outputs the standard pattern word name that minimizes the cumulative distance. In order to efficiently perform the above cumulative distance calculation, dynamic programming (DP:
The DP matching method that introduces the method of Dynamic Programming) is widely used. For the unknown input voice, the word recognition processing unit 26 uses all standard patterns of all words and, for example, D
Perform P matching and calculate the distance as similarity,
The standard pattern word with the highest degree of similarity is output as the recognition result.

The operation unit 8 has a browser mode selection key 8a for activating a browser and browsing a home page, and a determination key used for determining an operation, on a surface which is an inner side when the lower housing 22 is folded. 8b, a menu key 8c for displaying an operation menu, an input mode switching key 8d for switching the character input mode, and a cursor key for moving the cursor on the display screen displayed on the main display section 9 vertically and horizontally. 8e, a phonebook key 8f for registering or searching a phonebook, a call start key 8g used for making a voice call, a clear key 8h for returning the operation to the previous state, turning on / off the power, and the like. A power key 8i used to stop the operation, a ten-key 8j for inputting numbers and the like are arranged.

The main display portion 9 is arranged on the inner surface when the upper casing 21 is folded and is composed of, for example, a transmissive liquid crystal display device. The liquid crystal display device has a liquid crystal panel and a backlight arranged behind the liquid crystal panel. The liquid crystal panel is, for example, a TFT type panel, and includes a TFT substrate having a large number of TFTs and transparent pixel electrodes,
The counter substrate is fixed to face the FT substrate with a gap of several [μm] and has a colored layer (color filter) formed thereon, the liquid crystal layer sealed in the gap, the TFT substrate, and the outside of the counter substrate. And a pair of deflecting plates arranged.
The auxiliary display unit 11 is arranged in the central portion of the outer surface when the upper housing 21 is folded, is made of, for example, a transmissive liquid crystal display device, and displays the current time, incoming call notification, and the like. The liquid crystal display device of the auxiliary display unit 11 has substantially the same configuration as the liquid crystal display device of the main display unit 9 described above.

Next, the operation of the mobile phone 1 of this example will be described with reference to FIGS. First, the message memo is set in advance. After the power is turned on, the control unit 3
A standby mode for waiting for an input operation or an incoming call by the user is entered, and a standby screen is displayed on the main display section 9. When the user presses the menu key 8c and then the cursor key 8e or the like to perform a predetermined operation, the control unit 3 causes the main display unit 9 to display a message memo setting screen, as shown in FIG. Here, when the user selects "ON", a response message prompting the caller to make a message is transmitted when an incoming call arrives. Furthermore, here, settings are made to display the contents of the message memo.

In the standard pattern storage unit 25, a large number of standard patterns that represent variations in sound quality due to individuals are prepared and stored in advance for each word. That is, when a voice of a speaker is input to the acoustic analysis processing unit 23, the acoustic analysis processing unit 23 obtains a short-time spectrum of the voice, and the classification processing unit 24 determines a spectrum sequence of many speakers for each word. After classification (clustering), the average spectral sequence of the sounds at the center of each cluster or belonging to each cluster is obtained. The standard pattern accumulation unit 25 is a classification processing unit 24.
The spectrum series obtained in step 1 is stored as a standard pattern. In this example, these standard patterns are stored in the standard pattern storage unit 25 at the time of factory shipment.

In this example, the portable telephone 1 is used in a state in which the user switches to the manner mode in which the sound is muted by the user performing a predetermined setting operation using the operation unit 8.
The control unit 3 monitors whether there is an incoming call (step ST11).
(FIG. 6)) When the presence of an incoming call is detected, a response message for prompting the caller to send a message is transmitted via the wireless transmission unit 6. Next, the control unit 3 checks whether or not the caller is sending a message (step ST12), and the caller sends a message such as "Waiting at the hamburger shop in front of the station at 10 o'clock tomorrow." If it is,
The message content is stored in the storage unit 4 (step ST1
3) After that, the voice recognition unit 7 is caused to perform voice recognition processing (step ST14). Here, the control unit 3 causes the main display unit 9 to display a message indicating that recording is currently being performed, as shown in FIG.

In the voice recognition section 7, first, the acoustic analysis processing section 23 obtains a short-time spectrum of the voice information of the message content read from the storage section 4. Next, the word recognition processing unit 26
Matches the input voice with the standard pattern stored in the standard pattern storage unit 25 to calculate the similarity and outputs the word of the standard pattern with the highest similarity as the recognition result. When the control unit 3 receives the recognition result from the voice recognition unit 7, the control unit 3 temporarily stores it in the storage unit 4, and displays a message indicating that the message memo is left on the main display unit 9 as shown in FIG. Let Next, the control unit 3 monitors whether or not a reproducing operation is performed (step ST15), and when the user who confirms this message performs a predetermined reproducing operation, the recognition result of the message memo is as shown in FIG. Then, it is displayed on the main display section 9 (step ST16).

As described above, according to the configuration of this example, the content of the recorded message memo is recognized by voice and displayed on the main display portion. Therefore, by selecting the manner mode or the like to turn off the sound, the surroundings It is possible to confirm the reproduction content displayed on the main display unit 9 without giving discomfort to a person and without anxiety to the people around. In particular, even the hearing impaired person can easily confirm the reproduction content. Also, since the playback content can be known visually, the message content can be recognized more reliably than by hearing,
It is possible to improve convenience. Further, by storing the character information in the storage unit 4 instead of the voice information, the memory capacity of the storage unit 4 can be saved and the memory resources can be effectively used.

Second Embodiment FIG. 11 is a flow chart for explaining the operation of the folding type mobile phone according to the second embodiment of the present invention. This example is largely different from the above-described first example in that the voice recognition process is performed after the reproduction operation by the user. The rest of the configuration is substantially the same as that of the above-described first embodiment, and therefore its explanation is omitted.

Next, the operation of the mobile phone 1 of this example will be described with reference to FIG. Also in this example, the message memo is set in advance. In addition, the standard pattern storage unit 2
In FIG. 5, a large number of standard patterns that represent variations in sound quality due to individuals are prepared and stored in advance for each word, for example. The control unit 3 monitors whether there is an incoming call (step S
When the presence of an incoming call is detected at T21 (FIG. 11)), a response message for prompting the caller to send a message is transmitted via the wireless transmission unit 6. Next, the control unit 3 checks whether or not the caller is sending a message (step ST22), and if it is sending a message, temporarily stores the message content in the storage unit 4 (step ST23). Next, the control unit 3 monitors whether or not a reproduction operation is performed (step ST24), and when the user performs a predetermined reproduction operation, causes the voice recognition unit 7 to perform a voice recognition process (step ST25). Upon receiving the recognition result from the voice recognition unit 7, the control unit 3 stores the recognition result in the storage unit 4 and the recognition result of the message memo on the main display unit 9 (step ST26).

According to the configuration of this example, it is possible to obtain substantially the same effect as that described in the first embodiment. In addition, since the voice recognition is performed after the user's reproduction operation, the voice recognition can be performed according to the convenience of the user. For example,
When the user does not need to confirm the message memo with characters, he / she can cancel the setting of the message content display and confirm with the voice, so that unnecessary processing need not be performed.

Third Embodiment FIGS. 12 to 16 are diagrams showing screen display examples of the main display portion of the folding type mobile phone according to the third embodiment of the present invention. The major difference of this example from the above-mentioned first example is that the character information as the voice recognition result can be quoted in the body of the electronic mail. The rest of the configuration is substantially the same as that of the above-described first embodiment, so the description thereof will be simplified. In the mobile phone of this example, the program storage unit of the storage unit 4 stores a display control program executed by the control unit 3. This display control program includes a procedure for displaying the e-mail screen and the message memo display screen in a divided manner by a predetermined operation, and copying contents specified in the range of the message memo display screen in a specified copy on the e-mail screen. The procedure of copying and displaying at the position is described.

Next, the operation of the mobile phone 1 of this example will be described with reference to FIGS. In this example, when the message memo of the recognition result of the voice recognition is displayed on the main display unit 9, the control unit 3 simultaneously displays the contents of the message memo on the message memo display screen as shown in FIG. A copy key 9a for designating a copy location, a selection key 9b for confirming the copy location, and a move key 9c for moving to, for example, an electronic mail screen are displayed. When the user confirms the contents of the displayed message memo and determines that the contents are urgent and needs to be transmitted to another person, and tries to compose a new e-mail, After pressing the menu key 8c, the cursor key 8e or the like is pressed to perform a predetermined operation.

As a result, the control section 3 causes the main display section 9 to display the sub-menu screen as shown in FIG. When the user moves the cursor display position with the cursor key 8e and selects "Create New Mail", the control unit 3 causes the main display unit 9 to display a new mail input screen as shown in FIG. After the user inputs the title and the destination, the control unit 3 causes the main display unit 9 to display the text input screen as shown in FIG. As shown in the figure, the control unit 3 displays, on the text input screen, a copy key 9d for designating the copy destination on the text input screen and a selection key for confirming the copy destination, together with the input text. 9e, a move key 9f for moving to the message memo display screen, a screen split key 9g for splitting the text input screen and the message memo display screen, a send key 9h, and a save key 9i as main displays. It is displayed on the section 9. Here, when the user selects the screen division key 9g, the control unit 3 causes the main display unit to divide and display the text input screen and the message memo display screen, as shown in FIG. The user selects the copy key 9a on the message memo display screen.
After selecting, the copy range of the predetermined display area is specified by using the cursor key 8i, and is confirmed by the selection key 9b. Next, when the user specifies the copy destination position on the text input screen and selects the selection key 9e, the copy content is copied. Then, the content of the text is transmitted by selecting the transmission key 9i.

According to the configuration of this example, it is possible to obtain substantially the same effects as those described in the first embodiment. In addition, since the content of the message memo can be copied and quoted as it is in the body of the e-mail to edit the body, the body of the e-mail can be created accurately and quickly.

Fourth Embodiment FIG. 17 is a block diagram showing the configuration of a voice recognition unit of a folding type mobile phone according to a fourth embodiment of the present invention, and FIG.
FIG. 8 is a flowchart for explaining the operation of the mobile phone. This example is largely different from the above-described first example in that the caller is identified, the caller information is obtained, and the voice recognition unit performs the specific speaker voice recognition based on the caller information. This is the point of composition. Other configurations are
Since it is substantially the same as the above-mentioned first embodiment, its explanation is simplified.

In the portable telephone of this example, the program storage section of the storage section 4 stores a voice recognition control program executed by the control section 3. The voice recognition control program in this example determines whether the caller is the preset voice recognition target person after the incoming call and the message memo are recorded. A procedure for causing the section to perform a specific speaker voice recognition process and displaying the recognition result on the display section by a predetermined reproduction operation is described. In the personal information storage area of the information storage unit of the storage unit 4, a voice recognition target person is marked in the column of voice recognition target person. In addition, in this example, the voice recognition unit 7A
As shown in FIG. 17, an acoustic analysis processing unit 31 for obtaining a short time spectrum of a voice, a standard pattern storage unit 32 for storing a time series of a short time spectrum of a word voice as a standard pattern, and an input voice of a specific speaker. It has a word recognition processing unit 33 that performs matching (for example, DP matching) between the time series of the short-time spectrum and the standard pattern to directly calculate the similarity and outputs the recognition result.

Next, the operation of the mobile phone 1 of this example will be described with reference to FIG. First, the voice recognition target person is set in advance. For example, a specific person who makes a call relatively frequently is registered as a voice recognition target person, and when an incoming call is received from this voice recognition target person, voice recognition processing is executed on the contents of the message memo. Keep it. Further, the learning voice of the voice recognition target person as the specific speaker is input to the voice recognition unit 7A, and the standard pattern is registered. In this example, when the voice recognition target person is changed, the standard pattern is updated. Next, the message memo is set by the method used in the first embodiment.

The control unit 3 monitors the presence or absence of an incoming call (step ST31 (FIG. 18)), and when detecting the presence of an incoming call, sends a response message prompting the caller to make a message via the wireless transmission unit 6. To do. Next, the control unit 3 checks whether or not the sender is sending a message (step ST32), and if it is sending a message, temporarily stores the message content in the storage unit 4 (step ST33). Next, the control unit 3 checks the telephone directory information in the storage unit 4 based on the caller's telephone number, and determines whether the caller is a voice recognition target person (step S
(T34), if the caller is a voice recognition target person, the process proceeds to step ST35. In step ST35, the control unit 3
Causes the voice recognition unit 7 to perform voice recognition processing. Next, the control unit 3 monitors whether or not a reproduction operation is performed (step S
T36), when the user who confirms the message with the message performs a predetermined reproduction operation, the recognition result of the message memo is displayed on the main display section 9 (step ST37). On the other hand, when the control unit 3 determines in step ST34 that the caller is not the voice recognition target person, the process proceeds to step ST38 to monitor whether or not the reproduction operation is performed, and confirm the message with the message. Performs a predetermined reproduction operation, the contents of the message memo are output by voice (step ST39).

According to the configuration of this example, it is possible to obtain substantially the same effect as that described in the first embodiment. In addition, the structure of the voice recognition unit can be simplified and the cost can be reduced.

The embodiment of the present invention has been described in detail above with reference to the drawings. However, the specific structure is not limited to this embodiment, and the design change and the like without departing from the gist of the present invention. Even this is included in this invention. For example, in the above-described embodiment, the case where the mobile phone is used as the mobile terminal has been described, but not limited to the mobile phone, for example,
A simple mobile phone (PHS) terminal, a personal digital assistant (PDA) having a wireless communication function, or a videophone may be used, and the same effect as that of the mobile phone can be obtained. Further, in the case of a mobile phone, the mobile phone does not necessarily have to be a foldable mobile phone. , Mobile phone 1
In the above, the case where the mode is used in a state where the sound is muted by selecting the manner mode or the like has been described, but it is also possible to output the voice at the same time as displaying the message content. This makes it possible to recognize the message content more reliably.

The auxiliary display unit 1 is not limited to the main display unit 9.
The contents of the message memo may be displayed in 1. In the first embodiment, the case of using a plurality of standard patterns to recognize the voices of an unspecified number of speakers has been described, but a method using an average feature may be used. Also, as the standard pattern, not the time series itself of the short-time spectrum of the word speech, but the short-time spectrum of the phoneme is registered, and the word similarity is calculated between the phoneme recognition result and the phoneme symbol series of the word dictionary. You can Instead of the standard pattern of phonemes, standard patterns such as syllables, semi-syllables, and VCV syllables may be used.

Further, for example, the voice recognition result may be output in katakana without being converted into kanji. Also,
The voice recognition result may be used as visualization information not only in characters but also in motion such as sign language in a moving image. In addition to the received voice information, the user may record a voice, convert it into text information, and quote it for use in the body of the e-mail. At this time, the standby voice memo screen is displayed as shown in FIG. 19 to record the voice of oneself. This allows emails to be created quickly and easily. In addition, in the third embodiment, the case where the screen is divided into the electronic mail screen and the message memo display screen is described, but it may be an interrupt screen, or the full screen can be displayed separately by switching. Is also good.

Further, although the case where the main display portion and the auxiliary display portion are configured by using the transmissive liquid crystal display device has been described, it may be configured by using the reflective liquid crystal display device. In this case, as a lighting device, a front light is used instead of the backlight. The light source of the lighting device is not limited to the LED, and may be, for example, a fluorescent lamp. Also,
Although the case where the main display unit and the auxiliary display unit are configured by using the transmissive liquid crystal display device has been described, the main display unit and the auxiliary display unit may be configured by using the reflective liquid crystal display device. In this case, as a lighting device, a front light is used instead of the backlight. Although the case where the liquid crystal display element is used as the main display portion and the auxiliary display portion has been described, for example, a self-luminous display using LEDs may be used. As the self-luminous display, a display using an electroluminescence element, a display using a fluorescent display tube, a plasma display, or a field electron emission display may be used.

[0051]

As described above, according to the present invention, the content of the recorded message memo is converted into character image information and displayed on the display means.
It is possible to confirm the reproduced content displayed on the display unit without giving discomfort to the surrounding people and without worrying about the surrounding people. Especially for deaf people,
You can easily check the message content. Also, since the reproduced content can be known visually, the message content can be recognized more reliably than by hearing, and the convenience can be improved. Further, by storing the character image information in the storage means instead of the voice information, the memory capacity of the storage means can be saved and the memory resources can be effectively used.

[Brief description of drawings]

FIG. 1 is a perspective view showing a configuration of a folding type mobile phone according to a first embodiment of the present invention, which is a front view of a state in which the housing of the mobile phone is opened.

FIG. 2 is a perspective view showing a configuration of the mobile phone, which is a view of a state in which the housing of the mobile phone is opened, viewed from the back side.

FIG. 3 is a perspective view showing a configuration of the mobile phone, showing a state in which a housing of the mobile phone is closed.

FIG. 4 is a block diagram showing a configuration of the mobile phone.

FIG. 5 is a block diagram showing a configuration of a voice recognition unit of the mobile phone.

FIG. 6 is a flowchart for explaining the operation of the mobile phone.

FIG. 7 is a diagram showing a screen display example of a main display unit of the mobile phone.

FIG. 8 is a diagram showing a screen display example of a main display unit of the mobile phone.

FIG. 9 is a diagram showing a screen display example of a main display unit of the mobile phone.

FIG. 10 is a diagram showing a screen display example of a main display unit of the mobile phone.

FIG. 11 is a flowchart for explaining the operation of the folding type mobile phone according to the second embodiment of the present invention.

FIG. 12 is a diagram showing a screen display example of a main display unit of the folding type mobile phone according to the third embodiment of the present invention.

FIG. 13 is a diagram showing a screen display example of the main display unit.

FIG. 14 is a diagram showing a screen display example of the main display unit.

FIG. 15 is a diagram showing a screen display example of the main display unit.

FIG. 16 is a diagram showing an example of a screen display on the main display unit.

FIG. 17 is a block diagram showing a configuration of a voice recognition unit of a folding type mobile phone according to a fourth embodiment of the present invention.

FIG. 18 is a flowchart for explaining the operation of the mobile phone.

FIG. 19 is a diagram showing an example of a screen display of a main display unit of a foldable mobile phone which is a modification of the first embodiment of the present invention.

FIG. 20 is an explanatory diagram for explaining a conventional technique.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 Mobile phone (portable terminal) 3 Control part (a part of conversion means, quotation means, display control means) 4 Storage part (a part of conversion means, storage means) 6 Wireless communication part (communication means) 7 Voice recognition part ( Part of conversion means, character image information output means) 9 Main display section (display means) 11 Auxiliary display section

─────────────────────────────────────────────────── ─── Continuation of front page (51) Int.Cl. ⁷ Identification code FI theme code (reference) H04M 1/725 G10L 3/00 551A H04Q 7/38 571D F term (reference) 5D015 KK02 LL05 LL10 5K027 AA11 BB01 CC08 FF01 FF22 HH20 MM17 5K039 BB04 EE01 JJ09 5K067 AA21 BB04 BB21 DD51 EE02 FF02 FF23 FF31 GG11

Claims (16)

[Claims] 1. A method of converting voice information received by a mobile terminal having a call function into corresponding character image information, the voice information storing step of storing the received voice information,
And a character image information output step of converting predetermined voice information into associated character image information and outputting the converted character image information. 2. The character image information output step is executed after the voice information storage step is executed and before the stored content stored by the execution of the voice information storage step is reproduced. The method for converting received voice information into character image information in the mobile terminal according to claim 1, wherein 3. The portable terminal according to claim 1, wherein the character image information output step is executed after a reproduction operation of the stored content stored by the execution of the voice information storage step is performed. A method of converting received voice information into text image information. 4. The received voice information in the mobile terminal according to claim 1, further comprising a display control step of displaying on the display means the character image information output in the character image information output step. How to convert to text image information. 5. The mobile terminal according to claim 4, wherein the mobile terminal includes a foldable housing, and the display means is disposed on a side that is an inner side when folded. Method for converting received voice information to text image information in. 6. The mobile terminal according to claim 4, wherein the mobile terminal is provided with a foldable housing, and the display means is disposed on a side that is an outer side when folded. Method for converting received voice information to text image information in. 7. The method according to claim 1, further comprising a quoting step of quoting the character image information output in the character image information output step to edit a predetermined sentence.
5. A method for converting received voice information into character image information in the mobile terminal according to any one of 1. 8. The received voice information from the received voice information to the character image information in the mobile terminal according to claim 1, wherein in the character image information output step, unspecified speaker voice recognition is performed. How to convert. 9. A caller identification step of obtaining a caller information by identifying a caller after receiving the voice information, wherein the character image information output step includes a specific speaker voice recognition based on the caller information. The method according to claim 1, wherein
7. A method of converting received voice information into character image information in the mobile terminal according to any one of items 1 to 7. 10. A mobile terminal comprising: a communication unit for receiving voice information; and a conversion unit for converting the received voice information into corresponding character image information, wherein the conversion unit converts the received voice information. A method for converting received voice information into character image information, characterized by having storage means for storing and character image information output means for converting predetermined voice information into associated character image information for output. A mobile terminal having the function of performing. 11. The conversion from received voice information to character image information according to claim 10, further comprising display control means for displaying the character image information output from said character image information output means on a display means. A mobile terminal having the capability of performing the method. 12. The received voice information to character image information according to claim 11, further comprising a foldable housing, wherein the display means is arranged on an inner side when folded. A mobile terminal having a function of executing the conversion method. 13. The received voice information to the character image information according to claim 11, wherein the display means is provided with a foldable housing, and the display means is arranged on a side which becomes an outer side at the time of folding. A mobile terminal having a function of executing the conversion method. 14. A quoting unit for quoting the character image information output from the character image information output unit to edit a predetermined sentence is provided.
A mobile terminal having a function of executing the method for converting received voice information to character image information according to any one of 1 to 13. 15. The character image information output means performs unspecified speaker voice recognition.
A mobile terminal having a function of executing the method of converting received voice information to character image information according to any one of 4 above. 16. A caller identifying means for identifying a caller who sent the voice information to obtain caller information, wherein the character image information output means recognizes a specific speaker voice based on the caller information. The method according to claim 1, wherein
A mobile terminal having a function of executing the method of converting received voice information into character image information according to any one of 0 to 14.