JP2005274791A - Mobile communication terminal - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To enable a reader to better understand a document resulting from converting voice information to text information, without using images. <P>SOLUTION: A voice information measurement part 104 measures sound volumes and voice tones from a voice signal transmitted from a mobile phone 102 and a voice signal inputted by a user of a mobile phone 101. A voice/text conversion part 105 uses a voice recognition technique to convert voice information outputted from a transmission/reception part 103 to text information being a string of digital character codes. A rich text generation part 106 adds a measurement result corresponding to text information to the text information to generate rich characters on the basis of the text information outputted from the voice/text conversion part 105 and the measurement result outputted from the voice information measurement part 104. A display part 108 displays series of generated rich characters as a rich text while representing sound volumes by sizes of characters and representing heights of tones by line thickness of characters. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a mobile communication terminal apparatus having a speech text conversion device that converts speech information into text information.

  As a conventional speech-to-text converter, the technique described in Patent Document 1 is disclosed. The technique described in Patent Document 1 generates attribute information for identifying a speaker based on input speech, and displays the document converted from speech information to text information and the attribute information so as to be visually understood. It is. Thereby, the reader of a document can understand a document more clearly, confirming a voice input person.

  Patent Document 2 discloses that voice information is converted into text information and character input such as an e-mail is performed in order to eliminate the troublesomeness of inputting characters with a mobile phone or the like having a small key arrangement. .

Also, in Patent Document 3, message data is organized from the voice of a message sender and proxy image data that transmits a message message on behalf of the message sender, and the emotion set by the input operation of the message sender Since it is selected on the basis of the quantified parameters, it is disclosed that a message message by voice and image is sent, thereby making it possible to communicate intentionally.
Japanese Patent Laid-Open No. 11-242669 JP 2002-344574 A Japanese Patent Laid-Open No. 2002-041279

  However, in the techniques described in Patent Documents 1 and 2, the document converted from the voice information to the text information does not include many elements for reading the emotion of the speaker, and performs high-level and smooth communication. Not suitable for. Further, the technique described in Patent Document 3 requires a large amount of image data to be prepared in advance, resulting in a problem that the memory capacity increases.

  The present invention has been made in view of this point, and an object of the present invention is to provide a mobile communication terminal device that allows a reader to better understand a document converted from voice information to text information without using an image.

  The mobile communication terminal apparatus according to the present invention includes a voice measurement unit that measures a volume and a tone based on input voice information, a voice text conversion unit that converts the input voice information into text information, and the voice text conversion unit. Generating means for generating rich text using a combination of the converted text information and the volume and tone measured by the voice measuring means as parameters for modifying characters, and display means for displaying the rich text generated by the generating means The structure which comprises these is taken.

  According to this configuration, since the sound information can be visually displayed by displaying the volume and tone measured from the sound information with the modification of the characters, the reader can understand the content of the conversation more. .

  The mobile communication terminal device according to the present invention is transmitted from the communication partner, receiving means for receiving voice information and physical information transmitted from the communication partner, detection means for detecting the physical information of the user of the user device, in the above configuration. Physical information analyzing means for analyzing the physical information detected by the detecting means and the physical information detected by the detecting means, and the generating means is analyzed by the physical information analyzing means in addition to the combination of the volume and the tone. A configuration is adopted in which rich text is generated using the result as a parameter for modifying characters.

  According to this configuration, by displaying the body information by modification of characters, the psychological situation of the speaker can be visually displayed, and the reader can further understand the content of the conversation.

  The mobile communication terminal device of the present invention employs a configuration in which, in the above configuration, the physical information is at least one of body temperature, blood pressure, and heart rate.

  According to this configuration, by displaying the body information by modification of characters, the psychological situation of the speaker can be visually displayed, and the reader can further understand the content of the conversation.

  As described above, according to the present invention, the voice information can be visually displayed by displaying the voice information such as the magnitude of the voice of the speaker and the pitch of the voice by character modification. The reader can understand the content of the conversation without using it.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

(Embodiment 1)
FIG. 1 is a block diagram showing a configuration of mobile phone 101 according to Embodiment 1 of the present invention. In this figure, a mobile phone 101 performs a voice call with a mobile phone 102. Here, the configuration of the mobile phone 101 will be described, but the mobile phone 102 may or may not have the same configuration as the mobile phone 101.

  The transmission / reception unit 103 receives the voice information transmitted from the mobile phone 102, performs a predetermined reception process on the received voice information, and outputs the voice information after the reception process to the voice information measurement unit 104 and the voice text conversion unit 105. To do. In addition, the voice input by the user of the mobile phone 101 is output to the voice information measurement unit 104 and the voice text conversion unit 105.

  The audio information measurement unit 104 measures the audio signal (audio information) output from the transmission / reception unit 103. Specifically, the sound volume (voice level) and tone (voice level) are measured, and the measurement result is converted into a digital multi-value. Details of the voice information measuring unit 104 will be described later. The measurement result is output to the rich text generation unit 106.

  The voice text conversion unit 105 converts the voice information output from the transmission / reception unit 103 into a digital character code string (hereinafter referred to as “text information”) using a voice recognition technique. The text information is output to the rich text generation unit 106. In addition, it determines the paragraph or voice break and outputs the end-of-line code to the rich text generator 106.

  Based on the text information output from the speech text conversion unit 105 and the measurement result output from the speech information measurement unit 104, the rich text generation unit 106 adds a measurement result corresponding to the text information to the text information. To generate a rich character. In this way, the text information (character code) sequentially input to the rich text generation unit 106 is changed to a rich character in the input order. A series of rich characters generated in this way is called rich text. When the rich text generation unit 106 acquires the end-of-line code from the voice text conversion unit 105, the rich text generation unit 106 outputs the rich text to the log storage unit 107.

  The log storage unit 107 adds a time stamp (time information) to the rich text output from the rich text generation unit 106 and stores the rich text.

  The display unit 108 reads the rich text and the time stamp from the log storage unit 107, converts the volume level to the size of the character, and converts the tone level to the thin text.

  FIG. 2 is a block diagram showing an internal configuration of audio information measuring section 104 in Embodiment 1 of the present invention. In this figure, the volume measuring unit 201 measures the volume (volume of voice) from the audio signal (audio information) output from the transmission / reception unit 103, and the measured value is a digital multivalue (for example, 16 levels from 0 to 15). And output to the measurement information storage unit 203.

  The tone measurement unit 202 measures the tone (voice level) from the voice signal (voice information) output from the transmission / reception unit 103, converts the measurement value into a digital multivalue (for example, 16 levels of 0 to 15), Output to the measurement information storage unit 203.

  The measurement information storage unit 203 is a dual port RAM (DPRAM) or the like, stores the measurement values output from the volume measurement unit 201 and the tone measurement unit 202, and the stored measurement values are read from the rich text generation unit 106. .

  Next, operations of the voice information measurement unit 104 and the rich text generation unit 106 will be described. Here, as a specific example, it is assumed that the audio information of "Hello" to the rich text. Audio information measuring unit 104, the audio information input "Hello", respectively by character-volume measuring unit 201 and the measure of tonality unit 202 measures the volume and tone. The measured volume and tone are stored in the measurement information storage unit 203 as digital values.

  Rich Text generation unit 106 acquires the "Hello" became character codes (text data) from the audio text conversion unit 105, for each character, i.e. the volume and tone audio information measuring unit 104 corresponding to each one of character codes Is read from the measurement information storage unit 203. Here, a specific example of the volume and tone corresponding to each character (character code) is shown in FIG. Note that. The volume and tone are divided into 16 levels from 0 to 15. In FIG. 3, “ko” is volume 4, tone 3, “n” is volume 3, tone 2, “ni” is volume 3, tone 2, “chi” is volume 3, tone 2, and “ha” is volume 2. In this case, the tone 3 is shown. A rich character whose volume and tone are combined with a character code is stored in the log storage unit 107 with an end-of-line code and a time stamp added thereto. Examples of rich characters and rich text are as shown in FIG. In FIG. 4, char_code represents a character code, volume represents a volume, and tone represents a tone.

  FIG. 5 is a diagram illustrating a display example of rich text in the first embodiment. This display example shows a conversation record (log) between A who has called the customer service and B who is in charge of the customer service. Here, the volume and tone are 16 levels from 0 to 15, and the corresponding numerical value is reduced when the volume is low, and the corresponding numerical value is increased when the volume is high. If the tone is low, the corresponding numerical value is also reduced, and if the tone is high, the corresponding numerical value is increased.

  In this figure, the first to fifth lines indicate that the volume and tone of A are both medium (for example, volume 6, tone 6), and the volume of B is medium (for example, volume 6). This indicates that the tone is slightly low (for example, tone 2).

  The sixth line shows that A who has been angry with B's response has made a loud voice (eg, volume 15, tone 2). The seventh line shows that B, who has shrunk to A's anger, has produced a small voice (eg, volume 2, tone 5). The eighth line shows that A, who has somewhat recovered calmness, has uttered a voice with a somewhat lower volume (eg, volume 13, tone 2). The ninth line shows that B, which has been atrophied, still corresponds with a small voice (for example, volume 2 and tone 5).

  As this figure shows, not only the content of the conversation is documented, but also the third person who reads the log from the sentence alone by visually displaying the loudness and height of the speaker's voice with modifier characters This makes it easy to feel a sense of reality that cannot be read, and to guess the emotions of the speaker.

  As described above, according to the present embodiment, the voice information can be visually displayed by expressing the magnitude of the voice of the speaker with the magnitude of the letters and expressing the pitch of the voice with the thin letters. The reader can better understand the content of the conversation.

  In the present embodiment, the case where the speech text conversion unit converts Japanese speech information into Japanese rich text has been described, but the speech text conversion unit may include a translation function. That is, for example, Japanese speech information may be converted into English rich text.

(Embodiment 2)
In the first embodiment, the volume and tone are parameterized and displayed with the size of the character and the thickness. However, in the second embodiment of the present invention, a case where a parameter that conveys emotion other than the voice is used will be described.

  FIG. 6 is a block diagram showing a configuration of mobile phone 601 according to Embodiment 2 of the present invention. However, the parts in FIG. 6 that are the same as those in FIG. 1 are given the same reference numerals as those in FIG. 6 differs from FIG. 1 mainly in that a sensor unit 604 and a body information analysis unit 605 are provided. Here, the configuration of the mobile phone 601 will be described, but the configuration of the mobile phone 602 is the same as that of the mobile phone 601.

  FIG. 6 shows that the mobile phone 601 and the mobile phone 602 perform data communication simultaneously with the voice call.

  The transmission / reception unit 603 receives the voice information transmitted from the mobile phone 602, performs a predetermined reception process on the received voice information, and outputs the voice information after the reception process to the voice information measurement unit 104 and the voice text conversion unit 105. To do. In addition, the voice input by the user of the mobile phone 601 is output to the voice information measuring unit 104 and the voice text converting unit 105. Furthermore, it receives data transmitted from the mobile phone 602 by data communication, performs a predetermined reception process on the received data, and outputs the data after the reception process to the physical information analysis unit 605.

  The sensor unit 604 measures the body temperature, blood pressure, heart rate, and the like (hereinafter collectively referred to as “body information”) of the user of the mobile phone 601, and outputs the measurement result to the body information analysis unit 605.

  The physical information analysis unit 605 acquires the physical information of the user of the mobile phone 602 included in the data output from the transmission / reception unit 603 and the physical information output from the sensor unit 604, and the measured value is digital multivalued (for example, , 0 to 15 steps), and outputs the analysis result to the rich text generation unit 606.

  The rich text generation unit 606 generates text based on the text information output from the speech text conversion unit 105, the measurement result output from the speech information measurement unit 104, and the analysis result output from the body information analysis unit 605. A rich character is generated by adding a result corresponding to the information to the text information. When the rich text generation unit 606 acquires the end-of-line code from the speech text conversion unit 105, the rich text generation unit 606 outputs the rich text to the log storage unit 107.

  The display unit 607 reads the rich text and the time stamp from the log storage unit 107, converts the volume level to the size of the character, converts the tone level to a thin character, and converts the physical information to the color of the character and displays it. . For example, when the body temperature, blood pressure, and heart rate are both high, the color is red. Conversely, when the body temperature, blood pressure, and heart rate are both low, the color is blue. As a result, the reader can infer whether the speaker is excited or calm.

  As described above, according to the present embodiment, in addition to the volume and tone of the voice uttered by the speaker, the body information such as the body temperature, blood pressure, and heart rate of the speaker is digitally multi-valued, and the body information is displayed in the character color. By doing so, the psychological situation of the speaker can be visually displayed, and the reader can further understand the content of the conversation.

  In the present embodiment, the character color may be displayed based on at least one of body temperature, blood pressure, and heart rate, which are body information. In the present embodiment, body temperature, blood pressure, and heart rate are listed as the body information. However, the present invention is not limited to this, and the user may add a force for holding the mobile phone, the amount of sweat, and the like. In short, any parameter that reflects the psychological situation of the speaker may be used.

  Further, in the present embodiment, it has been described that the volume is displayed in correspondence with the size of the character, the tone is displayed in a thin character, and the body information is displayed in correspondence with the color of the character, but the present invention is not limited thereto, Each may be arbitrarily combined. Furthermore, in addition to volume, tone, and body information, voice quality may be added, and fonts may be added in addition to the size of characters, thickness, and color.

  In each of the above-described embodiments, description has been made assuming that conversation records and minutes are created, but the present invention can also be applied to creation of e-mails.

  The mobile communication terminal device according to the present invention can visually display the voice information by reflecting the voice information such as the loudness of the speaker and the pitch of the voice in the modification of the character, so that the reader can It has the effect of being able to understand the content more and is useful for creating conversation records and minutes.

1 is a block diagram showing a configuration of a mobile phone according to Embodiment 1 of the present invention. The block diagram which shows the internal structure of the audio | voice information measurement part in Embodiment 1 of this invention. The figure which shows the specific example of the volume and tone corresponding to each character (character code) The figure which shows the example of a rich character and rich text The figure which shows the example of a display of the rich text in this Embodiment 1. Block diagram showing a configuration of a mobile phone according to a second embodiment of the present invention

Explanation of symbols

101, 102 Cellular phone 103, 603 Transmission / reception unit 104 Audio information measurement unit 105 Audio text conversion unit 106, 606 Rich text generation unit 107 Log storage unit 108, 607 Display unit 201 Volume measurement unit 202 Tone measurement unit 203 Measurement information storage unit 604 Sensor unit 605 Body information analysis unit

Claims (3)

Voice measuring means for measuring volume and tone based on the input voice information;
Voice text conversion means for converting the input voice information into text information;
Generating means for generating rich text using the text information converted by the voice text converting means and the combination of the volume and tone measured by the voice measuring means as parameters for modifying characters;
Display means for displaying the rich text generated by the generating means;
A mobile communication terminal device comprising: Receiving means for receiving voice information and physical information transmitted from a communication partner;
Detecting means for detecting physical information of the user of the device;
Physical information analyzing means for analyzing physical information transmitted from a communication partner and physical information detected by the detecting means;
Further comprising
2. The mobile communication according to claim 1, wherein the generation unit generates a rich text by using a result analyzed by the body information analysis unit in addition to a combination of the volume and a tone as a parameter for modifying a character. Terminal device.   The mobile communication terminal apparatus according to claim 2, wherein the physical information is at least one of body temperature, blood pressure, and heart rate.

Images