JP2015064446A - Communication device, communication method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To make a response faster by reducing time and effort of character input of a text, in communication of talking with a voice and a text.SOLUTION: A communication device 10 includes: a receiving unit 106 for receiving input from a user; an obtaining unit 1131 for obtaining voice information; a converting unit 1132 for converting the voice information obtained by the obtaining unit 1131 into semantic information; a display unit 105 for displaying information based on the semantic information converted by the converting unit 1132; a generating unit 112 for extracting an interrogative sentence from the semantic information converted by the converting unit 1132, generating options selected by the user on the basis of the extracted interrogative sentence to display that in the display unit 105, and generating information related to a response to the semantic information on the basis of the option selected by the used from the displayed options and received by the receiving unit 106; and a transmitting unit for transmitting the information related to the response which is generated by the generating unit 112 to another device.

Description

  The present invention relates to a communication device, a communication method, and a program.

  At present, many portable terminal devices have functions such as mail and Internet connection in addition to telephone communication. In the future, a voice call service using data packets such as VoLTE (Voice Over LTE (Long Term Evolution)) will be started. At present, when a conversation is performed using a portable terminal device, in a voice call, a caller who makes a call and a callee who receives the call have a conversation. Moreover, in a portable terminal device, it is also possible to have a conversation by inputting characters instead of voice calls using mail, chat, or the like.

  If the caller receives an incoming call from a caller while traveling on a public transport such as a train, the voice call should be refrained as a public manner, so it is common to hold the call without making a call. If it is determined that the caller cannot make a voice call, the caller can end the phone call and enter characters by e-mail, chat, or the like to have a conversation with the caller. However, in mail, chat, etc., there is a problem that it takes time to input characters for both the calling party and the called party.

  Therefore, as a conversation method according to the situation of the other party, there is one that enables communication between a healthy person and a person with an ear disorder by enabling mutual compatibility between voice and text format (see, for example, Patent Document 1). ). In the technology described in Patent Document 1, a conversation between a healthy person and a person with a hearing impairment is performed by converting speech spoken by a healthy person into text and displaying it on a display of a portable terminal device of a person with a hearing impairment. Is realized.

JP 2003-188948 A

  However, with the technique described in Patent Document 1, a person with an ear disorder must respond by inputting all text in the same manner as in email and chat, so the response to a healthy person is inevitably slow. End up. In addition, even in the case of a call between healthy people, if the caller is in an environment where voice call is not possible, the caller must respond by inputting text even if the caller's voice is converted to text and displayed. Therefore, there is a problem that the response to the caller is delayed.

  The present invention has been made in view of the above points, and an object of the present invention is to provide a communication apparatus and communication that can reduce the time and effort of inputting text and speed up a response in communication in which voice and text are conversed. It is to provide a method and a program.

  The present invention has been made to solve the above-described problems, and one aspect of the present invention is acquired by a receiving unit that receives input from a user, an acquiring unit that acquires audio information, and the acquiring unit. A conversion unit that converts speech information into semantic information; a display unit that displays information based on the semantic information converted by the conversion unit; and a question sentence extracted from the semantic information converted by the conversion unit, Based on the sentence, an option to be selected by the user is generated and displayed on the display unit, the option selected by the user from the displayed options, and based on the option received by the reception unit, A communication apparatus comprising: a generation unit that generates information related to a response to the semantic information; and a transmission unit that transmits information related to a response generated by the generation unit to another device.

  ADVANTAGE OF THE INVENTION According to this invention, in the communication which carries out a conversation with an audio | voice and a text, the effort of inputting the character of a text can be reduced and a response can be made quick.

It is a front view which shows the external appearance structure of the communication apparatus by embodiment of this invention. It is a block diagram which shows the structure of the communication apparatus by this embodiment. It is a sequence diagram which shows the operation | movement of the telephone call process in the communication apparatus by this embodiment. It is an image figure which shows an example of the option which the communication apparatus by this embodiment displays. It is a sequence diagram which shows the operation | movement of the telephone call process in the communication apparatus by this embodiment. It is an image figure which shows an example of the character input screen which the communication apparatus by this embodiment displays. It is a flowchart which shows the procedure of the call control process in the communication apparatus by this embodiment.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. FIG. 1 is a front view showing an external configuration of a communication device 10 according to the present embodiment. The communication device 10 is an electronic device such as a mobile phone, a smartphone, or a tablet terminal. The communication device 10 has a voice call function by data communication such as VoLTE, for example. The communication device 10 includes a display unit 105 and a reception unit 106. The display unit 105 is an LCD (Liquid Crystal Display) or the like, and displays information. The receiving unit 106 includes a button (key), a touch panel that detects contact on the screen of the display unit 105, and the like, and receives an operation input from the user.

  Next, the configuration of the communication device 10 according to the present embodiment will be described. FIG. 2 is a block diagram illustrating the configuration of the communication apparatus 10 according to the present embodiment. The communication device 10 includes a control unit 101, a transmission / reception unit 102, a voice input unit 103, a voice output unit 104, a display unit 105, and a reception unit 106.

  The transmission / reception unit 102 is a communication unit that wirelessly communicates with other devices. For example, the transmission / reception unit 102 receives audio information from another communication device 10. The voice information is voice data related to a call. In addition, the transmission / reception unit 102 transmits information related to the response to the semantic information to the other communication device 10. The semantic information is text data obtained by converting the voice indicated by the voice information into text (characters). The voice input unit 103 is a microphone or the like and inputs voice. The audio output unit 104 is a speaker or the like and outputs audio.

  The control unit 101 controls each unit of the communication device 10 in an integrated manner. The control unit 101 includes a generation unit 112, a speech / speech conversion unit 113, and a semantic speech conversion unit 114. The speech meaning conversion unit 113 includes an acquisition unit 1131 and a conversion unit 1132. The acquisition unit 1131 receives audio information from another communication device 10 via the transmission / reception unit 102. Then, the conversion unit 1132 converts the voice information received by the acquisition unit 1131 into semantic information. For example, the conversion unit 1132 converts voice information into semantic information by a voice recognition technique using a general sensitivity control technique. The conversion unit 1132 converts the voice input by the voice input unit 103 into semantic information. The conversion unit 1132 displays the converted semantic information on the display unit 105. The acquisition unit 1131 and the conversion unit 1132 are examples of “acquisition unit” and “conversion unit” in the claims, respectively.

  The generation unit 112 generates information related to the response to the semantic information based on the input from the user received by the reception unit 106 and the semantic information converted by the conversion unit 1132 of the speech meaning conversion unit 113. Specifically, first, the generation unit 112 extracts a question sentence from the semantic information. For example, the generation unit 112 determines a sentence including words (hereinafter referred to as “question words”) such as “? Or the production | generation part 112 may determine a question sentence with the speech recognition technique using a general sensitivity control technique. And the production | generation part 112 produces | generates the choice which a user selects based on the extracted question sentence. However, the generation unit 112 may display “who”, “when”, “where”, “what”, “why” within a predetermined number of characters before the extracted question word. When there are words such as “how” (hereinafter referred to as 5W1H words), an input field (character input screen) for the user to input characters is generated. And the production | generation part 112 produces | generates the information regarding the response to a semantic information based on the choice from a user, or the input from the user performed with respect to the character input screen. Then, the generation unit 112 transmits information regarding a response to the generated semantic information to the other communication device 10 via the transmission / reception unit 102.

  The semantic voice conversion unit 114 converts text data (semantic information) into voice information, and causes the voice output unit 104 to output voice related to the converted voice information. For example, the semantic speech conversion unit 114 converts the semantic information into speech information by a syllable connection method, a corpus base method, a large-scale corpus base method, or the like. The semantic voice conversion unit 114 and the voice output unit 104 are output units that convert semantic information into voice information and perform voice output.

  The display unit 105 displays information based on the semantic information acquired by the acquisition unit 1131 of the speech meaning conversion unit 113 and converted by the conversion unit 1132. The accepting unit 106 accepts an option selected by the user or an input from the user made on the character input screen.

  Next, with reference to FIGS. 3 to 6, a call method in the communication apparatus 10 according to the present embodiment will be described. 3 to 6 show the operation of the communication apparatus 10 when the called party is in a situation where a voice call cannot be made because he / she is moving in a public transportation system or the like. Hereinafter, for convenience of explanation, the communication device 10 possessed by the calling party is referred to as a communication device 10B, and the communication device 10 possessed by the called party is referred to as a communication device 10A.

FIG. 3 is a sequence diagram showing an operation of a call process in the communication device 10 according to the present embodiment.
First, the communication device 10B transmits a voice to the communication device 10A (step S101). When there is an incoming voice call from the communication device 10B, the communication device 10A displays that fact on the display unit 105 and notifies the user. The user uses the receiving unit 106 of the communication device 10A to perform an input operation for setting the character call mode. The character call mode is an operation mode for making a call using characters. For example, when there is an incoming voice call, the communication device 10A responds with either a “voice call mode” in which a voice call is made to the display unit 105 or a “character call mode” in which the callee responds with a call by text. Display the option to select. Then, the user selects any mode. Based on the input operation from the user accepted by the accepting unit 106, the communication device 10A responds to the communication device 10B with information indicating that the character call mode has been selected (step S102).

  Then, the communication device 10A and the communication device 10B start a call in the character call mode (step S103). At this time, the communication device 10 </ b> B inputs voice related to the user's call from the voice input unit 103. Then, the communication device 10B transmits audio information indicating the input audio to the communication device 10A. In addition, the communication device 10 </ b> B converts the speech information into semantic information by the conversion unit 1132 of the speech semantic conversion unit 113 and displays the converted semantic information on the display unit 105.

  The communication device 10B transmits voice information including the question sentence “Do you want to?”. Here, it is assumed that “Is it in time?” Is transmitted. (Step S104). When receiving the voice information, the communication device 10A converts the voice information into semantic information. Subsequently, the communication device 10A determines that the question section “Do you make it in time?” Includes the question sentence “Do you have time?” By the generation unit 112, and the semantic information “Does it in time?” At the same time, the option “Yes” and the option “No” generated by the generation unit 112 are displayed on the display unit 105. Then, the communication device 10 </ b> A accepts selection of options displayed on the display unit 105 by the accepting unit 106. Here, it is assumed that the user operates the receiving unit 106 and selects the option “Yes”. The communication device 10A transmits text data (information related to the response to the semantic information) indicating the selected option “Yes” to the communication device 10B (step S105). Upon receiving the text data, the communication device 10B displays “Yes” indicated by the text data on the display unit 105. At this time, the communication device 10 </ b> B converts the text data into speech by the semantic speech conversion unit 114 and pronounces (outputs) “Yes” from the speech output unit 104. Here, the communication device 10B displays “Yes” on the display unit 105, and further converts the sound into a voice by the semantic voice conversion unit 114 and causes the voice output unit to pronounce the sound. Control may be performed so that only one of the sound generations from the output unit is performed, or the user of the communication device 10B may select which control is performed.

  FIG. 4 is an image diagram showing an example of options displayed by the communication apparatus 10 according to the present embodiment. The screen shown in this figure is a screen that is displayed when the communication device 10A receives the voice information including the question sentence “Do you make it in time?” In step S104 described above. When the communication apparatus 10A receives the voice information including the question sentence “Do you make time?” Q1 and the choice “Yes” Y and the choice “No” N are displayed on the display unit. 105. The option “Yes” Y and the option “No” N can be selected by the reception unit 106. In this example, the communication apparatus 10 displays only the choice “Yes” Y and the choice “No” N on the display unit 105. However, the present invention is not limited to this, and a character input screen is provided so that a response other than the choice can be made. May be displayed on the display unit 105 together with the options.

  FIG. 5 is a sequence diagram showing an operation of a call process in the communication device 10 according to the present embodiment. The operations shown in this figure are “who”, “when”, “where”, “what”, “why”, “how”, etc. This is an operation when the 5W1H word is included in the question sentence of the semantic information.

  First, the communication device 10B transmits a voice to the communication device 10A (step S201). When there is an incoming voice call from the communication device 10B, the communication device 10A displays that fact on the display unit 105 and notifies the user. The user makes an input operation for switching to the character call mode to the accepting unit 106 of the communication device 10A from the choice of a response in the voice call mode or a response in the character call mode. Based on the input operation from the user accepted by the accepting unit 106, the communication device 10A responds to the communication device 10B with information indicating that the character call mode has been selected (step S202). Then, the communication device 10A and the communication device 10B start a call in the character call mode (step S203).

  The communication device 10B transmits voice information including the question sentence “What is necessary?” (Step S204). At this time, the communication device 10 </ b> B converts the speech information into semantic information by the conversion unit 1132 of the speech semantic conversion unit 113 and displays the converted semantic information on the display unit 105. When receiving the voice information, the communication device 10A converts the voice information into semantic information. Subsequently, the communication apparatus 10A displays a character input screen on the display unit 105 together with the question sentence “What is necessary?” Because the semantic information is a question sentence including “what” and “what?”. Communication device 10 </ b> A accepts character input on the character input screen by accepting unit 106. The user operates the reception unit 106 and inputs a response “question required” to the question sentence on the character input screen. 10 A of communication apparatuses transmit the text data (information regarding the response to a semantic information) which shows the response "a seal is required" input into the character input screen to the communication apparatus 10B (step S205). When the communication device 10B receives the text data, the communication device 10B displays “The seal is necessary” indicated by the text data on the display unit 105. At this time, the communication device 10 </ b> B converts the text data into speech by the semantic speech conversion unit 114 and pronounces (outputs) “the seal is necessary” from the speech output unit 104. As described above, the communication device 10B is controlled to perform both the display by the display unit 105 and the sound generation by the sound output unit 104. However, either the display on the display unit 105 or the sound output from the sound output unit is selected. You may control to perform only one side. Further, the communication device 10B may allow the user of the communication device 10B to select which one to control.

  FIG. 6 is an image diagram illustrating an example of a character input screen displayed by the communication device 10 according to the present embodiment. The screen shown in this figure is a screen displayed when the communication apparatus 10A receives voice information including the question sentence “What is necessary?” In step S204 described above. When the communication device 10A receives the voice information including the question sentence “What is necessary?”, The communication device 10A displays the question sentence “What is necessary?” Q2 and the character input screen A on the display unit 105. On the character input screen A, the accepting unit 106 can input characters such as “Needs a seal”.

  Next, a call control process for realizing the call method in the communication apparatus 10 according to the present embodiment will be described. FIG. 7 is a flowchart showing the procedure of the call control process in the communication apparatus 10 according to the present embodiment. The process shown in this figure is a process executed by the communication device 10A of the called party when making a call in the character call mode.

  First, the control unit 101 starts a call with the communication device 10 of the caller in the character call mode via the transmission / reception unit 102 (step S501). Subsequently, the acquisition unit 1131 of the voice meaning conversion unit 113 determines whether or not the transmission / reception unit 102 has received voice information from the communication device 10 of the caller (step S502). If the acquisition unit 1131 determines that no audio information has been received (step S502: No), the acquisition unit 1131 returns to the process of step S502. On the other hand, if the acquisition unit 1131 determines that the audio information has been received (step S502: Yes), the acquisition unit 1131 acquires the audio information received by the transmission / reception unit 102. Then, the conversion unit 1132 converts the audio information acquired by the acquisition unit 1131 into semantic information (step S503).

  Subsequently, the generation unit 112 of the control unit 101 determines whether or not a question sentence is included in the acquired semantic information (step S504). Specifically, the generation unit 112 determines that a question sentence is included when the semantic information includes a question word, and includes a question sentence when the semantic information does not include the question word. Judge that it is not. If the generation unit 112 determines that the question information is not included in the semantic information (step S504: No), the generation unit 112 displays the semantic information on the display unit 105 (step S505), and returns to the process of step S502.

  On the other hand, when it is determined that the semantic information includes a question sentence (step S504: Yes), the generation unit 112 determines whether the semantic information includes a predetermined word (for example, a 5W1H word). (Step S506).

  When the generation unit 112 determines that the predetermined word is not included in the semantic information (step S506: No), the generation unit 112 displays the option on the display unit 105 together with the received semantic information (step S507). The accepting unit 106 accepts an input for selecting an option displayed on the display unit 105. Then, the generation unit 112 determines whether or not the user has selected an option using the reception unit 106 (step S508). If the generation unit 112 determines that no option is selected (step S508: No), the generation unit 112 returns to the process of step S508. On the other hand, when the generation unit 112 determines that the option has been selected (step S508: Yes), the text data indicating the selection result (selected option) via the transmission / reception unit 102 (related to the response to the semantic information). Information) is transmitted to the communication device 10 of the caller (step S509).

  On the other hand, when determining that the semantic information includes a predetermined word (step S506: Yes), the generation unit 112 displays a character input screen on the display unit 105 together with the received semantic information (step S510). . The accepting unit 106 accepts character input for the character input screen displayed on the display unit 105. Then, the generation unit 112 determines whether the user has completed the character input through the reception unit 106 (step S511). If the generation unit 112 determines that the character input has not been completed (step S511: No), the process returns to the process of step S511. On the other hand, when the generation unit 112 determines that the character input is completed (step S511: Yes), the generation unit 112 transmits text data (information related to the response to the semantic information) indicating the input character to the communication device 10 of the caller. (Step S512).

  Subsequent to step S509 or step S512, the control unit 101 determines whether or not the call with the communication device 10 of the caller is terminated (step S513). If the control unit 101 determines that the call has not ended (step S513: No), the control unit 101 returns to the process of step S502. On the other hand, if the control unit 101 determines that the call has ended (step S513: Yes), the call control process ends.

  Thus, according to the present embodiment, the communication device 10 converts the voice information received from the caller's communication device 10 into semantic information and displays it on the display unit 105 when making a call in the character call mode. As a result, the calling party can convey the message to the called party while maintaining the voice call. That is, the caller does not need to activate mail or chat in order to terminate the call once and convey the message in text when the caller cannot make a voice call.

  In addition, when the received voice information includes a question sentence, the communication device 10 displays options on the display unit 105 together with the received semantic information. Then, the communication device 10 accepts selection of an option by the callee at the accepting unit 106, and transmits text data indicating the selected option to the communication device 10 of the caller. Thereby, the called party can respond to the question sentence from the calling party without inputting characters, and can speed up the response to the calling party.

  Further, when a 5W1H word is included in a question sentence including question words such as “??” or “Masu?”, The communication device 10 displays a character input screen on the display unit 105 together with the received semantic information. indicate. Then, the communication device 10 receives character input on the character input screen in the receiving unit 106 and transmits text data indicating the input characters to the caller's communication device 10. Thereby, the called party can respond to the question sentence from the calling party without making a voice.

Note that a part of the communication device 10 in the above-described embodiment, for example, the control unit 101 may be realized by a computer. In that case, the program for realizing the control function may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be read by a computer system and executed. Here, the “computer system” is a computer system built in the communication apparatus 10 and includes an OS and hardware such as peripheral devices. The “computer-readable recording medium” refers to a storage device such as a flexible medium, a magneto-optical disk, a portable medium such as a ROM and a CD-ROM, and a hard disk incorporated in a computer system. Furthermore, the “computer-readable recording medium” is a medium that dynamically holds a program for a short time, such as a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line, In such a case, a volatile memory inside a computer system serving as a server or a client may be included and a program that holds a program for a certain period of time. The program may be a program for realizing a part of the functions described above, and may be a program capable of realizing the functions described above in combination with a program already recorded in a computer system.
In addition, a part or all of the communication device 10 in the above-described embodiment may be realized as an integrated circuit such as an LSI (Large Scale Integration).
Each functional block of the communication device 10 may be individually made into a processor, or a part or all of them may be integrated into a processor. Further, the method of circuit integration is not limited to LSI, and may be realized by a dedicated circuit or a general-purpose processor. Further, in the case where an integrated circuit technology that replaces LSI appears due to progress in semiconductor technology, an integrated circuit based on the technology may be used.

  As described above, the embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to the above, and various design changes and the like can be made without departing from the scope of the present invention. It is possible to

For example, in the above-described embodiment, the caller's communication device 10 converts voice information into semantic information. However, the present invention is not limited to this, and the caller's communication device 10 converts voice information into semantic information. The converted semantic information may be transmitted to the communication device 10 of the called party. Alternatively, voice information may be converted into semantic information at a base station of a network connecting the calling party communication device 10 and the called party communication device 10.
Similarly, for conversion from semantic information to voice information, the caller communication device 10, the caller communication device 10, or the caller communication device 10 and the caller communication device 10 are connected. It may be performed at any of the base stations.

  In the embodiment described above, the communication device 10 displays the character input screen when the 5W1H word is included in the question sentence of the semantic information. However, the present invention is not limited to this. For example, “what time” If a fixed expression such as “1 o'clock”, “2 o'clock”,..., Etc. may be displayed. Specifically, if the semantic information includes a question sentence “What time will you arrive today?”, The communication device 10 may select the options “1 o'clock”, “2 o'clock”,. , “24 o'clock” is displayed on the display unit 105, and selection of an option is received by the receiving unit 106.

(1) According to one aspect of the present invention, a reception unit that receives input from a user, an acquisition unit that acquires voice information, a conversion unit that converts voice information acquired by the acquisition unit into semantic information, and the conversion A display unit for displaying information based on the semantic information converted by the unit, and extracting a question sentence from the semantic information converted by the conversion unit, and generating an option for the user to select based on the extracted question sentence A generating unit configured to generate information related to a response to the semantic information based on the option selected by the user from the displayed options displayed on the display unit and received by the receiving unit; And a transmission unit that transmits information related to the response generated by the generation unit to another device.

(2) Another aspect of the present invention is the communication device according to (1), in which the generation unit extracts a question sentence from the semantic information, and the user is based on the extracted question sentence. Generates an input field for inputting semantic information, and is an input from the user made to the input field for the user to input semantic information, based on the input received by the receiving unit A communication device that generates information related to a response to the semantic information.

(3) Moreover, the other aspect of this invention is a communication apparatus as described in (1) or (2), Comprising: The communication apparatus provided with the output part which converts the said semantic information into audio | voice information, and outputs an audio | voice. It is.

(4) Moreover, the other aspect of this invention receives the input from a user, acquires audio | voice information, converts the acquired audio | voice information into semantic information, displays the information based on the converted semantic information, The communication method generates information related to a response to the semantic information based on the received input from the user and the converted semantic information, and transmits the generated information related to the response to another device.

(5) Moreover, the other aspect of this invention makes a computer accept the input from a user, acquires audio | voice information, converts the acquired audio | voice information into semantic information, and converts into the converted said semantic information. Based on the received input from the user and the converted semantic information, information related to the response to the semantic information is generated, and the information related to the generated response is transmitted to another device. It is a program that sends.

  DESCRIPTION OF SYMBOLS 10 ... Communication apparatus 101 ... Control part 102 ... Transmission / reception part 103 ... Audio | voice input part 104 ... Audio | voice output part 105 ... Display part 106 ... Reception part 111 ... Acquisition part 112 ... Generating part 113 ... Voice meaning conversion part 114 ... Meaning voice conversion part

Claims (5)

A reception unit that receives input from the user;
An acquisition unit for acquiring audio information;
A conversion unit that converts the audio information acquired by the acquisition unit into semantic information;
A display unit for displaying information based on the semantic information converted by the conversion unit;
Extracting a question sentence from the semantic information converted by the conversion unit, generating an option to be selected by the user based on the extracted question sentence, displaying the option on the display unit, and selecting the user from the displayed option A generation unit that generates information related to a response to the semantic information based on the option received by the reception unit;
A transmission unit that transmits information related to the response generated by the generation unit to another device;
A communication device comprising: The communication device according to claim 1,
The generation unit extracts a question sentence from the semantic information, generates an input field for the user to input semantic information based on the extracted question sentence, and an input for the user to input the semantic information Generating information related to the response to the semantic information based on the input received by the reception unit, which is input from the user with respect to a column;
Communication device. The communication device according to claim 1 or 2,
An output unit that converts the semantic information into audio information and performs audio output;
Communication device. Accepts user input,
Get audio information,
Converting the acquired voice information into semantic information;
Displaying information based on the converted semantic information;
Based on the received input from the user and the converted semantic information, information on a response to the semantic information is generated,
Sending information about the generated response to another device;
Communication method. On the computer,
Accept input from the user,
Get audio information,
Converting the acquired voice information into semantic information;
Displaying information based on the converted semantic information;
Based on the received input from the user and the converted semantic information, information on a response to the semantic information is generated,
Sending information about the generated response to another device;
program.

Images