JP2020030324A - Combination program, combination device, and combination method - Google Patents

Abstract

To improve usability relating to a voice input.SOLUTION: A combining program causes a computer to execute: a display procedure for displaying a plurality of pieces of information input via a voice (e.g., a character string resulting from voice recognition of a speech input of a user); and a combination procedure for combining selected information with other information and displaying combined information when a prescribed operation (e.g., a tap operation or slide operation) that involves selection of any information out of the plurality of pieces of information displayed by the display procedure is accepted.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a combining program, a combining device, and a combining method.

  2. Description of the Related Art Conventionally, a technique for editing input audio information is known. For example, a technology is disclosed in which, when a voice input is received, the voice is converted into voice data, and conversion candidate kanji of the converted voice data is displayed.

JP 2001-285413 A

  However, in the above-mentioned conventional technology, usability may not be good.

  The present application has been made in view of the above, and has as its object to improve usability related to voice input.

  The combination program according to the present application receives a display procedure for displaying a plurality of pieces of information input via voice, and a predetermined operation involving selection of any one of a plurality of pieces of information displayed by the display procedure. And combining the selected information with other information and displaying the selected information.

  According to an aspect of the embodiment, there is an effect that usability related to voice input can be improved.

FIG. 1 is a diagram illustrating an example of a terminal device according to the embodiment. FIG. 2 is a diagram illustrating a configuration example of the terminal device according to the embodiment. FIG. 3 is a diagram illustrating a configuration example of the voice input application according to the embodiment. FIG. 4 is a flowchart illustrating an example of a display processing procedure according to the embodiment. FIG. 5 is a hardware configuration diagram illustrating an example of a computer that realizes the functions of the terminal device.

  Hereinafter, a mode (hereinafter, referred to as an “embodiment”) for implementing a combination program, a combination device, and a combination method according to the present application will be described in detail with reference to the drawings. Note that the combination program, the combination device, and the combination method according to the present application are not limited by this embodiment. In the following embodiments, the same portions are denoted by the same reference numerals, and redundant description will be omitted.

[1. Join processing)
With reference to FIG. 1, a combining process realized by the combining program and the like of the present embodiment will be described. FIG. 1 is a diagram illustrating an example of a terminal device according to the embodiment. In FIG. 1, a character string that is a result of voice recognition in response to a user's voice input is displayed, and the displayed character string is output to another application (hereinafter, may be referred to as a “voice input application”). ) Shows an example in which the terminal device 100 is mounted on the terminal device 100. In FIG. 1, it is assumed that the combining process according to the embodiment is realized by the function of the voice input application.

  A terminal device 100 shown in FIG. 1 is a smart device such as a smartphone or a tablet used by a user, and communicates with an arbitrary server device via a wireless communication network such as 3G (Generation) or LTE (Long Term Evolution). It is a portable terminal device that can be used. The example illustrated in FIG. 1 illustrates a case where the terminal device 100 is a smartphone used by the user U1. The terminal device 100 has an output unit such as a liquid crystal display (hereinafter, sometimes referred to as a screen) and employs a touch panel. That is, the user U1 performs various operations of the terminal device 100 by touching the display surface of the output unit with a finger or a dedicated pen. In the example of FIG. 1, it is assumed that the terminal device 100 includes a sound collecting device (a microphone or the like) that collects sound. Note that the terminal device 100 is not limited to a smart device, and may be an information processing device such as a notebook PC (Personal Computer), a desktop PC, a mobile phone, or a PDA (Personal Digital Assistant).

  Hereinafter, the display processing of the voice input application performed by the terminal device 100 will be described with reference to FIG. In the following description, when the state transition of the screen of the terminal device 100 is illustrated, the state transition is represented as a first state, a second state,..., An N-th state (N is an arbitrary number).

  The first state in FIG. 1 shows a state of a screen displaying a character string as a result of voice recognition for a voice input by the user U1. Here, in the first state of FIG. 1, it is assumed that the terminal device 100 displays the character string “Kenji” in the area AR1 as a result of voice recognition in response to the voice input of the user U1.

  Subsequently, the second state of FIG. 1 shows a state of the screen on which the conversion candidate of the reading kana “Kenji” of the character string “Kenji” displayed in the area AR1 in the first state to another kanji is displayed. In the second state of FIG. 1, the terminal device 100 receives a predetermined touch operation (for example, a tap on the area AR1) or the like by the user U1 and performs another conversion corresponding to the reading pseudonym “Kenji” of the character string “Kenji”. The candidate kanji is displayed in the area AR2.

  Here, in the second state of FIG. 1, the conversion candidate desired by the user U1 is not displayed in the area AR2, and the user U1 converts only “Ken” displayed in the area AR1 into another conversion candidate kanji. You wish to do so. In this case, the terminal device 100 displays the screen in the third state of FIG. 1 in which the character string “Kenji” is divided and displayed at the edit point (between “Ken” and “Ken”) selected by the finger F10 of the user U1. Is changed. For example, in the example of FIG. 1, the terminal device 100 receives the touch operation (for example, a tap) of the finger F10 of the user U1 on the area between “smart” and “healing” displayed in the area AR1. The screen is transited to the third state in FIG.

  Note that the terminal device 100 may receive a selection operation of various editing points from the user U1. For example, the terminal device 100 may display the character string by dividing the character string at a predetermined editing point by receiving a touch operation by the finger F10 of the user U1 on the character string as a result of the voice recognition. Specifically, the terminal device 100 may set the editing point between each kanji included in the character string or between kanji and hiragana. In the example of FIG. 1, the terminal device 100 displays a screen by accepting an operation of tracing between “smart” and “healing” from a state in which the finger F10 of the user U1 touches the character string “Kenji”. A transition to the first third state may be made.

  Subsequently, a third state in FIG. 1 shows a state of a screen in which the character string “Kenji” is divided and displayed at the editing point selected in the second state. In the third state of FIG. 1, the terminal device 100 displays “smart” in the area AR3 and displays “healing” in the area AR4. Here, the terminal device 100 accepts a touch operation (for example, a double tap) on the area AR3 by the finger F10 of the user U1 to change “Ken” displayed in the area AR3 into a reading pseudonym “Ken” of “Ken”. The screen is transited to the fourth state in FIG.

  Subsequently, a fourth state in FIG. 1 illustrates a state of a screen in which the reading kana of “Ken” in the area AR3 where the touch operation of the user U1 has been performed in the third state is divided and displayed. In the fourth state of FIG. 1, the terminal device 100 displays “ke” in the area AR5 and displays “n” in the area AR6. Here, in the fourth state of FIG. 1, it is assumed that the user U1 desires to combine “ke” displayed in the area AR5 and “n” displayed in the area AR6. In this case, the terminal device 100 transitions the screen to the fifth state in FIG. 1 in which “ke” and “n” are combined and displayed by receiving a touch operation with the finger F10 of the user U1. For example, in the example of FIG. 1, the terminal device 100 receives a touch operation of sliding the area AR5 in the direction of the area AR6 from a state in which the finger F10 of the user U1 touches the area AR5, thereby changing the screen of the terminal U in FIG. Transition to state 5.

  Note that the terminal device 100 may change the screen from the second state in FIG. 1 to the fourth state without going through the third state. In other words, the terminal device 100 divides a character string into a plurality of pieces of information using a position selected by the finger F10 of the user U1 as an editing point. Then, the terminal device 100 converts the kanji which is the kanji included in each piece of information and whose reading kana is composed of a plurality of hiragana, into hiragana, divides each hiragana and displays the hiragana on the screen.

  Subsequently, the fifth state in FIG. 1 illustrates a state of the screen in which the areas AR5 and AR6 in which the touch operation regarding the connection by the user U1 has been received in the fourth state are combined. In the fifth state of FIG. 1, the terminal device 100 displays “KEN” in the area AR7. Further, the terminal device 100 accepts a predetermined touch operation (for example, a tap on the area AR7) by the user U1, and displays a conversion candidate kanji corresponding to “ken” in the area AR2.

  Here, it is assumed that the voice input performed by the user U1 has an erroneous input, and the user wants to re-input the information displayed in the area AR7 in the fifth state of FIG. In this case, the terminal device 100 changes the screen to the sixth state in FIG. 1 by receiving a predetermined touch operation (for example, a long tap on the area AR7) on the area AR7 by the finger F10 of the user U1. .

  Subsequently, the sixth state in FIG. 1 shows a state of a screen for receiving a voice input by the user U1 again. In the sixth state in FIG. 1, the terminal device 100 displays an icon image indicating that voice input can be accepted in the area AR7 instead of “ken”. The terminal device 100 receives a voice input from the user U1, and transitions the screen to the seventh state in FIG.

  Subsequently, a seventh state in FIG. 1 shows a state of a screen displaying a character string as a result of voice recognition for a voice input by the user U1. In the seventh state of FIG. 1, the terminal device 100 displays “KO”, which is the result of voice recognition, in the area AR7 instead of the icon image. That is, the terminal device 100 updates the character string input by voice by receiving the voice input again. Further, the terminal device 100 receives a predetermined touch operation (for example, a tap on the area AR7) or the like by the user U1, and displays a conversion candidate kanji corresponding to “ko” in the area AR2. Here, the terminal device 100 accepts a kanji as a conversion destination of the character string displayed in the area AR7 by a touch operation on the area AR2 by the finger F10 of the user U1, and shifts the screen to the eighth state in FIG.

  Subsequently, an eighth state in FIG. 1 shows a state of the screen in which the character string displayed in the area AR7 is converted into the conversion destination kanji received from the user U1. In the eighth state of FIG. 1, the terminal device 100 displays “Ko” in which “Ko” has been converted into Chinese characters in the area AR7 instead of “Ko”. Here, the terminal device 100 receives the information displayed in the area AR4 and the area AR7 by receiving a touch operation of sliding the area AR7 in the direction of the area AR4 from a state in which the finger F10 of the user U1 touches the area AR7. Display the combined character string "Koji" on the screen. Then, the terminal device 100 receives the character string output operation (for example, a touch operation on a screen area other than the area where the character string is displayed) from the user U1, and thereby converts the character string “Koji” to a character other than the voice input application. Output to other applications.

  In the above-described embodiment, an example has been described in which the terminal device 100 performs the screen display processing when the user U1 such as a tap, a double tap, a slide, and a long tap is received as a touch operation. However, the terminal device 100 may also perform screen display processing when an operation other than the above-described touch operation is performed. For example, the terminal device 100 may receive a touch operation such as a pinch out, a pinch in, a flick, a swipe, a drag, a touch and hold, a shake, a peak (Peek), and a pop (Pop), and perform the above-described screen display processing. Good.

  As described above, the terminal device 100 according to the embodiment performs a process of dividing and combining information and a process of updating information with respect to a character string input by voice in response to a user's touch operation on a screen. . Thus, the user can perform voice input by a user's intuitive operation without performing an operation using a keyboard or the like. That is, the terminal device 100 according to the embodiment can improve usability in voice input.

[2. Configuration of terminal device)
Next, a terminal device 100 for implementing the above-described display processing will be described with reference to FIG. FIG. 2 is a diagram illustrating a configuration example of the terminal device according to the embodiment. As shown in FIG. 2, the terminal device 100 includes a communication unit 110, an input unit 120, an output unit 130, a physical sensor 140, and a control unit 150.

(About the communication unit 110)
The communication unit 110 is realized by, for example, a NIC (Network Interface Card) or the like. The communication unit 110 is connected to the network N by wire or wirelessly, and transmits and receives information to and from an application distribution server that distributes a voice input application and other applications.

(About the input unit 120 and the output unit 130)
The input unit 120 is an input device that receives various operations from a user. For example, the input unit 120 is realized by a keyboard, a mouse, operation keys, and the like. The output unit 130 is a display device for displaying various information. That is, the output unit 130 corresponds to a display screen of the terminal device 100. For example, the output unit 130 is realized by a liquid crystal display or the like. When a touch panel is used for the terminal device 100, the input unit 120 and the output unit 130 are integrated.

(About the physical sensor 140)
The physical sensor 140 is a sensor that detects the physical state of the terminal device 100. For example, the physical sensor 140 is a pressure sensor that detects a pressure amount in a user's touch operation or the like on the screen of the terminal device 100.

(About the control unit 150)
The control unit 150 is a controller, and various programs stored in a storage device inside the terminal device 100 use a RAM as a work area by, for example, a CPU (Central Processing Unit) or an MPU (Micro Processing Unit). It is realized by being executed. When the control information or the application for realizing the above-described display processing is distributed from an application distribution server or the like, the control unit 150 operates according to the control information or the application to realize the display processing.

  The control unit 150 is a controller, and is realized by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array). As illustrated in FIG. 2, the control unit 150 according to the embodiment includes a voice input application 151, a first application 152, a second application 153, and the like. The voice input application 151 realizes or executes the functions and actions of the above-described display processing, and transmits information that is a result of voice recognition for a user's voice input to another application such as the first application 152 or the second application 153. Output. Hereinafter, an example of a functional configuration of the voice input application 151 will be described with reference to FIG. FIG. 3 is a diagram illustrating a configuration example of the voice input application according to the embodiment. As illustrated in FIG. 3, the voice input application 151 according to the embodiment includes a recognition unit 1511, a display unit 1512, a reception unit 1513, a division unit 1514, an update unit 1515, a combination unit 1516, and an output unit 1517. And implements or executes the functions and operations of the information processing described below.

(About the recognition unit 1511)
The recognizing unit 1511 performs a voice recognition process on a user's voice input received by a receiving unit 1513 described below. For example, in the example of FIG. 1, the recognition unit 1511 executes a voice recognition process for a voice input by the user U1, and converts the voice into a character string.

(About the display unit 1512)
Display unit 1512 displays information input via voice. For example, the display unit 1512 displays information that is a result of a voice recognition process performed by the recognition unit 1511 in response to a voice input by a user.

  The display unit 1512 may display a character string that is a result of the voice recognition. For example, the display unit 1512 displays a character string that is input via voice and includes a Chinese character. For example, in the example of FIG. 1, the display unit 1512 displays a character string “Kenji” as a result of voice recognition in response to a voice input of the user U1.

  Further, the display unit 1512 may display a plurality of pieces of information input via voice. For example, the display unit 1512 displays a plurality of pieces of information obtained by dividing information that is a result of speech recognition. Specifically, the display unit 1512 displays a plurality of pieces of information obtained by dividing a character string as a result of the voice recognition processing performed by the recognition unit 1511. Here, display unit 1512 may divide and display each Chinese character included in the character string. The display unit 1512 may display the Chinese characters included in the character string and the hiragana separately.

(About the reception unit 1513)
The receiving unit 1513 receives input of new information via voice when receiving an operation of selecting any one of the plurality of pieces of information displayed by the display unit 1512. In addition, after a dividing unit 1514, which will be described later, divides information into a plurality of pieces of information and displays the information, the receiving unit 1513 receives new information via voice when receiving an operation of selecting one of the plurality of pieces of information. Accept input. For example, in the example of FIG. 1, when a predetermined touch operation is performed on the area AR7 by the finger F10 of the user U1, the reception unit 1513 displays an icon image indicating that voice input can be received in the area AR7. , Accepts input of new information from the user U1 via voice.

  In addition, the receiving unit 1513 may receive an operation of combining a plurality of pieces of information. For example, in the example of FIG. 1, the reception unit 1513 performs a touch operation of sliding the area AR7 toward the area AR4 from a state in which the finger F10 of the user U1 touches the area AR7, and an operation of combining the area AR4 and the area AR7. Accept as.

(About the dividing unit 1514)
The division unit 1514 divides information at the editing point and displays the information when an operation of selecting a position to be an editing point is received from the information displayed on the display unit 1512. For example, in the example of FIG. 1, the dividing unit 1514 accepts a touch operation with the finger F10 of the user U1 as an edit point in an area between “Ken” and “Haru” displayed in the area AR1, and thereby performs the area AR1 Are divided and displayed in an area AR3 displaying “smart” and an area AR4 displaying “health”.

  The division unit 1514 may divide and display information when a predetermined operation involving selection of information displayed on the display unit 1512 is received. For example, in the example of FIG. 1, the dividing unit 1514 divides and displays a character string displayed in the area AR1 by receiving a predetermined touch operation from a state in which the finger F10 of the user U1 touches the area AR1. .

  In addition, the dividing unit 1514 divides and displays the information displayed on the display unit 1512 at a position where a predetermined operation is performed as an editing point. For example, in the example of FIG. 1, the division unit 1514 accepts an operation of tracing between “smart” and “healing” from a state in which the finger F10 of the user U1 touches the area AR1, thereby “smart” and “healing”. The character string displayed in the area AR1 is divided and displayed using the area between "" as an edit point.

  The dividing unit 1514 divides the information into a plurality of pieces of information displayed on the display unit 1512 by using the position where the first operation involving the selection of the information is performed as the first editing point. After that, when a second operation involving selection of any of a plurality of pieces of information is received, the position where the second operation was performed is displayed as a second editing point by dividing the selected information. Good. For example, in the example of FIG. 1, the dividing unit 1514 sets the area between “Ken” and “Haru” displayed in the area AR1 as the first editing point, and sets the character string displayed in the area AR1 to the area AR3 and the area AR3. The display is divided into areas AR4. Then, the division unit 1514 receives the second operation (touch operation) of the user U1 on the area AR3, and thereby, the area between the reading pseudonyms “Ken” and “N” of “Ken” displayed in the area AR3. As a second editing point, the information displayed in the area AR3 is divided into an area AR5 and an area AR6 and displayed.

  In addition, the dividing unit 1514 may convert Chinese characters included in the character string displayed by the display unit 1512 into a plurality of hiragana, and may display each hiragana in a divided manner. For example, in the example of FIG. 1, the dividing unit 1514 receives a touch operation in which the finger F10 of the user U1 has an edit point in an area between “Ken” and “Haru” displayed in the area AR1 and displays the touch operation in the area AR1. The divided character string is divided into an area AR3 and an area AR4, and the information displayed in the area AR3 is divided and displayed in an area AR5 and an area AR6.

(About the update unit 1515)
The updating unit 1515 updates the selected information with new information received by the receiving unit 1513. For example, in the example of FIG. 1, the updating unit 1515 displays, in the area AR7, a character string that is a result of voice recognition for the voice input by the user U1 received by the receiving unit 1513 in the sixth state, and is displayed in the area AR7. Is updated from "ken" to "ko".

(About the joint 1516)
The combining unit 1516 combines and displays a plurality of pieces of information when the receiving unit 1513 receives an operation of combining a plurality of pieces of information. For example, the combining unit 1516 combines the selected information with other information when a predetermined operation involving selection of any of the information displayed on the display unit 1512 is received, and displays the combined information. . Specifically, when the display unit 1512 divides and displays a character string input through a user's voice into a plurality of pieces of information (for example, when each kanji included in the character string is divided and displayed, In the case where a kanji and a hiragana included in a character string are divided and displayed, for example), the combining unit 1516 accepts a predetermined operation involving selection of information from the user and combines the selected information with other information and displays the combined information. .

  Further, the combining unit 1516 may combine and display a plurality of pieces of information when the dividing unit 1514 divides the information into a plurality of pieces of information and displays the information. For example, in the example of FIG. 1, the combining unit 1516 displays a character string “Koji” on the screen in which information displayed in the area AR4 and the information displayed in the area AR7 are combined by accepting a touch operation on the area AR7 by the user U1. I do.

  Also, the combining unit 1516 combines the selected information with the other information and displays the selected information when the operation of moving the selected information among the plurality of information in the direction in which the other information is displayed is received. Good. For example, the combining unit 1516 combines and displays the information on which the slide operation has been performed in accordance with the selection with the information on the sliding direction among the plurality of pieces of information. For example, in the example of FIG. 1, the coupling unit 1516 accepts a touch operation of sliding the area AR7 in the direction of the area AR4 from a state in which the finger F10 of the user U1 touches the area AR7, thereby changing the area AR4 and the area AR7. Display the combined character string "Koji" on the screen.

  In addition, the combining unit 1516 may combine the information on which the tap operation has been performed in accordance with the selection with the other information and display the information. For example, the combining unit 1516 combines and displays the information on which the tap operation has been performed with the information displayed in a predetermined direction of the information.

  Also, the combining unit 1516 may combine and display a plurality of pieces of information without changing them. For example, in the example of FIG. 1, the combining unit 1516 combines and displays the information displayed in the area AR7 with the information displayed in the area AR4 while being converted into a kanji.

(About the output unit 1517)
The output unit 1517 outputs information that is a result of voice recognition for a user's voice input to another application such as the first application 152 and the second application 153. For example, in the example of FIG. 1, the output unit 1517 outputs the character string “Koji” displayed on the screen to another application when the character string output operation is received from the user U1.

[3. Display processing flow)
Here, the procedure of the display process of the voice input application according to the embodiment will be described with reference to FIG. FIG. 4 is a flowchart illustrating an example of a display processing procedure according to the embodiment.

  As shown in FIG. 4, the terminal device 100 receives a voice input from a user (Step S101). Subsequently, the terminal device 100 displays a result of the voice recognition for the voice input (Step S102). In the present description, the terminal device 100 displays a character string as a result of voice recognition. Subsequently, when a division operation on the character string recognized by the user is received (step S103; Yes), the terminal device 100 divides and displays the character string recognized by speech (step S104). For example, the terminal device 100 divides and displays a character string at an editing point selected by a user's touch operation or the like. On the other hand, when the division operation on the character string is not received (Step S103; No), the terminal device 100 displays the character string as it is.

  Subsequently, when any information constituting the character string displayed on the screen is selected (Step S105; Yes), the terminal device 100 receives a new voice input from the user and updates the selected information ( Step S106). For example, in the example of FIG. 1, the terminal device 100 determines that the area AR7 is selected and the voice input can be accepted in a state where the character string displayed in the area AR1 is divided into the area AR4 and the area AR7 and displayed. The icon image to be displayed is displayed in the area AR7. Then, the terminal device 100 newly displays a character string as a result of the voice recognition for the voice input in the area AR7. On the other hand, when the information forming the character string is not selected (Step S105; No), the terminal device 100 displays the character string as it is.

  Subsequently, when a combining operation of the information constituting the character string displayed on the screen is performed (Step S107; Yes), the terminal device 100 combines and displays the information to be operated (Step S108). For example, in the example of FIG. 1, the terminal device 100 accepts a touch operation of sliding the area AR7 in the direction of the area AR4 from a state in which the finger F10 of the user U1 touches the area AR7, and A character string combining the displayed information is displayed on the screen. On the other hand, when the operation of combining the information constituting the character string is not performed (Step S107; No), the terminal device 100 displays the character string as it is.

  Subsequently, when the output operation of the character string displayed on the screen is performed (Step S109; Yes), the terminal device 100 outputs the character string to another application (Step S110), and ends the processing. On the other hand, when the output operation of the character string has not been performed (step S109; No), the terminal device 100 displays the character string as it is, and accepts the division operation, the information update, and the combining operation again.

[4. Modification)
The above-described embodiment is an example, and various modifications and applications are possible.

[4-1. About character strings)
In the above-described embodiment, the example in which the terminal device 100 performs the display processing of the character string configured in Japanese has been described. However, the terminal device 100 is not limited to the above-described example, and is configured by characters used in various natural languages. A character string display process may be performed. For example, when accepting an English voice input, the terminal device 100 is a character string composed of alphabets such as “right” and “light” in response to a voice input of “light” by the user, and has a similar pronunciation. Is displayed as a result of voice recognition or a conversion candidate. When displaying the English as a result of the speech recognition, the terminal device 100 may display the inflection of each part of speech in English as a conversion candidate. For example, when displaying “go” as a result of speech recognition, the terminal device 100 may display “goes”, “went”, “gone”, and the like as conversion candidates. Further, when displaying a compound word as a result of voice recognition of English, the terminal device 100 may be capable of dividing the compound word into parts of speech of constituent elements. For example, when the terminal device 100 displays “policeman” as a result of the voice recognition, the terminal device 100 accepts a user's division operation and displays the division as “police” and “man” (for example, “police” and “man” are displayed). (A blank area may be provided between them for display).

  In addition, the terminal device 100 can display a character string including a symbol (for example, a circle, a cross, a triangle, a square, an arrow, a hyphen, a pictogram, etc.) or a number, if it can be input into an input field such as a text in various applications. Display processing may be performed.

[4-2. Insertion of new information)
The receiving unit 1513 may receive new information to be inserted into information input via voice. For example, in the fifth state of FIG. 1, the receiving unit 1513 receives a touch operation that causes a predetermined range of a blank area on the left side of the area AR7, between the area AR4 and the area AR7, and on the right side of the area AR4. In this case, an icon image indicating that voice input can be received is displayed in the blank area, and voice input of information to be inserted into the blank area is received from the user U1. Then, the updating unit 1515 inserts the new information received by the receiving unit 1513 into the blank area, and updates the character string displayed on the terminal device 100.

[5. effect〕
As described above, the voice input application 151 of the terminal device 100 according to the embodiment includes the recognition unit 1511, the display unit 1512, the reception unit 1513, the division unit 1514, the update unit 1515, and the connection unit 1516. , And an output unit 1517. The recognizing unit 1511 performs a voice recognition process on a user's voice input. Display unit 1512 displays information input via voice. The accepting unit 1513 accepts an input of new information via voice when the dividing unit 1514 divides the information into a plurality of pieces of information and displays the information, and then receives an operation of selecting one of the plurality of pieces of information. The division unit 1514 divides information at the editing point and displays the information when an operation of selecting a position to be an editing point is received from the information displayed on the display unit 1512. The updating unit 1515 updates the selected information with new information received by the receiving unit 1513. The combining unit 1516 combines and displays a plurality of pieces of information when the dividing unit 1514 divides the information into a plurality of pieces of information and displays the information. The output unit 1517 outputs information that is a result of voice recognition for a user's voice input to another application.

  Accordingly, the terminal device 100 according to the embodiment can perform voice input by a user's intuitive operation, and thus can improve usability related to voice input.

  In the terminal device 100 according to the embodiment, for example, the display unit 1512 displays a plurality of pieces of information input via voice. Then, the receiving unit 1513 receives an input of new information via voice when receiving an operation of selecting any one of the plurality of pieces of information displayed by the display unit 1512. Then, the updating unit 1515 updates the selected information with new information received by the receiving unit 1513.

  As a result, the terminal device 100 according to the embodiment can select any one of the displayed information and update the information with new information, and thus can improve the usability related to the voice input of the character string.

  In the terminal device 100 according to the embodiment, for example, the display unit 1512 displays a plurality of pieces of information input via voice. Then, the combining unit 1516 combines the selected information with other information when a predetermined operation involving selection of any of the plurality of information displayed by the display unit 1512 is received, and displays the combined information. .

  Accordingly, the terminal device 100 according to the embodiment can select information to be combined from the plurality of pieces of displayed information, and thus can improve the usability regarding voice input of a character string.

  Further, in the terminal device 100 according to the embodiment, for example, the display unit 1512 displays a plurality of pieces of information obtained by dividing information that is a result of voice recognition.

  Thereby, the terminal device 100 according to the embodiment can perform an operation on the divided information, so that usability for voice input of a character string can be improved.

  In the terminal device 100 according to the embodiment, for example, the display unit 1512 displays a character string as a result of voice recognition.

  Thus, the terminal device 100 according to the embodiment can display the character string as a result of the voice recognition, thereby improving the usability related to the voice input of the character string.

  In the terminal device 100 according to the embodiment, for example, the division unit 1514 divides and displays information when a predetermined operation involving selection of information displayed on the display unit 1512 is received.

  Accordingly, the terminal device 100 according to the embodiment can divide the displayed information according to the operation of the user, and thus can improve the usability related to the voice input.

  Further, in the terminal device 100 according to the embodiment, for example, the dividing unit 1514 divides the information displayed on the display unit 1512 at a position where a predetermined operation is performed as an editing point and displays the information.

  Thereby, the terminal device 100 according to the embodiment can divide the position where the user's operation is performed as an edit point and perform various operations on the divided information, thereby improving usability related to voice input. it can.

  Further, in the terminal device 100 according to the embodiment, for example, the division unit 1514 sets, as the first edit point, the position where the first operation involving the selection of the information is performed among the information displayed by the display unit 1512. After the information is divided into a plurality of pieces of information and displayed, and a second operation involving selection of one of the plurality of pieces of information is received, a position where the second operation is performed is set as a second editing point. Divide and display the selected information.

  Accordingly, the terminal device 100 according to the embodiment can further divide the divided information and perform various operations, so that usability for voice input can be improved.

  Further, in the terminal device 100 according to the embodiment, for example, the display unit 1512 displays a character string that is input via voice and includes a kanji. Then, the dividing unit 1514 converts the kanji included in the character string displayed by the display unit 1512 into a plurality of hiragana, and divides and displays each hiragana.

  Accordingly, the terminal device 100 according to the embodiment can easily perform the conversion operation on the character string including the kanji, and thus can improve the usability regarding the voice input.

  In the terminal device 100 according to the embodiment, for example, the receiving unit 1513 receives an operation of combining a plurality of pieces of information. Then, the combining unit 1516 combines and displays a plurality of pieces of information when the receiving unit 1513 receives an operation of combining a plurality of pieces of information.

  Accordingly, the terminal device 100 according to the embodiment can combine the displayed information, and thus can improve the usability related to the voice input.

  Further, in the terminal device 100 according to the embodiment, for example, the combining unit 1516 converts the information selected when the operation of moving the selected information among the plurality of information in the direction in which the other information is displayed is received. Display in combination with other information.

  Accordingly, the terminal device 100 according to the embodiment can combine the information displayed by the intuitive operation, so that usability related to voice input can be improved.

  In addition, in the terminal device 100 according to the embodiment, for example, the combining unit 1516 combines and displays information on which a tap operation has been performed along with selection with other information.

  Accordingly, the terminal device 100 according to the embodiment can combine the information displayed by the intuitive operation, so that usability related to voice input can be improved.

  In addition, in the terminal device 100 according to the embodiment, for example, the combining unit 1516 combines and displays the information on which the slide operation has been performed in accordance with the selection from the plurality of pieces of information with the information on the slide direction.

  Accordingly, the terminal device 100 according to the embodiment can combine the information displayed by the intuitive operation, so that usability related to voice input can be improved.

  In the terminal device 100 according to the embodiment, for example, the combining unit 1516 combines and displays a plurality of pieces of information without changing.

  Accordingly, the terminal device 100 according to the embodiment can combine information operated by the user without changing the information, thereby improving usability related to voice input.

[6. Hardware configuration)
Further, the terminal device according to each embodiment described above is realized by, for example, a computer 1000 having a configuration as shown in FIG. Hereinafter, the terminal device 100 will be described as an example. FIG. 5 is a hardware configuration diagram illustrating an example of a computer that realizes the functions of the terminal device. The computer 1000 has a CPU 1100, a RAM 1200, a ROM 1300, an HDD 1400, a communication interface (I / F) 1500, an input / output interface (I / F) 1600, and a media interface (I / F) 1700.

  The CPU 1100 operates based on a program stored in the ROM 1300 or the HDD 1400, and controls each unit. The ROM 1300 stores a boot program executed by the CPU 1100 when the computer 1000 starts up, a program that depends on hardware of the computer 1000, and the like.

  The HDD 1400 stores a program executed by the CPU 1100, data used by the program, and the like. The communication interface 1500 receives data from another device via the communication network 500 (corresponding to the network N of the embodiment) and sends the data to the CPU 1100, and also transmits data generated by the CPU 1100 via the communication network 500 to another device. Send to device.

  The CPU 1100 controls output devices such as a display and a printer and input devices such as a keyboard and a mouse via the input / output interface 1600. The CPU 1100 obtains data from an input device via the input / output interface 1600. Further, CPU 1100 outputs data generated through input / output interface 1600 to an output device.

  The media interface 1700 reads a program or data stored in the recording medium 1800 and provides the program or data to the CPU 1100 via the RAM 1200. The CPU 1100 loads the program from the recording medium 1800 onto the RAM 1200 via the media interface 1700, and executes the loaded program. The recording medium 1800 is, for example, an optical recording medium such as a DVD (Digital Versatile Disc), a PD (Phase Change Rewritable Disk), a magneto-optical recording medium such as an MO (Magneto-Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory. And so on.

  For example, when the computer 1000 functions as the terminal device 100, the CPU 1100 of the computer 1000 realizes the function of the control unit 150 by executing a program loaded on the RAM 1200. Further, the HDD 1400 stores various data in the storage device of the terminal device 100. Although the CPU 1100 of the computer 1000 reads these programs from the recording medium 1800 and executes them, as another example, these programs may be obtained from another device via a predetermined communication network.

[7. Others)
As described above, some of the embodiments of the present application have been described in detail with reference to the drawings. However, these are exemplifications, and various modifications based on the knowledge of those skilled in the art, including the aspects described in the section of the disclosure of the invention, The invention can be implemented in other modified forms.

  The configuration of the terminal device 100 described above can be flexibly changed, for example, by realizing an external platform or the like by calling it with an API (Application Programming Interface) or network computing depending on the function.

  Further, “parts” described in the claims can be read as “means” or “circuit”. For example, the recognition unit can be replaced with a recognition unit or a recognition circuit.

Reference Signs List 100 terminal device 110 communication unit 120 input unit 130 output unit 140 physical sensor 150 control unit 1511 recognition unit 1512 display unit 1513 reception unit 1514 division unit 1515 update unit 1516 coupling unit 1517 output unit 152 first application 153 second application

Claims (9)

A display procedure for displaying a plurality of pieces of information input via voice,
And performing a combining step of combining the selected information with other information and displaying the selected information when a predetermined operation involving selection of any of the plurality of pieces of information displayed by the display procedure is received. A combining program characterized by causing The combining procedure comprises:
The information selected from the plurality of pieces of information is displayed in combination with another piece of information when an operation of moving the selected piece of information in the direction in which the other piece of information is displayed is received. The binding program according to 1. The combining procedure comprises:
The combination program according to claim 1, wherein, among the plurality of pieces of information, information on which a tap operation has been performed according to selection is combined with other information and displayed. The combining procedure comprises:
The combination program according to claim 1, wherein, among the plurality of pieces of information, information on which a slide operation is performed according to selection is combined with information on a slide direction and displayed. The display procedure includes:
The combination program according to any one of claims 1 to 4, wherein a character string resulting from speech recognition is displayed. The display procedure includes:
The combined program according to any one of claims 1 to 5, wherein a plurality of pieces of information obtained by dividing information resulting from speech recognition are displayed. The combining procedure comprises:
The combination program according to any one of claims 1 to 6, wherein the plurality of pieces of information are combined and displayed without being changed. A display unit for displaying a plurality of pieces of information input via voice;
A coupling unit that, when a predetermined operation involving selection of any one of the plurality of information displayed by the display unit is received, combines the selected information with other information and displays the combined information. Characteristic coupling device. A combining method performed by a computer,
A display step of displaying a plurality of pieces of information input via voice;
A combining step of combining the selected information with other information and displaying the selected information when a predetermined operation involving selection of any of the plurality of pieces of information displayed in the display step is received. Characteristic joining method.

Images