JP2022111842A - Update program, update device, and update method - Google Patents

Abstract

To propose appropriate search queries in a service used by users.SOLUTION: An update program pertaining to the present invention is characterized in causing a computer to execute: a display step for displaying character strings indicating candidates of a search query corresponding to a service used by a user, which are the candidates of the search query to be inputted through voice; a reception step for receiving an input of information through voice from the user; and an update step for updating and displaying a display mode of the character strings based on a voice recognition result of the information received by the reception step.SELECTED DRAWING: Figure 1

Description

The present invention relates to an update program, an update device, and an update method.

Conventionally, techniques related to voice input of search queries are known. For example, if the user does not know the words related to the search target, based on the speech recognition result of recognizing the voice input, the reference word extracted from the speech recognition result, and the user's instruction estimated from the user's behavior Techniques for creating user search instructions are disclosed.

JP 2015-153324 A

However, with the conventional technology described above, there are cases where it cannot be said that an appropriate search query can be proposed in the service used by the user.

For example, in the above conventional technology, only a search instruction is created when the user does not know the words related to the search target, and an appropriate search query is proposed in the service used by the user. It can not be said.

The present application has been made in view of the above, and aims to provide an update program, an update device, and an update method capable of proposing appropriate search queries in services used by users.

The update program according to the present application is a search query candidate input via voice, and includes a display procedure for displaying a character string indicating a search query candidate corresponding to a service used by a user; causing a computer to execute a receiving procedure for receiving input of information from the user and an updating procedure for updating and displaying the display mode of the character string based on the result of speech recognition of the information received by the receiving procedure; It is characterized by

According to one aspect of the embodiment, there is an effect that an appropriate search query can be proposed in the service used by the user.

FIG. 1 is a diagram illustrating an example of update processing according to the embodiment. FIG. 2 is a diagram illustrating an example of a screen of the terminal device according to the embodiment; FIG. 3 is a diagram illustrating a configuration example of a terminal device according to the embodiment; FIG. 4 is a diagram illustrating a configuration example of a voice input application according to the embodiment. FIG. 5 is a flowchart illustrating an example of an update processing procedure according to the embodiment; FIG. 6 is a hardware configuration diagram showing an example of a computer that implements the functions of the terminal device.

Embodiments for implementing the update program, update device, and update method according to the present application (hereinafter referred to as "embodiments") will be described in detail below with reference to the drawings. Note that the update program, update device, and update method according to the present application are not limited to this embodiment. Also, in each of the following embodiments, the same parts are denoted by the same reference numerals, and overlapping descriptions are omitted.

[1. embodiment]
The update processing realized by the update program etc. of this embodiment will be described with reference to FIG. FIG. 1 is a diagram illustrating an example of update processing according to the embodiment. In FIG. 1, a voice input application (hereinafter referred to as a "voice input application") displays a character string resulting from voice recognition in response to a user's voice input, and outputs the displayed character string to a predetermined external device or other application. ) is installed in the terminal device 100 used by the user. In FIG. 1, it is assumed that the updating process according to the embodiment is realized by the function of the voice input application.

As shown in FIG. 1 , an update system 1 according to the embodiment includes a service server 10 and terminal devices 100 . The service server 10 and the terminal device 100 are communicably connected to each other by wire or wirelessly via a network N (see FIG. 3, for example). The network N is, for example, a WAN (Wide Area Network) such as the Internet. Note that the update system 1 shown in FIG. 1 may include multiple service servers 10 and multiple terminal devices 100 .

A service server 10 shown in FIG. 1 is an information processing device that provides various services to users, and is realized by, for example, a server device, a cloud system, or the like. For example, the service server 10 is a portal site, a news site, an auction site, a weather forecast site, a shopping site, a finance (stock price) site, a route search site, a map providing site, a travel site, a restaurant introduction site, a web blog, and the like. and various applications installed on the terminal device 100 (for example, a portal application, a news application, an auction site, a weather forecast application, a shopping application, a finance (stock price) application, a route search application, a map providing application, a travel application, (e.g. restaurant introduction application, blog browsing application, etc.) is distributed to the terminal device 100 .

Note that the service server 10 may function as a distribution device that distributes control information to the terminal device 100 . Here, the control information is described in, for example, a script language such as JavaScript (registered trademark) or a style sheet language such as CSS (Cascading Style Sheets). Note that the application itself distributed from the service server 10 may be regarded as control information.

A terminal device 100 shown in FIG. 1 is an information processing device used by a user. The terminal device 100 is implemented by, for example, a smart phone, a tablet terminal, a notebook PC (Personal Computer), a desktop PC, a mobile phone, a PDA (Personal Digital Assistant), or the like. Note that in the example shown in FIG. 1, the terminal device 100 is a smart phone used by a user. Also, the terminal device 100 has an output unit such as a liquid crystal display (hereinafter sometimes referred to as a screen), and adopts a touch panel. That is, the user U1 performs various operations of the terminal device 100 by touching the display surface of the output unit with a finger or a dedicated pen. In the example of FIG. 1, the terminal device 100 includes a sound collecting device (such as a microphone) that collects sound.

Hereinafter, update processing of the voice input application performed by the terminal device 100 will be described with reference to FIG. Note that the following description shows an example in which the terminal device 100 is used by the user U1. Also, in the following description, an example in which the service server 10 provides a route search site or a route search application that provides a route search service will be shown.

First, the terminal device 100 displays a screen regarding a service corresponding to an operation by the user U1 (step S1). For example, in the example of FIG. 1, the terminal device 100 displays a screen C11 regarding the route search service provided by the service server 10 in response to an operation by the user U1. To give a specific example, the terminal device 100 includes an area AR11 for inputting search conditions (search queries) such as a departure station and an arrival station, and a button B11 for instructing voice input of the search query to the area AR11. and a screen C11 is displayed.

Here, in the example of FIG. 1, it is assumed that the user U1 presses the button B11. In this case, the terminal device 100 activates the voice input application, and changes the screen C11 to a screen displaying a character string indicating search query candidates that can be input in the route search service provided by the service server 10 (step S2). . For example, the terminal device 100 displays an area AR12 that displays the recognition result of the input voice, and a character string preset as a search query candidate for the route search service that can be input in the area AR11. and a button B12 for instructing the start of voice input.

As a specific example, the terminal device 100 includes a character string for designating a departure station, an arrival station, and the first train (first train) or the last train on the route from the departure station to the arrival station (hereinafter referred to as "character string #11”) is displayed in the area AR13. To give a more specific example, the terminal device 100 displays areas AR131 and AR132 that respectively display the departure station and arrival station input by voice from the user U1, and alternatively displays the "first train" and "last train". A character string #11 including the display area AR133 is displayed.

The terminal device 100 also includes a character string (hereinafter referred to as “character string #12”) for designating the departure station, the arrival station, the departure time from the departure station, or the arrival time at the arrival station. may be) is displayed in the area AR14. To give a more specific example, the terminal device 100 has areas AR141 and AR142 that respectively display the departure station and arrival station input by voice by the user U1, and an area AR143 that displays the time input by voice by the user U1. and AR144, and a character string #12 including an area AR145 that alternatively displays "departure" and "arrival".

Subsequently, the user U1 presses the button B12 and performs voice input (utterance) according to either of the character strings #11 and #12 displayed on the screen C12 (step S3). In the example of FIG. 1, the user U1 desires to search for the last train on a route with Tokyo Station as the departure station and Ikebukuro Station as the arrival station. ``The last train from Tokyo to Ikebukuro'' is uttered.

Subsequently, the terminal device 100 performs voice recognition of the utterance from the user U1, and updates the display mode of the character strings #11 and #12 based on the result of voice recognition (step S4). For example, the terminal device 100 displays the character string "last train from Tokyo to Ikebukuro", which is the result of speech recognition, in the area AR12. Then, among the character strings #11, the character strings that match the speech recognition results are highlighted by changing the character color, and the character strings indicated by the speech recognition results are displayed in the regions included in the character strings #11 and #12. display. To give a specific example, the terminal device 100 changes the character color of character strings corresponding to "from" and "to" uttered by the user U1 in the character string #11. In addition, the terminal device 100 displays "Tokyo" spoken by the user U1 before "kara" in the area AR131, and displays "Ikebukuro" spoken between "kara" and "made" in the area AR132. do. In addition, the terminal device 100 changes the character color of "last train" corresponding to user U1's utterance, out of "first train" and "last train" displayed in area AR133. Then, the terminal device 100 completes the updating process for the character string and each area included in the character string #11 (in other words, the information that is the essential item for performing the search process based on the character string #11 is normally voiced). input) is displayed in the area AR13.

Similarly, the terminal device 100 updates the display mode of the character string #12 based on the speech recognition result. To give a specific example, the terminal device 100 changes the character color of character strings corresponding to "from" and "to" uttered by the user U1 in the character string #12. In addition, the terminal device 100 displays "Tokyo" uttered by the user U1 before "kara" in the area AR141, and displays "Ikebukuro" uttered between "kara" and "made" in the area AR142. do. In the example of FIG. 1, the user U1 uttered "last train" after "until", and the utterance ended with "last train". 'last train' is displayed in the area AR143 located after 'until', and the process of updating the display mode of the subsequent character string is not performed. That is, the update processing for the character string and each area included in the character string #11 is incomplete.

Subsequently, the terminal device 100 transmits the updated character string as a search query to the service server 10 in response to an operation from the user U1 (step S5). For example, of the updated character strings #11 and #12, the terminal device 100 transmits the updated character string #11 selected by the user on the screen C13 as the search query.

Note that the terminal device 100 may select only the updated character string #11 from the updated character strings #11 and #12, and uses the updated character string #11 as a search query. It may be automatically sent to the service server 10 .

Subsequently, the terminal device 100 receives search results from the service server 10 (step S6). For example, the terminal device 100 receives transfer guidance for the last train on the route from "Tokyo Station" to "Ikebukuro Station" specified from "Tokyo" and "Ikebukuro" included in the updated character string #11.

Subsequently, the terminal device 100 displays the received search results (step S7). For example, a screen C14 containing information on transfer guidance from Tokyo Station to Ikebukuro Station (for example, the route to be used, stations to go through, fares, etc.) and a button B13 for instructing voice input of a new search query. indicate. As a specific example, the terminal device 100 displays a button B13 for instructing voice input of a search query for narrowing down information on transfer guidance.

Here, an example of a screen displayed by the terminal device 100 when the button B13 is pressed will be described with reference to FIG. FIG. 2 is a diagram illustrating an example of a screen of the terminal device according to the embodiment;

As shown in FIG. 2, the terminal device 100 activates the voice input application, and displays a character string indicating search query candidates corresponding to the narrowing function available in the route search service, as shown in a screen C21. For example, the terminal device 100 includes an area AR21 for displaying the search query sent to the route search service in step S5 of FIG. A screen C21 including areas AR23 to AR25 for displaying character strings preset as search query candidates and a button B21 for instructing the start of voice input is displayed.

As a specific example, the terminal device 100 displays the fare for the route from Tokyo Station to Ikebukuro Station as the fare when using an IC card (IC card priority), or when using cash (tickets). A character string (hereinafter sometimes referred to as “character string #21”) for designating whether the fare is to be displayed (cash is preferred) is displayed in the area AR23. To give a more specific example, the terminal device 100 displays a character string #21 including an area AR231 that alternatively displays "IC card priority" and "Cash priority".

The terminal device 100 also displays a character string (hereinafter referred to as may be described as "character string #22") is displayed in area AR24.

In addition, the terminal device 100, when changing trains on a route from Tokyo Station to Ikebukuro Station, uses a character string (hereinafter referred to as , may be described as “character string #23”) is displayed in the area AR25. To give a more specific example, when the user U1 moves in a hurry (“hurry”), when the user U1 moves a little in a hurry (“a little in a hurry”), the terminal device 100 may A character string #23 including an area AR251 for alternatively displaying the case where U1 moves slowly (“slowly”) and the case where user U1 moves a little slowly (“slightly slowly”) is displayed.

Here, in the example of FIG. 2, the user U1 wishes to display the fare with cash priority, presses the button B21, follows the character string #21, and says "check with cash priority" to the microphone included in the terminal device 100. Suppose you speak to it. In this case, the terminal device 100 performs voice recognition of the utterance from the user U1, and displays a screen C22 in which the display mode of the character strings #21 to #23 is updated based on the result of voice recognition. For example, the terminal device 100 displays the character string "check with cash priority", which is the voice recognition result, in the area AR22. In addition, the terminal device 100 changes the character color of "cash priority" corresponding to the utterance of the user U1 out of "IC card priority" and "cash priority" displayed in the area AR231 included in the character string #21. . In addition, the terminal device 100 changes the character color of the character string corresponding to "check with" uttered by the user U1 after the "cash priority" among the character strings included in the character string #21. Then, the terminal device 100 displays in the area AR23 a check mark indicating that the character string included in the character string #21 and the update process for each area have been completed.

Similarly, the terminal device 100 updates the display mode of the character strings #22 and #23 based on the speech recognition result. For example, the terminal device 100 changes the character color of the character string corresponding to "check with" uttered by the user U1 among the character strings included in the character string #22. Then, the terminal device 100 inserts into the character string #22 a character string corresponding to "cash priority" uttered by the user U1 before "check with". Note that the terminal device 100 does not update the display mode of the character string "considering the regular interval" included in the character string #22 because the user U1 does not include the character string.

In addition, for example, the terminal device 100 inserts into character string #23 the character string corresponding to "check with cash priority" uttered by user U1 among the character strings included in character string #22. Terminal device 100 does not include each character string displayed in area AR251 included in character string #22 and the character string "transfer" included in character string #22 in user U1's utterance. Processing for updating the display mode of these character strings is not performed.

After the above update process is completed, the terminal device 100 sends the updated character string #21 selected by the user on the screen C22 from among the updated character strings #21 to #23 to the service server 10 as a search query. Send. Then, the terminal device 100 displays, on the screen, information relating to transfer guidance from Tokyo Station to Ikebukuro Station, which includes a cash-preferred fare. In other words, the terminal device 100 displays, on the screen, information corresponding to the information displayed on the screen C14 of FIG. 1, which is narrowed down based on the new character string #21.

It should be noted that the terminal device 100 may accept changes in the information displayed in the area AR21. For example, when the area AR211 is pressed, the terminal device 100 deletes the character string "Tokyo" and accepts voice input of a new departure station. Further, when the area AR211 is pressed, the terminal device 100 deletes the character string "Ikebukuro" and accepts voice input of a new arrival station.

As described above, the terminal device 100 according to the embodiment displays a character string that is a search query candidate corresponding to a function that can be used in a service that the user uses, and allows the user to enter a search query according to the character string. Allows voice input. As a result, when the user performs an operation of calling a service function using the voice UI, the terminal device 100 according to the embodiment can perform the following operations: Even if the user does not know information such as "what functions can be operated by voice" in advance, it is possible to make the user recognize what functions can be used by voice input. That is, the terminal device 100 according to the embodiment can propose an appropriate search query for the service used by the user.

[2. Configuration of terminal device]
Next, the terminal device 100 for implementing the update process described above will be described with reference to FIG. FIG. 3 is a diagram illustrating a configuration example of a terminal device according to the embodiment; As shown in FIG. 3, the terminal device 100 has a communication section 110, an input section 120, an output section 130, and a control section 140. FIG.

(Regarding communication unit 110)
The communication unit 110 is realized by, for example, a NIC (Network Interface Card) or the like. The communication unit 110 is connected to the network N by wire or wirelessly, and transmits and receives information to and from the service server 10, the interpretation server 200 that executes processing for interpreting the content of information input by voice, and the like. For example, the interpretation server 200 provides a function of interpreting the content of voice input information via an API (Application Programming Interface).

(Regarding the input unit 120 and the output unit 130)
The input unit 120 is an input device that receives various operations from the user. For example, the input unit 120 is implemented by a keyboard, mouse, operation keys, and the like. The output unit 130 is a display device for displaying various information. In other words, the output unit 130 corresponds to the display screen of the terminal device 100 . For example, the output unit 130 is realized by a liquid crystal display or the like. When the terminal device 100 employs a touch panel, the input unit 120 and the output unit 130 are integrated.

(Regarding the control unit 140)
The control unit 140 is a controller, and various programs stored in a storage device inside the terminal device 100 are executed by a CPU (Central Processing Unit), an MPU (Micro Processing Unit), or the like, using the RAM as a work area. It is realized by being executed. When control information or an application that implements the above-described display processing is delivered from an application delivery server or the like, the control unit 140 operates according to the control information or application to implement the display processing.

Also, the control unit 140 is a controller, and is implemented by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array). The control unit 140 according to the embodiment has a voice input application 141, a first application 142, a second application 143, etc., as shown in FIG. The voice input application 141 realizes or executes the functions and actions of the updating process described above, and transmits to the service server 10 information that is the result of voice recognition for the user's voice input. The voice input application 141 also outputs information resulting from voice recognition to other applications such as the first application 142 and the second application 143 . An example of the functional configuration of the voice input application 141 will be described below with reference to FIG. FIG. 4 is a diagram illustrating a configuration example of a voice input application according to the embodiment. The voice input application 141 according to the embodiment has a display unit 1411, a reception unit 1412, an update unit 1413, a presentation unit 1414, and a transmission unit 1415, as shown in FIG. Realize or perform the function or action of a process.

(Regarding the display unit 1411)
The display unit 1411 displays a character string indicating a search query candidate input via voice and corresponding to a service used by the user. For example, in the examples of FIGS. 1 and 2, the display unit 1411 is a character string indicating search query candidates that can be input in the route search service provided by the service server 10, and is a search query candidate for the route search service. is displayed on the screen C12, the screen C21, or the like.

Also, the display unit 1411 may display a character string corresponding to a search function provided by the service. For example, in the examples of FIGS. 1 and 2, the display unit 1411 displays a character string indicating a search query candidate corresponding to a route search function in a search service provided by a route search service or a refinement function that can be used in a route search service. is displayed on the screen C12, the screen C21, or the like.

Also, the display unit 1411 may display a character string indicating a searchable target using a search function provided by the service. For example, in the example of FIG. 1, the display unit 1411 displays character strings related to routes that can be searched using the search function provided by the route search service on the screen C12, the screen C21, or the like.

Moreover, the display unit 1411 may display a character string including a display area in which a character string input via the user's voice is displayed. For example, in the example of FIG. 1, the display unit 1411 displays a character string #11 including areas AR131 and AR132 respectively displaying a departure station and an arrival station voice-inputted by the user U1. In addition, the display unit 1411 displays a character string #, which includes areas AR141 and AR142 for displaying the departure station and the arrival station input by voice by the user U1, and areas AR143 and AR144 for displaying the time input by the user U1 by voice. Display 12.

In addition, the display unit 1411 may display the types of information that can be input via voice in the display area. For example, in the example of FIG. 1, the display unit 1411 displays the information type "station" that can be input via voice in areas AR131, AR132, and areas AR141 and AR142. In addition, the display unit 1411 displays the type “number” of information that can be input via voice in the areas AR143 and AR144.

Note that when a character string that does not correspond to the type displayed in the display area is voice-inputted as a character string to be displayed in the display area, the updating unit 1413 described later displays the character string in the display area. No need.

Moreover, the display unit 1411 may display a character string including an area for displaying a plurality of character strings having a predetermined relationship. For example, in the example of FIG. 1, the display unit 1411 displays a character string #12 including an area AR145 that alternatively displays "departure" and "arrival". Further, in the example of FIG. 1, the display unit 1411 displays a character string #12 including an area AR145 that alternatively displays "departure" and "arrival". In the example of FIG. 2, the display unit 1411 displays a character string #21 including an area AR231 that alternatively displays "IC card priority" and "cash priority". In addition, in the example of FIG. 2, the display unit 1411 displays a character string #23 including an area AR251 that alternatively displays "quickly", "quickly", "slowly", and "slightly slowly". .

(Regarding reception unit 1412)
The reception unit 1412 receives input of information from the user via voice. For example, in the example of FIG. 1, the accepting unit 1412 accepts an utterance made into the microphone included in the terminal device 100 .

(Regarding update unit 1413)
The updating unit 1413 updates and displays the display mode of the character string based on the speech recognition result of the information received by the receiving unit 1412 . For example, in the example of FIG. 1, the updating unit 1413 changes the display mode of the character strings #11 and #12 based on the character string "last train from Tokyo to Ikebukuro" which is the result of speech recognition of the utterance from user U1. Update. Also, in the example of FIG. 2, the updating unit 1413 updates the display mode of the character strings #21 to #23 based on the character string "check with cash priority" that is the result of voice recognition of the utterance from the user U1. .

Further, the updating unit 1413 may update the display mode of the portion of the character string corresponding to the speech recognition result. For example, in the examples of FIGS. 1 and 2, the updating unit 1413 updates the display mode of the portions of the character strings #11, #12, and character strings #21 to #23 that correspond to the results of speech recognition.

In addition, the updating unit 1413 may highlight a character string that matches the speech recognition result among the character strings. For example, in the example of FIG. 1, the updating unit 1413 changes the character color of character strings #11 and #12 that match the speech recognition result of the utterance from user U1. Also, in the example of FIG. 2, the updating unit 1413 changes the character color of the character strings that match the voice recognition result of the utterance from the user U1 among the character strings #21 to #23.

Further, the update unit 1413 may display the character string indicated by the speech recognition result in the display area. For example, in the example of FIG. 1, the updating unit 1413 displays "Tokyo" uttered by the user U1 before "kara" in the area AR131, and displays "Ikebukuro ” is displayed in the area AR132.

Further, the updating unit 1413 may display one character string corresponding to the speech recognition result among the plurality of character strings in the area. For example, in the example of FIG. 1, the updating unit 1413 changes the character color of "last train" corresponding to the utterance of user U1 out of "first train" and "last train" displayed in area AR133 included in character string #11. change. In the example of FIG. 2, the updating unit 1413 selects "cash priority" corresponding to the utterance of user U1, out of "IC card priority" and "cash priority" displayed in the area AR231 included in the character string #21. change the text color of

Here, even if the result of voice recognition of the user's utterance does not match the character string displayed by the display unit 1411, if the contents of the interpreted result of voice recognition correspond to the character string, There may be a desire to update the display mode of the character string. Therefore, the update unit 1413 may update and display the display mode of the character string based on the result of interpretation by the external device. For example, the update unit 1413 displays the character string displayed by the display unit 1411 based on the interpretation of the information input by the user by voice, which is interpreted by the interpretation server 200 from the information input by the user by voice. Update the aspect. To give a specific example, when the content interpreted by the interpretation server 200 and the content indicated by the character string displayed by the display unit 1411 correspond, the updating unit 1413 changes the display mode of the character string.

(About presentation unit 1414)
Here, for example, if the speech from the user is not correctly recognized and part of the character string displayed by the display unit 1411 is not updated, or if part of the character string displayed by the display unit 1411 (e.g., essential items) ) is not uttered by the user, it is conceivable that the user wants to prompt for voice input again. Therefore, when the character string updated by the update unit 1413 satisfies a predetermined condition, the presentation unit 1414 presents the user with content for receiving input of new information via voice. For example, the presentation unit 1414 causes the update unit 1413 to update the character string corresponding to the service used by the user, which is the character string displayed by the display unit 1411 and corresponds to the essential items set for the service. If it is not updated, the user is presented with content indicating that necessary information has not been input by voice, and content for accepting input of new information via voice. As a specific example, the presentation unit 1414 presents content for accepting voice input of portions corresponding to essential items. Then, the updating unit 1413 updates the display mode of the portion corresponding to the essential item based on the information input by voice via the content presented by the presentation unit 1414 .

In addition, when there is a part of the character string whose display mode has not been updated, the presentation unit 1414 may present content for inputting information corresponding to the part. For example, when all of the character strings displayed by the display unit 1411 have not been updated by the update unit 1413, the presentation unit 1414 presents content for accepting voice input of the portion of the character string that has not been updated.

In addition, when the character string is not displayed in the area included in the character string and in which the character string input via the user's voice is displayed, the presentation unit 1414 displays the character string corresponding to the area. Content for entering information may be presented. For example, using FIG. 1 as an example, if the character string is not displayed in either of the areas AR131 and AR132 included in the character string #11, the presentation unit 1414 displays the information corresponding to the area AR131 or AR132. Present content for input. Further, using FIG. 1 as an example, the presentation unit 1414 displays “first train” and A content for selecting one of "last trains" is presented.

In addition, the presentation unit 1414 provides character string candidates to be displayed in an area that is included in the character string and in which the character string input via the user's voice is displayed. If there are multiple candidates for the character string based on, content may be presented for selecting one of the candidates for the character string. For example, when there are a plurality of conversion candidates for converting the information input by voice from the user into kanji, the presentation unit 1414 presents content for selecting one of the conversion candidates. If there are a plurality of place names corresponding to the information input by voice from the user, the presentation unit 1414 presents content for selecting one of the place names. As a specific example, when the user voice-inputs “Nakano”, the presentation unit 1414 selects “Nakano” located in Tokyo, “Nakano” located in Gunma Prefecture, and “Nakano” located in Nagano Prefecture. Contents for selecting one of the located "Nakano" are presented.

Note that when the display unit 1411 displays a plurality of character strings that are search query candidates, the presentation unit 1414 does not present the content to the user if any character string does not satisfy a predetermined condition. good. For example, using FIG. 1 as an example, the presentation unit 1414 updates the character string and each area included in the character string #11 among the character strings #11 and #12. It is not necessary to display the content for voice input of information about #12. In other words, if any of the character strings displayed by the display unit 1411 can be transmitted as a search query, the presentation unit 1414 does not have to present the content.

In addition, the presentation unit 1414 is not limited to content for accepting input of new information via voice, and accepts information input by means other than voice, such as the touch panel, keyboard, mouse, and operation keys of the terminal device 100. You may present content for

(Regarding the transmission unit 1415)
When a transmission request for a character string updated by the updating unit 1413 is received from the user, the transmitting unit 1415 transmits the updated character string as a search query to an external device that executes search processing corresponding to the service. . For example, in the example of FIG. 1, the transmission unit 1415 transmits the updated character string #11 selected by the user on the screen C13 to the service server 10 as a search query.

Note that the transmission unit 1415 may transmit the updated character string as a search query to an external device that executes search processing corresponding to a service different from the service currently being used by the user. For example, using FIG. 1 as an example, the transmission unit 1415 uses the updated character string #11 as a search query in response to an operation from the user, and the service server 10 provides a service different from the route search service. Send to Then, the terminal device 100 receives the search result from the service server 10 and displays it on the screen.

Further, the transmitting unit 1415 may transmit the information received by the receiving unit 1412 to an external device that executes processing for interpreting the contents of the voice. For example, the transmission unit 1415 transmits information input by voice to the interpretation server 200 . Then, the update unit 1413 executes update processing based on the contents interpreted by the interpretation server 200 .

[3. Update process flow]
Here, the procedure of update processing of the voice input application according to the embodiment will be described with reference to FIG. 5 . FIG. 5 is a flowchart illustrating an example of an update processing procedure according to the embodiment;

As shown in FIG. 5, the terminal device 100 displays a character string indicating search query candidates corresponding to the service used by the user (step S101). Subsequently, the terminal device 100 accepts input of information via voice (step S102). Subsequently, the terminal device 100 updates and displays the display mode of the character string based on the voice recognition result (step S103). Subsequently, the terminal device 100 determines whether or not the updated character string satisfies a predetermined condition (step S104). If the predetermined condition is satisfied (step S104; Yes), the terminal device 100 presents content for receiving input of new information via voice (step S105), returns to step S102, and repeats the process.

On the other hand, if the predetermined condition is not satisfied (step S104; No), the terminal device 100 determines whether or not a character string transmission request has been received from the user (step S106). If the transmission request has not been received (step S106; No), the terminal device 100 waits until the transmission request is received. On the other hand, when the transmission request is received (step S106; Yes), the terminal device 100 transmits the updated character string as a search query to the external device executing the search process (step S107), and ends the process.

[4. Modification]
The above-described embodiment is an example, and various modifications and applications are possible.

[4-1. Services Provided by Service Server 10]
In the above-described embodiment, an example in which the service server 10 provides a route search service has been shown, but the service provided by the service server 10 is not limited to such an example, and any service may be provided. The display unit 1411 may display a character string corresponding to the service provided by the service server 10. FIG. For example, when the service server 10 provides a shopping service, the display unit 1411 may display character strings corresponding to search functions available in the shopping service as search query candidates. As a specific example, the display unit 1411 displays a character string indicating the type of product sold in the shopping service, the manufacturer, the price range of the product, and the like.

[4-2. Update processing by update unit 1413]
In the above-described embodiment, an example in which the update unit 1413 performs update processing by changing the character color of a character string that matches the voice recognition result of the user's utterance among the character strings displayed by the display unit 1411. , the function of the updating unit 1413 is not limited to such an example. For example, the update unit 1413 may perform the character string update process in any manner as long as the user can determine whether or not the uttered content has been recognized by voice.

[4-3. Character String Displayed by Display Unit 1411]
In the above-described embodiment, the display unit 1411 displays a character string corresponding to the search function provided by the service used by the user, but the function of the display unit 1411 is not limited to such an example. For example, the display unit 1411 may display a character string based on what the user has uttered by activating a voice input application. For example, when the user activates a voice input application and inputs some information by voice, the transmission unit 1415 transmits the information to the interpretation server 200 . Then, the display unit 1411 displays a character string indicating search query candidates in the service corresponding to the result of the interpretation by the interpretation server 200 . As a specific example, when the user utters "I'm hungry," the display unit 1411 displays a character string indicating search query candidates for searching for foodstuffs in a shopping service, a restaurant search, and so on. Display a character string or the like that indicates search query candidates for searching for a store corresponding to the user's current location in the service.

[4-4. About processing mode]
Of the processes described in the above embodiments, all or part of the processes described as being automatically performed can be manually performed, and conversely, all of the processes described as being manually performed can be performed manually. Alternatively, some can be done automatically by known methods. In addition, information including processing procedures, specific names, and various data and parameters shown in the above text and drawings can be arbitrarily changed unless otherwise specified. For example, the various information shown in each drawing is not limited to the illustrated information.

Also, each component of each device illustrated is functionally conceptual, and does not necessarily need to be physically configured as illustrated. In other words, the specific form of distribution and integration of each device is not limited to the one shown in the figure, and all or part of them can be functionally or physically distributed and integrated in arbitrary units according to various loads and usage conditions. Can be integrated and configured.

Moreover, each of the embodiments described above can be appropriately combined within a range that does not contradict the processing contents.

[5. effect〕
As described above, the voice input application 151 included in the terminal device 100 according to the embodiment has the display unit 1411, the reception unit 1412, the update unit 1413, the presentation unit 1414, and the transmission unit 1415. The display unit 1411 displays a character string indicating a search query candidate input via voice and corresponding to a service used by the user. The reception unit 1412 receives input of information from the user via voice. The updating unit 1413 updates and displays the display mode of the character string based on the speech recognition result of the information received by the receiving unit 1412 . The updating unit 1413 updates and displays the display mode of the character string based on the speech recognition result of the information received by the receiving unit 1412 . If the character string updated by the update unit 1413 satisfies a predetermined condition, the presentation unit 1414 presents the user with content for receiving input of new information via voice. When a transmission request for a character string updated by the updating unit 1413 is received from the user, the transmitting unit 1415 transmits the updated character string as a search query to an external device that executes search processing corresponding to the service. .

As a result, the terminal device 100 according to the embodiment can recognize what functions can be used by voice input in the service used by the user. I can suggest.

Also, in the terminal device 100 according to the embodiment, for example, the display unit 1411 displays a character string corresponding to the search function provided by the service. The display unit 1411 also displays a character string indicating a searchable target using a search function provided by the service.

As a result, the terminal device 100 according to the embodiment can recognize what kind of search function can be used in the service used by the user, so that an appropriate search query can be proposed in the service used by the user. .

In addition, in the terminal device 100 according to the embodiment, for example, the updating unit 1413 updates the display mode of the part of the character string that corresponds to the voice recognition result. In addition, the updating unit 1413 highlights a character string that matches the speech recognition result among the character strings.

As a result, the terminal device 100 according to the embodiment can recognize whether or not the content uttered by the user has been correctly recognized as voice, thereby improving the convenience of voice input.

In addition, in the terminal device 100 according to the embodiment, for example, the display unit 1411 displays a character string including a display area in which a character string input via the user's voice is displayed. Then, the update unit 1413 displays the character string indicated by the speech recognition result in the display area. In addition, the display unit 1411 displays the types of information that can be input via voice in the display area.

As a result, the terminal device 100 according to the embodiment prompts the user to input the information by voice by displaying the type of information to be input by voice, and provides search results based on the arbitrarily input information. Therefore, convenience in voice input can be improved.

Also, in the terminal device 100 according to the embodiment, for example, the display unit 1411 displays a character string including an area for displaying a plurality of character strings having a predetermined relationship. Then, the updating unit 1413 displays one character string corresponding to the voice recognition result among the plurality of character strings in the area.

As a result, the terminal device 100 according to the embodiment can selectively display and allow the user to select information to be input by voice, thereby reducing the burden of voice input and improving convenience.

In addition, in the terminal device 100 according to the embodiment, for example, the transmission unit 1415 transmits information received by the reception unit 1412 to an external device that executes processing for interpreting the contents of the voice. Then, the updating unit 1413 updates and displays the display mode of the character string based on the result of the interpretation by the external device.

As a result, even if the result of speech recognition of the user's speech does not match the character string displayed by the display unit 1411, the terminal device 100 according to the embodiment can interpret the result of speech recognition as the character string. Since the display mode of the character string can be updated when it corresponds to the string, convenience in voice input can be improved.

In addition, in the terminal device 100 according to the embodiment, for example, when there is a portion of the character string whose display mode has not been updated, the presentation unit 1414 presents content for inputting information corresponding to the portion. do. In addition, when the character string is not displayed in the area included in the character string and in which the character string input via the user's voice is displayed, the presentation unit 1414 displays the character string corresponding to the area. Present content for entering information. In addition, the presentation unit 1414 provides character string candidates to be displayed in an area that is included in the character string and in which the character string input via the user's voice is displayed. If there are multiple candidates for the character string based on it, content for selecting one of the candidates for the character string is presented.

As a result, the terminal device 100 according to the embodiment does not correctly recognize the user's speech, and the character string displayed by the display unit 1411 is partially updated, or the character string displayed by the display unit 1411 is not updated. If the user does not speak a part of (essential items, etc.), it is possible to prompt voice input again, thereby improving the convenience of voice input.

[6. Hardware configuration]
Also, the terminal device according to each of the embodiments described above is implemented by a computer 1000 configured as shown in FIG. 6, for example. Hereinafter, the terminal device 100 will be described as an example. FIG. 6 is a hardware configuration diagram showing an example of a computer that implements the functions of the terminal device. Computer 1000 has CPU 1100 , ROM 1200 , RAM 1300 , HDD 1400 , communication interface (I/F) 1500 , input/output interface (I/F) 1600 and media interface (I/F) 1700 .

The CPU 1100 operates based on programs stored in the ROM 1200 or HDD 1400 and controls each section. The ROM 1200 stores a boot program executed by the CPU 1100 when the computer 1000 is started, programs dependent on the hardware of the computer 1000, and the like.

The HDD 1400 stores programs executed by the CPU 1100, data used by the programs, and the like. Communication interface 1500 receives data from another device via communication network 500 (corresponding to network N in the embodiment) and sends it to CPU 1100, and also transmits data generated by CPU 1100 via communication network 500 to other devices. Send to device.

The CPU 1100 controls output devices such as displays and printers, and input devices such as keyboards and mice, through an input/output interface 1600 . CPU 1100 acquires data from an input device via input/output interface 1600 . Also, CPU 1100 outputs the generated data to an output device via input/output interface 1600 .

Media interface 1700 reads programs or data stored in recording medium 1800 and provides them to CPU 1100 via RAM 1300 . CPU 1100 loads such a program from recording medium 1800 onto RAM 1300 via media interface 1700, and executes the loaded program. The recording medium 1800 is, for example, an optical recording medium such as a DVD (Digital Versatile Disc) or a PD (Phase change rewritable disc), a magneto-optical recording medium such as an MO (Magneto-Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory. etc.

For example, when the computer 1000 functions as the terminal device 100 , the CPU 1100 of the computer 1000 implements the functions of the control unit 140 by executing programs loaded on the RAM 1300 . Further, each data in the storage device of the terminal device 100 is stored in the HDD 1400 . CPU 1100 of computer 1000 reads these programs from recording medium 1800 and executes them, but as another example, these programs may be obtained from another device via a predetermined communication network.

[7. others〕
As described above, some of the embodiments of the present application have been described in detail based on the drawings. It is possible to carry out the invention in other forms with modifications.

In addition, the configuration of the above-described terminal device 100 can be flexibly changed, for example, by calling an external platform or the like using an API (Application Programming Interface), network computing, or the like, depending on the function.

In addition, the "unit" described in the claims can be read as "means", "circuit", or the like. For example, the display section can be read as display means or a display circuit.

100 terminal device 110 communication unit 120 input unit 130 output unit 140 control unit 1411 display unit 1412 reception unit 1413 update unit 1414 presentation unit 1415 transmission unit 142 first application 143 second application

Claims (16)

a display procedure for displaying a character string indicating a search query candidate input via voice and corresponding to a service used by a user;
a reception procedure for receiving input of information via voice from the user;
and an update program for causing a computer to execute an update procedure for updating and displaying the display mode of the character string based on a result of voice recognition of information accepted by the acceptance procedure. The display procedure is
2. The update program according to claim 1, wherein said character string corresponding to a search function provided by said service is displayed. The display procedure is
3. The update program according to claim 1, wherein the character string indicating an object that can be searched using a search function provided by the service is displayed. The update procedure includes:
The update program according to any one of claims 1 to 3, updating a display mode of a part of the character string corresponding to the speech recognition result. The update procedure includes:
5. The update program according to claim 4, wherein, among the character strings, a character string that matches the speech recognition result is highlighted. The display procedure is
displaying the character string including a display area in which the character string input via the user's voice is displayed;
The update procedure includes:
The update program according to any one of claims 1 to 5, wherein the character string indicated by the speech recognition result is displayed in the display area. The display procedure is
7. The update program according to claim 6, wherein a type of information that can be input via voice is displayed in the display area. The display procedure is
displaying the character string including an area for displaying a plurality of character strings having a predetermined relationship;
The update procedure includes:
The update program according to any one of claims 1 to 7, wherein one character string corresponding to the result of the speech recognition is displayed in the area among the plurality of character strings. and further executing a presentation procedure of presenting to the user content for receiving input of new information via voice when the character string updated by the update procedure satisfies a predetermined condition. The update program according to any one of claims 1 to 8. The presentation procedure includes:
10. The update program according to claim 9, wherein if there is a portion of the character string whose display mode has not been updated, the content for inputting information corresponding to the portion is presented. The presentation procedure includes:
To input information corresponding to the area included in the character string, if the character string is not displayed in the area where the character string input through the user's voice is displayed 11. The update program according to claim 9 or 10, presenting the content of . The presentation procedure includes:
A character string candidate to be displayed in an area that is included in the character string and in which the character string input via the user's voice is displayed, the character string based on the result of the speech recognition. 12. The update program according to any one of claims 9 to 11, wherein when there are a plurality of candidates, the content for selecting one of the candidates for the character string is presented. further executing a first transmission procedure for transmitting the information accepted by the acceptance procedure to an external device that executes a process of interpreting the contents of the voice;
The update procedure includes:
The update program according to any one of claims 1 to 12, wherein the display mode of the character string is updated and displayed based on the result of interpretation by the external device. When a transmission request for the character string updated by the updating procedure is received from the user, the updated character string is transmitted as a search query to an external device that executes search processing corresponding to the service. 14. The update program according to any one of claims 1 to 13, further causing a 2 transmission procedure to be executed. a display unit that displays a character string that is a search query candidate input via voice and that indicates a search query candidate corresponding to a service used by a user;
a reception unit that receives input of information via voice from the user;
and an updating unit that updates and displays a display mode of the character string based on a result of voice recognition of the information received by the receiving unit. A computer implemented update method comprising:
a display step of displaying a character string indicating a search query candidate input via voice, which is a search query candidate corresponding to a service used by a user;
a receiving step of receiving input of information via voice from the user;
and an updating step of updating and displaying the display mode of the character string based on a result of speech recognition of the information received by the receiving step.

Images