JP5035208B2 - Information processing apparatus, interface providing method, and program - Google Patents

Description

  The present invention allows a user to select each selection item included in each of a plurality of menus having a hierarchical structure from the first layer to the n-th layer (n is an arbitrary number), so that the menu having the selection item is separated from another menu. It is related with the user interface comprised so that it may change to a menu.

  Conventionally, in order to realize selection of selection items by the user's voice without being confused about what the user should speak, a menu consisting of items corresponding to user's button operations is displayed on the display unit and voice input And a system (conventional system 1) is proposed in which the processing corresponding to the item identified by the input voice is executed (refer to Patent Document 1).

On the other hand, when it is difficult for the user to start a voice input by performing a button operation, or when the user feels troublesome in the button operation, the user does not perform the button operation explicitly. A system (conventional system 2) that always recognizes utterances has been proposed (for example, Patent Document 2).
JP 2007-171809 A JP 2000-194393 A

  However, the above-described conventional system 1 cannot be applied to an always-recognition system in which no button operation is performed, as in the conventional system 2, because an item to be identified thereafter is not displayed as a menu unless there is a button operation by the user. .

  Furthermore, in a user interface device having a plurality of menus in a hierarchical structure, when the user wishes to perform voice input for selecting a plurality of hierarchies at once, the user selects the selection items of each hierarchy in the correct order and the correct selection route according to the hierarchy. I need to speak.

  However, when the conventional menu display technique represented by the conventional system 1 is used, the menu is not updated until the voice input is completed and the selection item is confirmed by voice recognition. There is a problem that it becomes difficult to continuously utter a plurality of selection items on a route, and the convenience as a user interface is lowered.

  The present invention has been made to solve such problems, and its purpose is to improve convenience as a user interface for realizing selection of a selection item by a user's voice as compared with the conventional one. is there.

In order to solve the above problems, a user interface device for implementing the operation according to the user's voice, better to first as configuration shown below.
In this configuration, by causing the user to select each selection item included in each of a plurality of menus having a hierarchical structure from the first hierarchy to the nth hierarchy (n is an arbitrary number), the menu having the selection item is changed to a different hierarchy. This is a user interface device implemented with a user interface configured to transition to the menu.

  Then, among each of the plurality of menus, the menu display means for displaying the current menu, which is the menu to be displayed at that time, on the display unit and the sound input from the outside are in the i-th layer (1 ≦ i <n). Item estimation means for estimating which selection item corresponds to which selection route among the selection routes including selection items that can be selected before the transition from the menu to the n-th layer menu, and the current Menu transition means for transitioning the menu to a menu of a different hierarchy to be displayed when the selection item estimated by the item estimation means is selected, and the menu display means includes the menu transition means. An image indicating the menu that has been changed as the current menu is displayed on the display unit.

  In the information processing apparatus configured as described above, first, it is assumed that the user's voice corresponds to which selection item in which selection route, and the selection item thus estimated is considered to be selected by the user. ing. Then, when the selected item is selected, the current menu is shifted to a menu of another level to be displayed, and this is displayed on the display unit.

In this way, the menu display can be changed at any time with the selection item estimated from the voice input up to the present time as the item selected by the user.
For this reason, for the user, while viewing the menu displayed on the display unit, the user can select any of the selection items and utter the contents in succession. As a result, it is highly convenient in that it does not take time and effort to utter items in the next layer after waiting for the voice recognition to be completed and the menu to be changed as before. .

  In this configuration, the current menu to be displayed on the display unit may be continuously displayed after being displayed once. However, when a certain period of time has elapsed without input of voice, the user can select a selection item. Since it can be said that the display is interrupted or stopped, the menu display may be erased after a certain period of time from the viewpoint of improving the visibility of the display area in the display unit.

For this purpose, for example, it may be like the second configuration shown below the structure. In this configuration, the menu display means erases the display of the menu by the display unit when an external audio input has not been made for a predetermined period or longer after the image indicating the menu is displayed on the display unit.

  With this configuration, the menu display can be erased when a period in which no external audio is input continues for a predetermined period or longer, and thereafter, the visibility of the display area in the display unit can be improved.

In the above structure, in a state in which the menu on the display section is not displayed, as a method for displaying a new menu, for example, it may be like the third configuration shown below the structure.

In this configuration, the menu display means displays the current menu on the display unit when a voice is input from the outside while the menu is not displayed on the display unit.
With this configuration, when the menu is not displayed, such as immediately after the information processing apparatus is started, the user can display the menu by performing some kind of utterance regardless of the contents. High information processing apparatus can be realized. Further, as in the second configuration, the current menu can be displayed even when the user utters after the display has been erased for a predetermined period of time during which no sound is input from the outside. An information processing apparatus with high convenience can be realized.

  In addition, the transition of the current menu in each of the above configurations may be performed only on the basis of sound input from the outside. However, if a certain period of time elapses without any voice input after the current menu transitions based on the voice input from the outside, it can be said that the selection of the selection item by the user is interrupted or stopped. In this case, thereafter, when the user tries to select an item again, the user forgets which menu is the current menu before the interruption or cancellation, and there is a risk of confusion in selecting the selection item.

Therefore, after a transition from the current menu based on the sound input from the outside, when a certain period of time has passed without any sound input, the selection of the selection item is performed again from a certain same menu, for example, the top menu. It can be said that it is preferable to start. For this purpose, for example, it may be like the fourth configuration shown below each structure.

In this configuration, the menu transition means transitions the current menu to a first-level menu when an external audio input has not been made for a predetermined period or longer.
With this configuration, even if the current menu is changed based on the sound input from the outside, the current menu is displayed in the highest hierarchy (the first layer) when the period in which the sound is not input from the outside continues for a predetermined period or longer. Since the menu is returned to the (one layer) menu, that is, the top menu, the selection item may be started from the top menu when selecting the selection item again.

  As a result, even if the user interrupts or cancels the selection of the selection item, it is only necessary to select the selection item from the menu of the first hierarchy when the selection is resumed, thereby preventing confusion when selecting the selection item. As a result, convenience as a user interface can be enhanced.

  Further, in each of the above configurations, estimation of which selection item in which selection route corresponds to the voice input from the outside may be started when the voice input from the outside is started. .

  The “external” to which the sound is input is outside the user interface device, and may be the microphone as long as the sound can be input via the microphone. If there is a route for inputting voice, this route may be used.

In addition, it may be estimated how the selected voice is input from which voice is input from the outside. As a specific example, for example, the following fifth configuration may be used. it is conceivable to make adult.

  In this configuration, the voice inputted from the outside is compared with each utterance pattern in the utterance pattern dictionary storing the user's utterance pattern for selecting the selection item, and the similarity as a result of the comparison is a predetermined threshold. Voice recognition means for outputting, as a recognition result, an utterance pattern that is equal to or greater than a value, and the item estimation means is a selection path comprising selection items corresponding to utterance patterns sequentially recognized by the voice recognition means. The latest selection item in the selected route is output as an estimation result.

  With this configuration, the hypothesis information at the present time in the hypothesis search formed by each utterance pattern up to the result of known speech recognition is used to estimate what the externally input speech is about to be uttered. In addition, it is possible to estimate which selection item is finally selected in which selection route.

Further, in this configuration, it may be like the sixth configuration shown below.
In this configuration, the item estimation unit corresponds to the utterance pattern with the highest similarity when there are multiple types of utterance patterns that have a similarity level equal to or higher than a threshold value at the time of recognition by the voice recognition unit. For the selected route including each selected item, the latest selected item in the selected route is output as an estimation result.

  With this configuration, every time speech recognition is performed, even if there are multiple types of utterance patterns that have a similarity level equal to or higher than the threshold value during the recognition, the selection corresponding to the utterance pattern with the highest similarity is selected from among them. It is possible to estimate a selection route including each item.

  Thus, it can be said that the utterance pattern with the highest similarity corresponds to the selection route and the selection item that are most likely as the actual utterance contents, and thus the selection route and the selection item can be estimated with high accuracy.

  As the similarity of a certain utterance pattern in this configuration, for example, the likelihood of a probability model corresponding to the utterance content of the utterance pattern, a value obtained by normalizing the likelihood with the processing time of a hypothesis search, or the like can be used.

Also, the transition of the current menu in each of the above configuration, for example, may be realized by updating the information indicating the contents, as the configuration for the above each configured as a seventh configuration shown below Good.

  In this configuration, the menu transition means updates the menu by updating the current information indicating the current menu to indicate a menu of a different hierarchy to be displayed based on the selection item estimated by the item estimation means. Transition. The menu display means displays a menu indicated by the current information on the display unit.

With this configuration, the current menu can be changed by updating the current information stored in a predetermined storage area.
In addition, the transition of the current menu in each configuration described above may be performed even when the user receives an operation on the operation unit.

  By the way, in each said structure, when displaying the menu by a display part, it is good also as making it always display in the same display mode, but you may make it change the display mode according to surrounding environment.

For example, it is conceivable to change the display mode depending on whether or not the input voice from the outside is the voice along the selected route.
As a structure for this, it is conceivable to make the eighth configuration shown below each structure.

  In this configuration, on the basis of the voice input from the outside, the current menu is shifted by the trust specifying means for specifying the reliability that the voice is the voice of the content along the selected route, and the menu transition means. And a first mode determining unit that determines a display mode of the menu according to the reliability specified by the trust specifying unit. Then, the menu display means displays the menu that has been changed as the current menu on the display unit in the display mode determined by the first mode determination unit for the menu.

  If it is this structure, according to the reliability that the input audio | voice from an audio | voice input part is the audio | voice of the content along the selection path | route mentioned above, the display mode of the image which shows a menu can be varied.

  The “reliability” in this configuration is, for example, a method of preparing a competition model and calculating a likelihood ratio as in JP-A-11-85188 (hereinafter referred to as “Patent Document 3”), or a maximum similarity. What is necessary is just to specify by using the method of using the value corresponding to the magnitude | size of the difference of the similarity degree of the hypothesis to have and another hypothesis.

In addition, any display mode may be used in this configuration. For example, a display mode based on the size of an image showing a menu is conceivable. The configuration for this is shown below. it is conceivable to ninth as configuration.

  In this configuration, the first aspect determining means determines the size of the display area of the current menu according to the reliability specified by the reliability specifying means, and the menu display means Are displayed on the display unit in a size corresponding to the display area determined by the first mode determining means.

With this configuration, the higher the reliability in the input voice from the voice input unit, the larger the display area in the menu.
Further, as a surrounding environment that is a factor for changing a display mode of an image showing a menu, a command from the outside of the information processing apparatus can be considered, and a display mode can be changed depending on the command.

As a structure for this, it is conceivable to make the tenth configuration shown below each structure.
In this configuration, after the current menu is transitioned by the menu transition means, second mode determining means for determining the display mode of the menu in response to an external command is provided. Then, the menu display means displays the menu changed as the current menu on the display unit in the display mode determined by the second mode determination unit for the menu.

If it is this structure, according to the command from the outside, the display mode of the image which shows a menu can be varied.
As the “command from the outside” in this configuration, for example, a result of detecting whether or not the predetermined device is operating by receiving voice input through communication with a predetermined device that operates by inputting a voice by the user Can be considered.

For this purpose, the above configuration may be obtained in the eleventh configuration of the below.
In this configuration, external voice input determination means for determining whether or not the predetermined device is operating by receiving a voice input through communication with an external predetermined device (external device) that operates by inputting a user's voice. , Is provided. The second mode determining means determines that the current menu should be displayed when the external audio input determining means determines that no audio input is performed on the external apparatus side, while the external apparatus side When it is determined that the voice input is performed in step S1, it is determined that the current menu is not displayed.

With this configuration, it is possible to prevent menu transitions when a predetermined device that operates by receiving voice input by a user is operated by receiving voice input.
As described above, the fact that the predetermined device is operating by receiving voice input is likely to be uttered regardless of the voice input to the information processing apparatus, and such irrelevant voice is input. If the subsequent processing is performed, menu transitions that are not intended by the user are performed.

  Therefore, as described above, when the predetermined device is operated by receiving voice input, the menu display is not performed, so that the menu transition not intended by the user can be prevented. .

  In addition, as the above-mentioned “command from the outside”, for example, a result of detecting that the operation unit of the information processing apparatus or a predetermined apparatus connected to the information processing apparatus is operated by the user is used. Conceivable.

For this purpose, the configuration of the tenth configuration or the 11 may be obtained in the twelfth configuration shown below.
In this configuration, operation detecting means for detecting that an operation unit of the information processing apparatus or a predetermined apparatus connected to the information processing apparatus is operated by a user is provided. The second mode determining means determines that the current menu should be displayed when it is determined that the operation detecting means is not operated, and the operation detecting means is operated. When it is determined that the current menu is present, it is determined that the current menu is not displayed.

  With this configuration, it is possible to prevent menu transition when the operation unit of the information processing apparatus or a predetermined apparatus connected to the information processing apparatus is operated.

  As described above, when the operation unit of the information processing apparatus or a predetermined device connected to the information processing apparatus is operated, the utterance may be performed regardless of the menu selection of the information processing apparatus. If such an irrelevant voice is input and the subsequent processing is performed, menu transition unintended by the user is performed.

  Therefore, as described above, it is possible to prevent menu transitions that are not intended by the user from being performed by preventing menu transitions when the operation unit is operated.

Further, as the above-mentioned “command from the outside”, for example, it is conceivable to use a result of detecting the number of users located around the information processing apparatus.
For this purpose, the configuration of any of the above tenth to twelfth may be obtained in the first 13 Configuring shown below.

  In this configuration, there is provided user detection means for detecting the number of users located around the information processing apparatus. The second aspect determining means determines the size of the display area of the current menu as a normal size when the user detecting means detects that only one user is located. On the other hand, when it is detected that a plurality of users are located, the size of the display area of the current menu is determined to be smaller than usual.

  With this configuration, when it is detected that only one user is located, the display area of the current menu is set to a normal size, but it is detected that a plurality of users are located. In this case, the display area of the current menu can be made smaller than usual.

  As described above, when a plurality of users are located in the vicinity, a menu displayed on the display unit is not necessarily necessary information from a user other than operating the information processing apparatus by voice. Therefore, by reducing the menu display area in such a case as in the above configuration, the visibility of the display unit for other users is reduced compared to a configuration in which such a display mode is not changed. This can be prevented.

  In this configuration, in order to detect the number of users located in the vicinity, a sensor is arranged in the vicinity of the area where the user can be located, and the number of users is determined based on the detection result. What is necessary is just to detect. In addition, it is only necessary to detect the number of users by photographing an area where the user can be located in the vicinity with a camera and specifying the user included in the video by image processing.

In the above structures, 14 as configuration of the processing means for executing a predetermined process assigned to the estimated selected item by the item estimating means, it may be provided with a .

  With this configuration, when a selection item to which a predetermined process is assigned in a menu at the lowest layer among the selection items in each menu is considered to be selected, the assigned process can be executed. .

Further, in order to solve the above-mentioned problem, the selection is performed by causing the user to select each selection item included in each of a plurality of menus having a hierarchical structure from the first hierarchy to the n-th hierarchy (n is an arbitrary number). item may be a user interface provided of how to provide a user interface that is configured to transition to a menu of another hierarchical menu having.

  In this interface providing method, a menu display procedure for displaying a current menu, which is a menu to be displayed at that time, among the plurality of menus on the display unit, and audio input from the outside are in the i-th layer (1 ≦ i An item estimation procedure for estimating which selection item corresponds to which selection route among the selection routes of the selection items that can be selected before the transition from the menu of <n) to the menu of the nth layer; A menu transition procedure for transitioning the current menu to a menu of another hierarchy to be displayed when the selection item estimated by the item estimation procedure is selected. In the menu display procedure, an image showing the menu that has been changed as the current menu in the menu transition procedure is displayed on the display unit.

With this interface providing method, the same operations and effects as those of the user interface device according to the first configuration described above can be obtained.
Note that this interface providing method may be a method in which each means in the user interface device according to any one of the second to fourteenth configurations described above is realized as a procedure. The same operations and effects as those of the user interface device according to the above configuration can be obtained.

In order to solve the above problems, even if a program for executing various processing procedures for functioning as all the means according to any one of the first to 14 described above in a computer system Good.

A computer system controlled by this program can constitute a part of the user interface device according to any one of the first to fourteenth configurations.
Note that the above-described program is composed of an ordered sequence of instructions suitable for processing by a computer system, and includes a user interface, an information processing apparatus, and a user using the same via various recording media and communication lines. Etc. are provided.

Embodiments of the present invention will be described below with reference to the drawings.
(0) Overall Configuration The information processing apparatus 1 is mounted on a navigation device in order to realize a user interface of a known navigation device. As shown in FIG. 1, a control including a CPU, a ROM, a RAM, and the like. 1 is a well-known computer system comprising a unit 10 and an input / output interface (I / O) 20, and includes a storage unit 2 for storing various information, an operation unit 3 for receiving operations by a user, and various types of information. Are connected to a display unit 4 for displaying the voice, a voice input unit 6 for controlling voice input via the microphone 5, and the like.

  Among these, the control part 10 performs various processes according to the program memorize | stored in ROM, and whether a user's speech voice is contained by the input level (sound volume) of the voice via the voice input part 6 is contained. The voice detection means 31 for detecting the sound, the item estimation means 33 for estimating a selection item (to be described later) indicated by the voice input via the microphone 5, and the content of the voice input via the microphone 5 is well-known voice recognition The voice recognition means 35 for analyzing by the above, the processing execution means 37 for executing the process according to the analysis result of the voice recognition means 35, the menu transition means 38 for changing the menu to be displayed based on the item estimation result by the item estimation means 33, It functions as a menu display unit 39 for controlling the display of the menu by the display unit 4. With these functions, the control unit 10 realizes a user interface of the navigation device.

  In this user interface, by allowing the user to select each selection item included in each of a plurality of menus having a hierarchical structure from the first layer to the nth layer (n is an arbitrary number), the user interface is separated from the menu having the selection item. It is configured to transition to a hierarchical menu.

  Specifically, after the user utters a sound toward the microphone 5, the first level menu (top menu) is displayed on the display unit 4 (see screen A in FIG. 2), and can be selected from this menu thereafter. When the sound generated from any one of the selection items is input from the microphone 5, it is assumed that the selection item has been selected, and the display content is changed to a menu of a different hierarchy to be shifted by the selection of the selection item. (Refer to screens B and C in FIG. 2). And the process corresponding to the selection item selected in the menu of the lowest layer will be implemented.

Hereinafter, for the information processing apparatus 1 configured as described above, embodiments in which processing procedures executed by the control unit 10 are different will be described in order.
(1) First Embodiment (1-1) Instruction Accepting Process First, after the information processing apparatus 1 is activated, the procedure of the instruction accepting process that the CPU of the control unit 10 repeatedly executes according to the program stored in the ROM. This will be described with reference to FIG.

  When this instruction receiving process is activated, the process first waits until voice input via the voice input unit 6 is started (s110: NO). Here, it is determined that the user's utterance has been started when the level of the sound detected by the sound detection means 31 becomes a certain level or more.

Thereafter, when the user's utterance is started (s110: YES), a display content determination process described later is performed (s120).
In this display content determination processing, the selection that can be selected until the voice input through the microphone 5 up to that point changes from the menu in the i-th layer (1 ≦ i <n) to the menu in the n-th layer. Of the selection paths for each item, it is estimated which selection item corresponds to which selection path, and the menu to be displayed on the display unit 4 is determined according to the estimation result.

  Next, a menu display process for displaying a menu on the display unit 4 is performed based on the items determined in s120 (s130). Here, the menu determined in s120 is displayed on the display unit 4 by the menu display means 39.

  When the menu is not determined in s120, the menu display unit 39 deletes the menu displayed on the display unit 4 at that time. Here, “the menu has not been determined” refers to a case where it has been determined that the menu display should be erased, or a case where there is no menu to be displayed.

  Next, the voice recognition means 35 checks whether or not the voice recognition for the user's voice should be terminated (s150). Here, it is determined that the voice recognition should be terminated if the voice input detection by the voice detection unit 31 described above has not been performed for a predetermined period or longer.

  When it is determined in s150 that the voice recognition is not to be terminated (s150: NO), the process returns to s120, and thereafter, the above steps s120 to s150 are repeated until the voice recognition is terminated. Is called.

  Then, when the situation in which voice recognition should be terminated is reached, this is determined in s150 (s150: YES), and the voice recognition means 35 is input through the microphone 5 by that time, and the internal memory of the CPU or By performing well-known voice recognition on the voice stored in the RAM, the character string indicated by the voice is specified (s152).

  Then, after the processing execution unit 37 executes a predetermined process assigned to the selected item based on the selected item corresponding to the character string specified in s152 (s160), the process returns to s110.

  In this s160, when there is no process assigned to the selected item, for example, the selected item is not a selected item in the n-th layer menu, the currently displayed menu and the current time are displayed as history information ( (Current information) is stored in a predetermined area of the memory or RAM (if already stored, the history information is updated), and the process returns to s110.

In the present embodiment, after it is determined that the voice input has been started in s110, information indicating the voice thus input is accumulated in the memory or RAM, and when the process returns to s110, The audio information accumulated up to this point is deleted.
(1-2) Display Content Determination Process Next, the processing procedure of the display content determination process that is s120 of the instruction reception process will be described with reference to FIG.

In this display content determination process, first, the speech recognition means 35 generates hypothesis information described later (s210).
Here, first, the voice indicated by the information accumulated at this time, that is, the voice inputted through the microphone 5 up to that time, the acoustic / linguistic probability model stored in advance and the utterance to be described later The position (current position) on the utterance pattern dictionary for the hypothesis (see “No. 1” in FIG. 5) that compares the pattern dictionary with a known hypothesis search and indicates the utterance pattern with the highest similarity as the comparison result. Information indicating the similarity and the current time is generated as hypothesis information. As the similarity, for example, the likelihood of the probability model corresponding to the utterance content of the utterance pattern, a value obtained by normalizing the likelihood with the processing time of the hypothesis search, or the like can be used.

  The utterance pattern dictionary described above stores utterance patterns representing how a user utters in order to select a selection item or a selection route. In this embodiment, as shown in FIG. It is represented in the form of an effective graph defined by the connection relationship for each word.

  In this utterance pattern dictionary, a plurality of assumed utterance patterns are decomposed into units of words, and the words are connected to each other so that menus in the i-th layer (1 ≦ i <n) can be changed to menus in the n-th layer. Selection paths of selection items that can be selected before transitioning to are formed.

Next, the speech recognition means 35 stores the hypothesis information generated in s210 in the hypothesis information storage area in the memory (s220).
Next, the item estimating means 33 determines the final current position in the utterance pattern dictionary according to the following procedure based on the hypothesis information stored in s220 (s230).

  Here, first, a predetermined initial position is set as a candidate position to be a final current position candidate (s310). In the present embodiment, a position where no word exists is defined as an initial position as a position corresponding to the top menu defined as the menu of the first hierarchy (see “initial position” in FIG. 6), and this position is set as a candidate position. Is done.

  Subsequently, whether or not history information (current information) is stored in the memory or RAM at this time is checked (s320). If history information is stored (s320: YES), this history information indicates this. It is checked whether or not the difference between the current time and the actual current time, that is, the elapsed time T0 after the history information is generated is equal to or greater than a predetermined threshold THs (THs ≦ To) (s330).

The “threshold value THs” is determined as an elapsed time that can be reached when selection of a selection item is interrupted or stopped after history information is generated.
When it is determined in s330 that the elapsed time T0 is less than the threshold value THs (T0 <THs) (s330: NO), the current position indicated by the history information is set as a candidate position (s340). ), The process proceeds to the next process (s350).

  If it is determined in s320 that no history information is stored (s320: NO), or if it is determined in s330 that the elapsed time T0 is greater than or equal to the threshold value THs (s330: YES), The process proceeds to the next process (s350) without performing the above s340.

  Next, it is checked whether hypothesis information is stored in the memory at this time (s350). If hypothesis information is stored (s350: YES), the similarity r indicated by the hypothesis information is a predetermined value. It is checked whether it is larger than the minimum value THa (THa <r) (s360).

  If it is determined in s360 that the similarity r is greater than the minimum value THa (s360: YES), the consistency of the current position indicated by the hypothesis information is checked (s370). Here, the current position indicated by the hypothesis information is located at a position that can be reached by following the utterance pattern dictionary in the forward direction from the current position indicated by the history information stored in the memory at this time. It is determined that there is consistency.

  If it is determined in s370 that the current positions are consistent (s370: YES), the current position indicated by the hypothesis information is set as a candidate position (s380), and then the process is the next process (s400). Migrate to

  If it is determined in s370 that there is no consistency between the current positions (s370: NO), whether or not the similarity r indicated by this hypothesis information is greater than a predetermined threshold value THb (THb <r). Is checked (s390). This “threshold value THb” is determined as a value larger than the threshold value THa.

  When it is determined in s390 that the similarity r is greater than the threshold value THb (s390: YES), the process proceeds to s380, and the current position indicated by the hypothesis information is set as a candidate position.

  Further, when it is determined in s350 that no hypothesis information is stored (s350: NO), when it is determined in s360 that the similarity r is equal to or lower than the minimum value THa (s360: NO), or When it is determined in s390 that the similarity r is equal to or less than the threshold value THb (s390: NO), the process proceeds to the next process (s400) without performing the above s380.

  Then, the candidate position at this time is determined as the final current position (s400). Thus, after the final current position in the utterance pattern dictionary in s310 to s400 is determined, the item estimation unit 33 determines a menu to be displayed on the display unit 4 based on the current position (s240).

  Since the utterance pattern dictionary defines the utterance pattern assumed in advance for selecting the selection item by the connection relation for each word, the final current position in the utterance pattern dictionary is any selected route. The selection items arrived along are shown.

  For this reason, in this s240, the selection item corresponding to the word at the final current position is regarded as the selection item selected by the user, and there is a menu of another level to be shifted when the selection item is selected. If so, the menu is determined as a menu to be displayed on the display unit 4.

  Note that if no hypothesis information is stored and no history information is stored, such as immediately after voice input is started, a predetermined initial position is set as a candidate position. As a menu to be displayed, the first layer menu corresponding to the initial position is selected.

  Next, the menu transition means 38 transitions the current menu, which is the menu to be displayed at that time, to the menu determined in s240 (s250). Here, the current menu and the current time are stored in a predetermined area of the memory or RAM (if already stored, the contents are updated), and the current menu changes accordingly.

  Next, the menu display means 39 checks whether or not there has been an input of sound through the microphone 5 within a certain period in the past (s260). Here, if the voice input detection by the voice detection unit 31 described above is performed within a certain period, it is determined that there is a voice input through the microphone 5 within the past certain period, while the voice input is detected. Is not performed within a certain period, it is determined that there is no input of sound via the microphone 5 within the past certain period.

The “certain period” is defined as an elapsed time that can be reached when selection of a selection item by the user is interrupted or stopped.
If it is determined in s260 that there has been an input of sound through the microphone 5 within a predetermined period in the past (s260: YES), after determining that the menu should be displayed (s270), the process Returns to the instruction reception process (shifts to s130).

On the other hand, if it is determined in s260 that there is no voice input via the microphone 5 within a certain period in the past (s260: NO), it is determined that the menu display should be deleted ( s280), the process returns to the instruction receiving process (shifts to s130).
(1-3) Action and Effect In the information processing apparatus 1 configured as described above, first, it is estimated which user's voice corresponds to which selection item in which selection route, and the selection thus estimated. The item is regarded as being selected by the user (s210 in FIG. 4). At this time, “which selection item corresponds to which selection route” is input from the outside using hypothesis information of the hypothesis search formed by each utterance pattern up to the result of known speech recognition. It is estimated what the selected voice is going to utter, and finally which selection item in which selection route is to be selected.

  Then, when the selected item is selected, the current menu is shifted to a menu of another hierarchy to be displayed (s230 to s250 in the figure), and this is displayed on the display unit 4 (s130 in FIG. 3). As described above, the display of the menu can be changed as needed in accordance with the item actually selected by the user.

  For this reason, the user simply selects a selection item from the menu displayed on the display unit 4 and utters the contents in order, and the menu is changed to a corresponding menu. As a result of being able to change the display (see FIG. 2), it does not take time and effort to utter the item of the next layer after waiting for the voice recognition to be completed and the menu to be changed as in the past. Convenient in terms.

  In the above embodiment, the menu display can be erased (s280 in FIG. 3 and s130 in FIG. 3) when a period in which no external sound is input continues for a predetermined period or longer (s260 “NO” in FIG. 4). ).

  The current menu to be displayed on the display unit 4 may be displayed continuously after it is displayed once. However, when a certain period of time has elapsed without voice input, selection of the selection item by the user is interrupted. Since it can be said that it has been canceled, it is desirable to delete the display of the menu after a certain period of time from the viewpoint of improving the visibility of the display area in the display unit 4.

  In the above-described embodiment, when the menu is not displayed, such as immediately after the information processing apparatus 1 is activated, the user performs some utterance (s110: YES in FIG. 3) to determine the initial position. The current menu is determined with the top menu as a candidate position (s310 to s400 in FIG. 4). Therefore, when the menu is not displayed, the top menu of the first hierarchy can be displayed when the user speaks something.

  In the above embodiment, when no hypothesis information is stored and history information is not stored, such as immediately after the start of voice input, the candidate position has a predetermined initial position. As a result of the setting, the first layer menu corresponding to the initial position is selected as the menu to be displayed (s310 to s400 in FIG. 4).

  That is, even if the current menu is changed based on the sound input from the outside, if the period in which the sound is not input from the outside continues for a predetermined period or longer ("NO" in s260 in the figure), then the current menu Is returned to the menu of the highest hierarchy (first hierarchy), that is, the top menu. Therefore, when the selection item is selected again, the selection item may be started from the top menu.

  As a result, even if the user interrupts or cancels the selection of the selection item, it is only necessary to always select the selection item from the top menu when the selection is resumed, so that confusion in selecting the selection item can be prevented. Convenience as a user interface can be improved.

In the above embodiment, the current menu can be changed by updating the current information stored in the predetermined storage area (s250 in FIG. 4).
Further, in the above embodiment, when it is considered that a selection item to which a predetermined process is assigned is selected from among the selection items in each menu, the assigned process can be executed (s160 in FIG. 3). ).

In the above embodiment, each time voice recognition is performed when estimating which selection item in which selection route corresponds to the user's voice, the similarity is equal to or higher than a threshold value in the recognition. Even if a plurality of types of utterance patterns are recognized, it is possible to estimate a selection route including selection items corresponding to the utterance pattern having the highest similarity among them.
(1-4) Modifications Embodiments of the present invention have been described above. However, the present invention is not limited to the above-described embodiments, and may take various forms as long as they belong to the technical scope of the present invention. Needless to say, you get.

  For example, in the above embodiment, each time the current menu is changed, a message to that effect may be displayed on the display unit 4 or a message or beep sound may be output from the speaker. In this case, it can be notified each time that the current menu is changed.

  Moreover, in the said embodiment, the information processing apparatus of this invention illustrated the structure mounted as an apparatus for implement | achieving the user interface in a navigation apparatus. However, the information processing apparatus of the present invention may be implemented as a device for realizing a user interface in a device other than the navigation device.

  In the above embodiment, when the period in which no sound is input from the outside continues for a predetermined period or longer after the current menu transitions, the current menu is immediately returned to the top-level menu, that is, the top menu. May be.

  For this purpose, if it is determined in s260 in FIG. 4 that there is no voice input through the microphone 5 within a certain past period (s260: NO), the current menu is displayed as shown in FIG. After the transition to the one-level menu (s282), the process proceeds to s280 to decide to delete the menu (FIG. 7 (a)), or the process proceeds to s270 to decide the menu display (FIG. 7 ( b)

  In this way, if the current menu is returned to the top menu of the highest hierarchy, the selection item is always selected from the menu of the first hierarchy when the selection is resumed regardless of the current menu before interruption or cancellation of the selection. As a result, it is possible to prevent confusion when selecting a selection item. As a result, convenience as a user interface can be improved.

  Moreover, in the said embodiment, although what was comprised so that a menu transition was implement | achieved based on the audio | voice input via the microphone 5, as an audio | voice for implement | achieving this menu transition, If there is a route for inputting voice via the network, voice input via this route may be used.

In addition, the transition of the current menu in the above embodiment may be performed when the user receives an operation on the operation unit 3. In this case, even if no voice input is detected within a certain period in the past, the menu is not shifted to the first layer or the menu display is not deleted, or it is different from the transition by voice input. It is desirable to provide a period.
(2) Second Embodiment In this embodiment, only part of the display content determination process is partially different, so only this difference will be described.

This difference is that the first embodiment always displays in the same display mode when displaying the menu by the display unit 4, whereas the present embodiment changes the display mode according to the surrounding environment. It is a point. Specifically, the display mode is configured to differ depending on whether or not the externally input voice is a voice having contents along the selected route.
(2-1) Display content determination process In the display content determination process in this embodiment, after s210 to s250 are performed as in the first embodiment, as shown in FIG. On the basis of the input voice, the reliability that the voice is the voice having the content along the above-described selected path is specified (s251). Here, based on the voice input through the microphone 5 up to that point and stored in the internal memory or RAM of the CPU, a competitive model is prepared as in Patent Document 3 described above, and voice recognition processing is performed. The reliability is specified by calculating the likelihood ratio between the similarity (likelihood) of the hypothesis information described above and the similarity (likelihood) of the hypothesis at the current time of the competitive model.

  The specification of the reliability may be performed not in s251 but in another process independent of the display content determination process. In this case, in s251, the reliability specified in this way is changed to another. What is necessary is just to acquire from a process.

  If the reliability thus specified is larger than the predetermined first threshold value TH1 (s252: YES), after determining that the menu should be displayed with the normal display size (s253), the process accepts the instruction. The process returns (shifts to s130).

  The “first threshold value TH1” serving as the determination criterion in s252 described above is satisfactory because it is determined that the sound input through the microphone 5 up to that point in time is the sound along the selected route. This is a value determined as the degree of reliability. Then, “indicating that the menu should be displayed with the normal display size” in s253 means that the display area when displaying the menu on the display unit 4 should be the same display area as in the first embodiment. means.

After returning to the instruction receiving process in this way, the menu is displayed in the normal display size in s130 of FIG.
If the reliability specified in s251 is larger than the second threshold TH2 determined as a value smaller than the first threshold TH1 (s252: NO, s254: YES), the display size is smaller than normal. After the determination that the menu should be displayed is made (S255), the process returns to the instruction receiving process (shifts to s130).

  The “second threshold TH2”, which is the determination criterion in s254 described above, is sufficient to determine that the sound input through the microphone 5 up to that point is the sound having the content along the selected route. It is a value determined as a non-reliability. In addition, “the menu should be displayed with a display size smaller than normal” in s255 means that the display area when the menu is displayed on the display unit 4 is a display area smaller than the display area in the first embodiment. Means that

After returning to the instruction receiving process in this way, a menu with a display size smaller than normal is displayed in s130 of FIG.
If the reliability specified in s251 is equal to or lower than the second threshold value TH2 (s254: NO), the process proceeds to s280, and after the decision to delete the menu display is made, the process Returns to the instruction reception process (shifts to s130).
(2-2) Actions and Effects According to the information processing apparatus 1 configured as described above, the following actions and effects are obtained in addition to the actions and effects obtained from the same configuration as the first embodiment. Can do.

  For example, in the above-described embodiment, the display mode of the image indicating the menu can be varied according to the reliability that the voice input from the outside is the voice having the content along the above-described selection path. Specifically, the higher the reliability of the input voice from the outside, the larger the display area in the menu.

In this embodiment, the display size is configured to be different as the display mode. However, the display mode to be changed in this way may be a mode other than the display size.
(3) Third Embodiment In this embodiment, only part of the display content determination process is partially different, so only this difference will be described.

This difference is that the first embodiment always displays in the same display mode when displaying the menu by the display unit 4, whereas the present embodiment changes the display mode according to the surrounding environment. It is a point. Specifically, the display mode is configured to differ depending on whether or not a voice is input to a predetermined device 7 (see FIG. 1) that operates by inputting a voice by the user. The predetermined device 7 is, for example, an information terminal (more specifically, a mobile phone terminal) connected to the information processing device 1 so as to be communicable.
(3-1) Display Content Determination Processing In the display content determination processing in the present embodiment, after s210 to s250 are performed as in the first embodiment, communication with a predetermined device 7 is performed as shown in FIG. Then, it is checked whether or not the predetermined device 7 is operating upon receiving voice input (s410).

  If it is determined in s410 that the predetermined device 7 is operating by receiving voice input (s410: YES), the process proceeds to s280, and a determination is made that the menu display should be deleted. After that, the process returns to the instruction receiving process (shifts to s130).

On the other hand, if it is determined in s410 that the predetermined device 7 is not operating in response to voice input (s410: NO), the process moves to s270 and a determination is made that the menu should be displayed. After that, the process returns to the instruction receiving process (shifts to s130).
(3-2) Actions and Effects According to the information processing apparatus 1 configured as described above, the following actions and effects are obtained in addition to the actions and effects obtained from the same configuration as the first embodiment. Can do.

  For example, in the above-described embodiment, when the predetermined device 7 that operates by inputting the voice of the user is operating by receiving the voice input, the menu can be prevented from being displayed.

  As described above, the fact that the voice input to the predetermined device 7 is performed is highly likely that the utterance is performed regardless of the voice input to the information processing apparatus 1, and such an irrelevant voice is input. If the subsequent processing is performed, menu transitions that are not intended by the user are performed.

Therefore, as described above, when the predetermined device 7 is operated by receiving voice input, the menu is not displayed, so that the menu transition not intended by the user is not performed. it can.
(4) Fourth Embodiment In this embodiment, only part of the processing content of the display content determination processing is different, so only this difference will be described.

This difference is that the first embodiment always displays in the same display mode when displaying the menu by the display unit 4, whereas the present embodiment changes the display mode according to the surrounding environment. It is a point. Specifically, the display mode is configured to differ depending on whether an operation is performed on the operation unit 3 or the predetermined device 7 connected to the information processing device 1.
(4-1) Display Content Determination Process In the display content determination process in the present embodiment, after s210 to s250 are performed as in the first embodiment, as shown in FIG. It is checked whether or not an operation is being performed in the predetermined device 7 connected to (S420).

  If it is determined in s420 that the operation is being performed on the operation unit 3 or the predetermined device 7 connected to the information processing apparatus 1 (s420: YES), the process proceeds to s280, and the menu After the determination that the display should be deleted is made, the process returns to the instruction receiving process (shifts to s130).

On the other hand, when it is determined in s420 that the operation is not performed in the operation device 3 or the predetermined device 7 connected to the information processing device 1 (s420: NO), the process proceeds to s270, After the determination that the menu should be displayed is made, the process returns to the instruction receiving process (shifts to s130).
(4-2) Actions and Effects According to the information processing apparatus 1 configured as described above, the following actions and effects are obtained in addition to the actions and effects obtained from the same configuration as the first embodiment. Can do.

  For example, in the above-described embodiment, when the operation unit of the information processing device 1 or the predetermined device 7 connected to the information processing device 1 is operated, the menu can be prevented from being displayed.

  As described above, when the operation unit 3 of the information processing device 1 or the predetermined device 7 connected to the information processing device 1 is operated, the utterance is performed regardless of the menu selection of the information processing device 1. If such an irrelevant voice is input and the subsequent processing is performed, a menu transition unintended by the user is performed.

Therefore, as described above, the menu transition not intended by the user can be prevented from being performed by preventing the menu transition when the operation unit 3 is operated.
(5) Fifth Embodiment In this embodiment, only part of the processing content of the display content determination processing is partially different, so only this difference will be described.

This difference is that the first embodiment always displays in the same display mode when displaying the menu by the display unit 4, whereas the present embodiment changes the display mode according to the surrounding environment. It is a point. Specifically, the display mode is configured to be different depending on the number of users located around the information processing apparatus 1.
(5-1) Display Content Determination Processing In the display content determination processing in this embodiment, s210 to s260 are performed as in the first embodiment, and after “YES” is determined in s260, as shown in FIG. Next, the number of users located around the information processing apparatus 1 is checked (s430).

  In this embodiment, in order to detect the number of users located in the vicinity of the information processing apparatus 1, sensors are arranged in the vicinity of the area where the user can be located. Therefore, in s430, those detection results Detect the number of users based on Here, the number of users may be detected by photographing a region where the user can be located in the vicinity with a camera and specifying the users included in the video by image processing.

  If the number of users checked in this way is “1” (s430: YES), after determining that the menu should be displayed with the normal display size (s440), the process returns to the instruction receiving process. (Transition to s130).

Also, if the number of users checked in s430 is plural (s430: NO), after determining that the menu should be displayed with a display size smaller than normal (s450), the process instructs The process returns to the reception process (shifts to s130).
(5-2) Actions and Effects According to the information processing apparatus 1 configured as described above, the following actions and effects are obtained in addition to the actions and effects obtained from the same configuration as the first embodiment. Can do.

  For example, in the above embodiment, when it is detected that only one user is located around the information processing apparatus 1 (s430 “YES” in FIG. 11), the current menu display area is set to the normal display area. If it is detected that a plurality of users are located (s430 “NO” in the figure), the current menu display area is set to a display area smaller than usual. (S450 in the figure).

  As described above, when a plurality of users are located in the vicinity, a menu displayed on the display unit 4 is not necessarily necessary information from a user other than operating the information processing apparatus 1 by voice. Therefore, by reducing the menu display area in such a case as in the above configuration, the visibility of the display unit 4 for other users is reduced compared to a configuration in which such a display mode is not changed. Can be prevented.

In this embodiment, the display size is configured to be different as the display mode. However, the display mode to be changed in this way may be a mode other than the display size.
(6) Correspondence with the Present Invention In the embodiment described above, s310 in FIG. 8 is the trust specifying means in the present invention, and s253 and s255 in FIG. 8 are the first mode determining means in the present invention. S440 and s450 are the second mode determining means in the present invention, s410 in FIG. 9 is the external voice input determining means in the present invention, s420 in FIG. 10 is the operation detecting means in the present invention, and FIG. s430 is user detection means in the present invention, and s160 in FIG. 3 is processing execution means in the present invention.

Block diagram showing the overall configuration of the information processing apparatus The figure which shows a mode that the menu displayed on a display part changes Flow chart showing instruction acceptance processing Flow chart showing display content determination processing The figure which shows the process of estimating the selection item which the user selected Diagram showing the structure of the utterance pattern dictionary The flowchart which shows the display content determination process in another embodiment The flowchart which shows the display content determination process in 2nd Embodiment. The flowchart which shows the display content determination process in 3rd Embodiment. The flowchart which shows the display content determination process in 4th Embodiment The flowchart which shows the display content determination process in 5th Embodiment

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Information processing apparatus, 2 ... Memory | storage part, 3 ... Operation part, 4 ... Display part, 5 ... Microphone, 6 ... Audio | voice input part, 7 ... Predetermined apparatus, 10 ... Control part, 20 ... Input / output interface, 31 ... Audio | voice Detection means, 33 ... item estimation means, 35 ... voice recognition means, 37 ... processing execution means, 38 ... menu transition means, 39 ... menu display means.

Claims (15)

By causing the user to select each selection item included in each of a plurality of menus having a hierarchical structure from the first layer to the n-th layer (n is an arbitrary number), the menu having the selection item is changed to a menu in another layer. An information processing apparatus implemented with a user interface configured to transition,
Menu display means for displaying a current menu, which is a menu to be displayed at that time among the plurality of menus, on the display unit;
During the continuous sound from outside is input, respectively the speech consists of choices that can be selected to transition from a menu of the i-th layer (1 ≦ i <n) to the menu of the n hierarchy Item estimation means for repeatedly estimating which selection item corresponds to which selection route among the selected routes;
Menu transition means for transitioning the current menu to a menu of another hierarchy to be displayed when the selection item is selected each time a selection item is estimated by the item estimation means ,
The menu display means causes the display unit to display an image indicating the menu transitioned as the current menu by the menu transition means,
further,
The speech continuously input from the outside until that time is compared with each utterance pattern in the utterance pattern dictionary storing each utterance pattern when each of the selected routes is uttered, and the degree of similarity as a comparison result is predetermined. Voice recognition means for outputting the utterance pattern exceeding the threshold of
The information processing apparatus according to claim 1, wherein the item estimation unit outputs the latest selection item as an estimation result among the selection routes corresponding to the utterance patterns sequentially recognized by the voice recognition unit . The menu display means deletes the display of the menu by the display unit when an external audio input has not been made for a predetermined period or longer after the menu image is displayed on the display unit. The information processing apparatus according to claim 1. The menu display means displays the current menu on the display unit when a voice is input from outside in a state where the menu is not displayed on the display unit. The information processing apparatus described in 1. The said menu transition means changes the said current menu to the menu of a 1st hierarchy, when the input of the audio | voice from the outside has not been made more than predetermined period, The menu of any one of Claim 1 to 3 characterized by the above-mentioned. Information processing device. The item estimation means, wherein each time by recognition speech recognition means, when the speech pattern becomes the threshold or more similarity when the recognition has a plurality of type recognition, that corresponds to a high speech patterns most similarity election The information processing apparatus according to claim 4, wherein the latest selection item in the selected route is output as an estimation result for the selected route. The menu transition means transitions the menu by updating the current information indicating the current menu to one indicating a menu of a different hierarchy to be transitioned based on the selection item estimated by the item estimation means. the information processing apparatus according to any one of claims 1-5, wherein. Trust specifying means for specifying the reliability that the voice is a voice of the content along the selected route based on the voice inputted from the outside;
First mode determining means for determining the display mode of the menu according to the reliability specified by the trust specifying means after the current menu is changed by the menu transition means;
It said menu display means, the menu was allowed to transition as the current menu, from claim 1, characterized in that on the display unit at a display mode for the menu the first mode determination means has determined 6 The information processing apparatus according to any one of the above. The first aspect determining means determines the size in the display area of the current menu according to the reliability specified by the trust specifying means,
Said menu display means, according to claim 7, characterized in that on the display unit a menu that is allowed to transition as the current menu at the combined size of the display area determined by the first mode determination means The information processing apparatus described in 1. A second mode determining unit that receives a command from the outside and determines a display mode of the menu after the current menu is transitioned by the menu transition unit;
It said menu display means, the menu was allowed to transition as the current menu, from claim 1, characterized in that on the display unit at a display mode for the menu the second mode determination means has determined 6 The information processing apparatus according to any one of the above. External voice input determination means for determining whether or not the predetermined device is operating by receiving voice input through communication with an external predetermined device (external device) that operates by inputting voice by the user; And
The second mode determining means determines that the current menu should be displayed when the external audio input determining means determines that no audio input is performed on the external apparatus side, while the external apparatus side 10. The information processing apparatus according to claim 9 , wherein when it is determined that voice input is performed, the current menu is not displayed. An operation detecting means for detecting whether an operation unit of the information processing apparatus or a predetermined apparatus connected to the information processing apparatus is operated by a user;
The second aspect determination means determines that the current menu should be displayed when it is determined that the operation detection means is not operated, and the operation detection means is operated. If it is determined, information processing apparatus according to claim 9 or claim 1 0, characterized in that to determine the effect that does not display the current menu. User detection means for detecting the number of users located around the information processing apparatus,
The second aspect determining means determines the size of the display area of the current menu to a normal size when the user detecting means detects that only one user is located, If it is detected that a plurality of users are located, according to any of claims 9 1 1, characterized in that to determine the size of the display area of the current menu to be smaller than normal Information processing device. The information processing apparatus according to claim 1 1 2, characterized in that it comprises a processing execution means, for implementing a predetermined process assigned to the estimated selected item by the item estimation means. By causing the user to select each selection item included in each of a plurality of menus having a hierarchical structure from the first layer to the n-th layer (n is an arbitrary number), the menu having the selection item is changed to a menu in another layer. A user interface providing method for providing a user interface configured to transition, comprising:
A menu display procedure for displaying the current menu, which is a menu to be displayed at that time, among the plurality of menus on the display unit,
During the externally continuous speech is inputted, the voice menu with the selection of each selection item may be selected to transition to the menu of the n layer of the i-th layer (1 ≦ i <n) An item estimation procedure for repeatedly estimating which selection item corresponds to which selection route among the routes;
A menu transition procedure for transitioning the current menu to a menu of another hierarchy to be displayed when the selected item is selected , each time the selected item is estimated by the item estimating means ;
The speech continuously input from the outside until that time is compared with each utterance pattern in the utterance pattern dictionary storing each utterance pattern when each of the selected routes is uttered, and the degree of similarity as a comparison result is predetermined. A speech recognition procedure for outputting an utterance pattern that exceeds the threshold of
In the menu display procedure, an image showing the menu transitioned in the menu transition procedure is displayed on the display unit ,
In the item estimation procedure, the latest selection item in the selected route corresponding to the utterance pattern sequentially recognized by the voice recognition procedure is output as an estimation result . Program for executing various processing procedures for functioning as all the means according to any of claims 1 1 3 of the computer system.